A Method Based on Artificial Intelligence To Fully Automatize The Evaluation of Bovine Blastocyst Images

Morphological analysis is the standard method of assessing embryo quality; however, its inherent subjectivity tends to generate discrepancies among evaluators. Using genetic algorithms and artificial neural networks (ANNs), we developed a new method for embryo analysis that is more robust and reliable than standard methods. Bovine blastocysts produced in vitro were classified as grade 1 (excellent or good), 2 (fair), or 3 (poor) by three experienced embryologists according to the International Embryo Technology Society (IETS) standard. The images (n = 482) were subjected to automatic feature extraction, and the results were used as input for a supervised learning process. One part of the dataset (15%) was used for a blind test posterior to the fitting, for which the system had an accuracy of 76.4%. Interestingly, when the same embryologists evaluated a sub-sample (10%) of the dataset, there was only 54.0% agreement with the standard (mode for grades). However, when using the ANN to assess this sub-sample, there was 87.5% agreement with the modal values obtained by the evaluators. The presented methodology is covered by National Institute of Industrial Property (INPI) and World Intellectual Property Organization (WIPO) patents and is currently undergoing a commercial evaluation of its feasibility.


Image standardization
For standardization, the software followed the steps below: 1. Image importing: The image was imported into the MATLAB ® platform, regardless of image format (BMP or JPG); 2. Conversion to greyscale: Colour information was not used in the interpretation of images and thus all images were converted to 8-bit greyscale. This process also rendered all subsequent steps faster since a decrease of the spectral image size occurs (from the three RGB bands to only one); 3. Resolution and proportion adjustment: In addition to different resolutions, the images also had different proportions. An image of 640x480 pixels has a proportion of 0.75 while for an image of 1280x1024 pixels the proportion is 0.8. Therefore, to only resize the images for one of the resolutions would distort the proportions of the other. We choose 640x480 pixels as default resolution, because it is the lowest standard and provides sufficient information for interpretation.
Using the lowest resolution also increases the efficiency of the subsequent steps of the algorithm; besides it does not create minimum resolution limitations for the final software. Therefore, the 1280x1024 resolution was adjusted for ratio of 0.75 (by cutting an upper region of 64 pixels), and then resizing to 640x480 pixels. The proportions can be visualized in Figure S3; 4. Intensity adjustment: To minimize the effects of different illuminations, the images were submitted to a histogram adjustment, in which 1% of all information becomes saturated between light and dark pixels. This process also increased the image contrast, which facilitates the subsequent step of segmentation (Fig. S4).

Image segmentation
The following steps were used in the segmentation algorithm.
Image gradient: Initially we calculated the magnitude of the image gradient, and the edges were highlighted for subsequent steps. This operation detected edges in all directions, essential characteristic for the circular shape of the embryo. The greater the intensity of the pixels variation, the greater the resulting magnitude value in the final gradient; Binary image: In this step we calculated the binary image and selected the value of 128, as the intensity threshold (because it was an 8-bit image with 255 as the maximum intensity value for each pixel; Fig. S6); Circular Hough Transform: The Hough Transform was originally used to detect straight lines, but adjustments allowed it to be used for the detection of any definite shape. In this work we used the Circle Hough Transform (CHT), which became widespread in circles detection processes in digital imaging 1 . Once the binary image was obtained, the Hough transform detects the embryo circumference by mapping the image and thus provides the isolated embryo background image. The algorithm performed the detection of circles in two stages: in the first stage, it searches by circles of radius between 100 and 150 pixels, then, it searches radius between 150 and 250 pixels for greater accuracy. These values were obtained after those steps by algorithm verification using the entire database. Therefore, both initial blastocysts (smaller) and expanded blastocysts (larger) can be correctly detected. At the end of both searches, in each image, the detected circles metrics are compared and the largest radius is used after the best circle is detected (Fig. S7); Blastocyst isolation: After circumference detection, representing the blastocyst, we used this next step to isolate it. Three versions of the blastocyst image were generated. In the first, the radius of the circumference is increased by 5 pixels to make sure the zona pellucida is included (called ER); in the second, the radius was reduced by 40 pixels, in order to discard the trophectoderm, selecting the inner cell mass (ICM) and blastocoel for analysis only (called RR); and in the third version we obtained the difference between ER and RR. Thereby, only the trophectoderm region was isolated in the image (called TE; Fig. S8). The pixel values, which determined the expansion (ER) and the contraction (RR) of the blastocysts images, were obtained by assessment of the image database.
After image standardization and segmentation, the 36 variables described below were extracted.
They determined the input vector for the ANN. The notation ER was used to refer to the blastocyst image version with expanded radius by 5 pixels, RR to refer the reduced radius by 40 pixels and TE to refer the difference between the two radii.
1) Contrast RR GLCM determined variable. Contrast is the measurement of the intensity difference between a pixel and its neighbours across the entire image with a constant image of zero contrast. This is calculated by Equation S1 ) Where , and are defined in the texture analysis item.

2) Correlation RR
Demonstrate the correlation between the image pixel in determined neighbour across the entire image. Values -1 or 1 shows a perfect correlated image, negative or positive respectively.

Determined by Equation S2
:

3) Energy RR
Square of sum of the GLCM elements. An energy value equals to 1 correspond to a constant image.

Represented by Equation S3
:

17) Radius
Calculated as half of the width of image ER.

18) Sum
A binary ER image is calculated, using the Otsu algorithm ref. 2 for the threshold detection. Next, the sum of all values from the binary image is calculated. Finally, this value is divided by the total area of the blastocyst.

19) Mean ER
Grey mean intensity of the pixels in ER.

Standard deviation of the grey intensity of the pixels in RR, which is calculated by the Equation S5,
where is the image intensity vector and the number of vector elements.

21) Mean RR
Calculated by the same way of Mean ER, but using the region of RR.

22) Mode RR
Mode value of RR, i.e., the total value of luminous intensity more frequently in RR.

23) Dark RR
Initially, pixels with luminous intensity less or equal to 25, which is 10% of the limit allowed (remembering as it uses 8-bit values, the luminous intensity varies between 0 and 255). Then, this value is divided by the total area of the embryo.

24) Mean Count RR
All the pixels with luminous intensity between 10 pixels below and 10 pixels above than the mean intensity were counted. Then, this value is divided by the embryo total area.

25) Bright RR
Same as Dark RR, but using the pixels counting of intensity higher or equal to 230 (10% lighter of the image).

34) Convex ICM
Area of the smallest convex polygon, which has the largest detected region by the Watershed transform. Thereby, a reliable representation of the ICM real area is intended.

35) Eccen ICM
Eccentricity of the largest region detected by the Watershed transform. The value of zero would indicate a perfect circular area and the value 1 a line segment. Considering an ellipse that has the approximate shape of the largest region detected by the Watershed transform, Eccen ICM is calculated as being the relationship between the ellipse focus distance and the length of its largest axis.

36) Mean ICM
Luminous intensity mean value of the largest region detected by the Watershed transform. Figure S1. Images of embryos where the inner cell mass is positioned perpendicular to (b, d and f) or parallel and into (a, c and e) the focal plane. Three embryos are shown on each column and the same embryo is shown on each row. Figure S2. Image of blastocysts captured in different moments. Note the differences in light between the two images, which requires standardization by image processing. Figure S3. Scales and standard demonstration of the resolution and the proportion adjustment algorithm. Figure S4. Images of a bovine embryo before (above) and after (below) the process of standardization. It is possible to visualize the correspondent histogram in the right of each image.       Figure S12. Interface of the Blasto3Q for a mobile smartphone or a PC.