Quantitative evaluation of digital-image enhancement during heads-up surgery

Image-processing is an advantage of heads-up surgery and expected to facilitate ophthalmic surgeries. To evaluate image-processing quantitatively, we analyzed the surgical images of twenty eyes that underwent vitrectomy with internal limiting membrane (ILM) peeling assisted by Brilliant Blue G (BBG). Still images of the peeling procedure were obtained from the surgical video, and the color difference was calculated between two adjacent spots inside and outside the ILM-peeling contour, i.e., without and with BBG staining, respectively. The color differences were compared between the two settings with and without image-processing, delivered by an algorithm to enhance the color and contrast. Color differences were calculated using two methods: the Euclidean distance based on RGB values (RGB distance) and the Delta-E00 formula provided by the International Commission on Illumination. In five cases, minimum light intensities required to recognize the contour of ILM-peeling were compared during surgeries between the two settings with and without enhancement. Image-processing increased the mean color difference significantly (P < 0.001) from 15.47 and 4.49 to 34.03 and 8.00, respectively, for the RGB distance and Delta-E00. The minimum light intensity was reduced from 15 to 5 on average by image-enhancement. These results showed image-processing enhances color differences and reduces light intensities during vitrectomy.

patients and observations. The surgical videos of the vitrectomies performed in 20 eyes of 20 patients (mean age, 64.4 ± 8.1 years) were collected. Validity of the number of the included cases was confirmed using the statistical power analysis based on the parameters shown in results and Table 1 (required sample size: 10, 11, 18, 9 and 10 for R, G, B, RGB distance and delta E00, respectively, to reach the statistical power 0.8). The surgeries were performed to treat epiretinal membranes (ERMs) (11 cases); foveal schisis (3 cases); and macular holes, rhegmatogenous retinal detachments (RRDs), and optic pit maculopathy (2 cases each). In all cases, the ILMs were peeled smoothly. RRDs were macula-sparing in both cases, and there was no abnormal finding such as an ERM in the macula; ILM was peeled to prevent post-surgical ERM growth 11 . One case with RRD was pseudo phakic. The other cases were phakic and were treated by the triple procedure combined with cataract surgery. Consequently, the lens status during ILM peeling procedure was pseudo phakic in one case and aphakic in 19 cases, as IOLs were implanted at the end of the surgery in these cases. Endo-illumination was provided by a xenon light source (Constellation Illuminator, Alcon Laboratories, Hünenberg, Switzerland) with a handheld light (Straight Endoilluminator 25 + , Alcon Laboratories, Fort Worth, TX) during the ILM peeling procedure.
All surgeries were performed as HUS procedures and viewed on a 3-D 4 K monitor for medical use (LMD-X550MT, Sony Corporation, Tokyo, Japan), which was connected to the surgical microscope (OMS-800, Topcon Corporation, Tokyo, Japan) via two CCD cameras (MKC-300HD, Ikegami Tsushinki Co., Ltd., Tokyo, Japan) attached to the beam splitter built-in the microscope. The splitting ratio was 20% to the cameras and 80% to the eye pieces. The auto-iris function was switched on, and the automatic gain control mode was off during the surgeries. An intrinsic image-processing algorithm, i.e., Advanced Image Multiple Enhancer (A.I.M.E., Sony Corporation), was activated intraoperatively for simultaneous image enhancement. The two-dimensional surgical view was recorded via a camera on the right side, because the monitor was not equipped with a recording device. Therefore, the recorded images were the original data without image processing by the A.I.M.E. The quality of images observed by each eye was considered equivalent to each other, because the optical properties of the barrels on both sides of the microscope were equal and the images from both sides were enhanced equally by A.I.M.E. algorithm, as certified by the manufacturer.
For analyses, a still image of the macula during ILM peeling was obtained as a snapshot from the recorded video (A.I.M.E.-off) (Fig. 1a). To assess the enhancement effect, the obtained image was processed by the A.I.M.E. algorithm for color and contrast (A.I.M.E.-on) (Fig. 1b). On each image, two adjacent spots were selected that were inside and outside of the contour of the circular area of ILM peeling, i.e., without and with BBG staining, respectively (Fig. 1a,b), and the color difference between these spots was calculated by 8-bit conversion. Those two spots were fixed at the same coordinates on both the paired A. Assessment of the color differences. Two methods were used to evaluate the color differences, i.e., the Euclidean distance of the two spots in the color space identified with the RGB values (RGB distance) and the Delta-E00 formula 12,13 . The RGB distance was defined as: where RGB 1 and 2 represent the values of each of the two spots to be compared. The International Commission of Illumination provided and recommended the Delta-E00 formula to assess the color differences most recently; the formula was reported to be more effective than other previous formulas for detecting large, small, or threshold color differences 14 . This formula is based on the CIELAB color space, which is defined by luminosity and color opponents (green-red and blue-yellow), and includes correction of the Euclidean distance between two color points based on the characteristics of human perception of luminosity, hue, chroma, and interactive  www.nature.com/scientificreports www.nature.com/scientificreports/ terms between the hue and chroma differences 12,13 . The Delta-E00 formula is expressed as follows, with several equations to calculate each factor that appears in the formula (not presented here; see references for details) 12,13 .
k L , k C and k H : parametric weighting factorsL, C and H: luminosity, chroma and hue, respectively The RGB values and coordinates of the analyzed spots were specified on the viewer software, and the values were translated into the luminosity and color opponents for the serial calculations of Delta-E00, using parameters based on BT.709 for high-definition television (defined by the International Telecommunication Union-Radiocommunication Sector) 15 , after inverse gamma correction (gamma = 2.2) and the definition of the white point values by the standard light source D65.
These two methods were applied independently to calculations of the color difference between two spots with and without BBG staining, and the color differences were compared between the A.I.M.E.-on and A.I.M.E.-off images using the two-tailed t-test. The effects of the underlying diseases on the color differences also were analyzed regarding either of the two methods using analysis of variance. Those analyses were performed using IBM-SPSS Statistics, Version 24.0 (IBM Corp., Armonk, NY). P < 0.05 was considered statistically significant.  A still image without image processing is obtained from the surgical video (a) and enhanced for color and contrast using the image-processing software built into the three-dimensional display used for heads-up surgery (b). The blue points indicate the two adjacent spots inside and outside of the ILM peeling contour, i.e., without and with BBG staining, respectively. These two spots are at exactly the same coordinates in both images. The color differences caused by BBG staining are calculated between the two spots and compared between images before (a) and after enhancement (b).

Results
All patients underwent surgeries performed on the 3-D monitor with the A.I.M.E. algorithm switched on. The manufacturer reported that the delay caused by the image processing was shorter than 30 milliseconds (a reference value provided by the manufacturer; no published data available), and, therefore, the surgeons did not notice any delay during the surgical procedures. Since the A.I.M.E.-on image (Fig. 1b) was extremely bright compared to the original A.I.M.E.-off image (Fig. 1a), the light intensity was set at 30% to 35% in all cases, which was 10% to 15% lower than our usual setting for conventional eye-piece observation, although only 20% of the total light beams were delivered to the display by the beam splitter in our system. Figure 2a,b (RGB distance and Delta-E00, respectively) showed that the color differences identified by both calculation methods clearly increased with enhancement in all cases. Table 1 Table 2).
The minimum light intensity for discrimination of the BBG-stained ILM is listed in Table 3. A.I.M.E. enhancement reduced the requirement of the light intensity to approximately half of that in A.I.M.E.-off condition. Figure 4 shows images of ILM peeling in Case 2 of Table 3 observed under the light intensity varying from 5 to 20. When A.I.M.E. was on (right), border of ILM-peeled area was observed clearly with the light intensity at 10, while light intensity 20 was necessary to distinguish the contour when A.I.M.E. was off (left).

Discussion
The effects of digital image enhancement for improving 3-D optical qualities have been reported based on the subjective impressions of the surgeons regarding resolution, color, stereopsis, illumination, and magnification 8,16 . A quantitative comparison between the digitally enhanced 3-D images and traditional microscopic observations also was performed, and the brightness of the surgical images increased as a result of digital image-processing during HUS 1 . In that report, Eckardt and Paulo described the benefits of electronic amplification for brightening the images and recognition of retinal structures that could not be seen previously 1 . Other investigators also have reported significant reductions in the minimal light intensity required for surgical maneuvers during vitrectomy using the HUS platform 2,3,7,8 .
The enhancement effect on color and contrast is another advantage of digital image processing 1 and should contribute to better visualization of the surgical view and lower light intensity; however, objective and quantitative evaluations have not been performed previously. The current study was designed to analyze the effects of digital enhancement, which was delivered automatically and simultaneously by an image-processing algorithm built into the commercially available 3-D display for medical use. We analyzed the color differences between two spots on the retina with and without ILM staining with BBG, because ILM peeling is one of the finest maneuvers in ophthalmic surgeries and appropriate visualization of the ILM is critical to evaluate vitrectomy performed on the HUS platform. Consequently, the color differences caused by BBG staining of the ILM increased significantly with the A.I.M.E. algorithm switched on. These results were consistent with the observation by surgeons that has been revealed in previous studies, and could be explained by the functional property of the image processing algorithm. It is noteworthy that use of the A.I.M.E. algorithm also resulted in a substantial improvement in brightness, as shown by each of the RGB values with/without BBG staining, that was almost doubled when the A.I.M.E. was on (Table 1). This effect was also proved by significant reduction of the minimum light intensity required to recognize the BBG-stained ILM.
Image processing using the A.I.M.E. algorithm is based on two factors defined as color and structure, with the settings varying among eight and four steps, respectively. Because the enhanced images observed intraoperatively could not be recorded directly on the monitor, we prepared the still images obtained from the recorded video on a computer using the A.I.M.E. algorithm under the setting Color 5, Structure 1, which was similar to the actual setting during our surgeries. The structural enhancement was not raised to 2 or more to avoid increased noise and decreased resolution. These two features are subject to image enhancement and difficult to eliminate totally 1 ; however, in our observation, neither was considered to cause an annoying reduction of image quality during surgery with our setting. In this respect, the adjustment scales of color and structure in the current system would be helpful for each surgeon to customize the image processing to the most comfortable level for various surgical procedures.
To assess the color differences, we used two distinct methods. The balance of the RGB color is calculated simply based on the RGB values measured physically on the viewer software, whereas those values do not correctly reflect the human visual perception, which is modified in a complicated manner with tremendous influences of brightness, hue, and chroma 12,13 . The Delta-E00 formula currently is considered the most reliable metric of color differences that suit human perception 14 . Therefore, the current results indicate the superiority of image enhancement with either of the two methods, including (Delta-E00) and excluding (RGB distance) the modification factors of the colors by the human perception.
The limitations of the current study were the absence of a comparison with other systems and absence of evaluation other than the effects of BBG staining of the ILM. A comparison with other systems could not be undertaken because of the limited equipment in our institute. Although structures other than the ILM should be evaluated in future studies to confirm the efficacy of image processing during HUS, the enhancement effect also could facilitate visualization of other structures, because adjustment scales are available for the color and structure in the current system to fit each aspect of the surgical procedures and the characteristics of the light source.
In conclusion, the current results strongly support the advantages of HUS vitreoretinal surgery regarding enhanced color, contrast, and brightness. Because our system differs from those described previously [1][2][3][4][5][6][7][8][9]16 , the same performance might not be duplicated exactly by other systems; however, any system with image-processing software is expected to improve the surgical images and help surgeons achieve better visualization.