Abstract
Spectral-domain optical coherence tomography (SDOCT) is the gold standard of imaging the eye in clinics. Penetration depth with such devices is, however, limited and visualization of the choroid, which is essential for diagnosing chorioretinal disease, remains limited. Whereas swept-source OCT (SSOCT) devices allow for visualization of the choroid these instruments are expensive and availability in praxis is limited. We present an artificial intelligence (AI)-based solution to enhance the visualization of the choroid in OCT scans and allow for quantitative measurements of choroidal metrics using generative deep learning (DL). Synthetically enhanced SDOCT B-scans with improved choroidal visibility were generated, leveraging matching images to learn deep anatomical features during the training. Using a single-center tertiary eye care institution cohort comprising a total of 362 SDOCT-SSOCT paired subjects, we trained our model with 150,784 images from 410 healthy, 192 glaucoma, and 133 diabetic retinopathy eyes. An independent external test dataset of 37,376 images from 146 eyes was deployed to assess the authenticity and quality of the synthetically enhanced SDOCT images. Experts’ ability to differentiate real versus synthetic images was poor (47.5% accuracy). Measurements of choroidal thickness, area, volume, and vascularity index, from the reference SSOCT and synthetically enhanced SDOCT, showed high Pearson’s correlations of 0.97 [95% CI: 0.96–0.98], 0.97 [0.95–0.98], 0.95 [0.92–0.98], and 0.87 [0.83–0.91], with intra-class correlation values of 0.99 [0.98–0.99], 0.98 [0.98–0.99], and 0.95 [0.96–0.98], 0.93 [0.91–0.95], respectively. Thus, our DL generative model successfully generated realistic enhanced SDOCT data that is indistinguishable from SSOCT images providing improved visualization of the choroid. This technology enabled accurate measurements of choroidal metrics previously limited by the imaging depth constraints of SDOCT. The findings open new possibilities for utilizing affordable SDOCT devices in studying the choroid in both healthy and pathological conditions.
Similar content being viewed by others
Introduction
The choroid, a vascular layer between the retina and sclera, plays a crucial role in maintaining ocular health1,2. It is associated with various retinochoroidal diseases like diabetic retinopathy (DR)3, age-related macular degeneration (AMD)4, polypoidal choroidal vasculopathy (PCV)5, and others6,7,8. Choroidal assessment metrics have evolved over time, from simple measurements like subfoveal thickness9 to more complex reconstructions like vascularity index10. These metrics serve as objective biomarkers for documenting choroidal status. Imaging the choroid is, however, challenging because of the anatomical location of the layer11.
While ultrasonography methods are limited by poor-resolution scans12, conventional angiography techniques with indocyanine green are limited by their invasive nature and lengthy examination time11. Optical coherence tomography (OCT) is the preferred clinic imaging method for non-invasive eye examination13, with spectral-domain OCT (SDOCT) being widely used in clinics. However, due to a progressive decrease of signal towards the choroidal structures because of scattering effects in the retinal pigment epithelium (RPE) and choriocapillaris layers, the choroidal region and the choroidal scleral interface (CSI) cannot be visualized and defined clearly. Therefore, the quantification of choroidal metrics from conventional SDOCT scans remains limited. Enhanced depth imaging (EDI) technology integrated into some SDOCT devices provides improved depth scans14,15, but only longer wavelengths, as in swept-source OCT (SSOCT), can fully visualize the choroid. SSOCT offers benefits like higher detection efficiency, expanded imaging range, and less sensitivity roll-off with depth16. Despite SSOCT’s potential in studying choroidal structures, its use is constrained to a small minority of specialized centers due to cost, resources, and expertise limitations, ultimately leading to a limited clinical appeal compared to SDOCT devices. This highlights the need to explore alternative solutions to study choroid morphology and vasculature.
The advancements in generative deep learning (DL) techniques may offer a promising avenue to enhance choroidal visualization from conventional SDOCT data. Generative DL models can artificially create new images based on real datasets and have shown potential in various applications within ophthalmology in fundus and OCT imaging17,18,19,20,21,22,23,24,25. However, the use of DL models for qualitative and quantitative assessment of retinal and choroidal biomarkers is still restricted to EDI and SSOCT scans26,27,28,29. Hence, the combination of SDOCT with DL may represent a cost-effective solution bridging the gap between the advantages of SSOCT and the practicality of SDOCT in clinical settings.
The objectives of this study were to (1) enhance the visualization of the choroid from conventional SDOCT scans using a generative DL model, and (2) to assess whether the choroidal metrics extracted from the synthetically enhanced SDOCT data match the measurements derived from the reference SSOCT. The motivation of this work was to improve the quality of deep eye structures from SDOCT to study processes and mechanisms of the choroid which were not assessable before in a clinical setting and opening new possibilities for utilizing affordable SDOCT devices to study the choroid in healthy and pathological conditions.
Results
A generative DL model was developed and evaluated with paired SDOCT-SSOCT volumetric B-scans. After quality control and the SDOCT-SSOCT eyes pairing, a meticulous pre-processing step (Supplementary Fig. 1) was performed to match volume size and field of view between the two devices, align the retina, and accurately register the image pairs (Supplementary Fig. 2). We trained the model with a total of 589 eyes from 362 subjects using SDOCT as the input with poor choroidal visibility and matched the same 589 eyes from SSOCT as a reference for choroidal enhancement. Among the total 150,784 B-scan pairs used during the training process, 84,736 images were from healthy subjects (56.2%, 410 eyes), 39,168 data had glaucoma (26.0%, 192 eyes), and 26,880 had DR (17.8%, 133 eyes), as presented in Table 1. In the test dataset, 146 independent eyes from SDOCT, with strictly no patient overlap with the training dataset, were used to generate synthetically enhanced SDOCT data. Specifically, from a total of previously unseen 37,376 SDOCT B-scans, new 37,376 synthetically enhanced SDOCT images were generated. The same distribution of normal, glaucoma, and DR cases was maintained in the test dataset, including 20,224 images from healthy subjects (79 eyes), 9,984 from glaucoma (39 eyes), and 7,168 from DR (28 eyes) patients (Table 1). The overview of the generative DL approach is presented in Fig. 1.
Synthetically enhanced SDOCT image generation
The SDOCT volumes in the independent test set were input in the trained generative DL model and the corresponding synthetically enhanced SDOCT images were successfully generated. The model results were evaluated for veracity via manual review of each image and 100% of the images appeared realistic to a layperson. Figures 2, 3, 4 illustrate samples of the synthetically enhanced SDOCT images in comparison with the original SDOCT and the reference SDOCT image to highlight the achieved choroidal improvement from SDOCT and choroidal structural similarity with SSOCT. From a qualitative analysis, realistic synthetic data with enhanced choroid were generated for healthy, glaucoma and DR subjects. Supplementary Fig. 3 shows a comparison of enface images from 4 different eyes at different depths to further demonstrate the improvement achieved by our DL-generated images over the input SDOCT data from a different projection.
Experts qualitative assessment
The qualitative plausibility and authenticity of the synthetically enhanced SDOCT images were assessed by three ophthalmologists: the experts, blind from any data information, were tasked to determine whether the examined images resembled real or synthetic SSOCT scans. The expert majority correctly identified 47.5% of images as being synthetic SDOCT images or real SSOCT images, with experts 1–3’ accuracy of 45.0%, 53.0%, and 56.5%, respectively. Images from healthy subjects were correctly classified with an accuracy of 52.1%, while glaucoma and DR showed a lower clinicians’ discriminative performance of 45.5% and 46.4%, respectively. Overall sensitivities and specificities of the experts’ majority were found to be between 44.4% and 54.2%. While the less experienced ophthalmologist (expert 1) achieved the tasks with a performance below 50%, the expert with more years of experience (expert 3) performed the tasks with slightly higher accuracy of 56.5%, sensitivity of 61.2%, and specificity of 51.5%. Detailed results are presented in Table 2 and further task-specific experts’ discriminative performance is described in Supplementary Table 2. Overall, these results suggest that the clinicians could not discern between synthetically enhanced SDOCT and SSOCT.
Retinal quantitative assessment
The retinal thickness was marked from the inner limiting membrane perpendicular to the outer surface of the RPE and computed on SDOCT images and corresponding synthetically enhanced SDOCT images to assess whether the synthetic data effectively preserved the anatomical structure of the retina layers (Fig. 5a). The mean retinal thickness for both SDOCT and synthetically enhanced SDOCT images was 0.22 ± 0.02 mm (minimum of 0.18 mm and maximum of 0.26 mm) with similar values across normal, glaucoma, and DR states (Supplementary Table 3). The statistical agreement between SDOCT and synthetically enhanced SDOCT retinal thickness measurements was represented by a Pearson’s r of 0.95 [95% CI 0.93–0.96] and an intra-class correlation (ICC) score of 0.97 [95% CI 0.96–0.98]. All the correlations were statistically significant with P values < 0.001, showing significant correlations also among individual normal, glaucoma and DR states, with small mean squared errors (MAE) values between 0.003 and 0.005 (Table 3). These results indicate that synthetically enhanced SDOCT images effectively preserved the SDOCT sub-foveal retina layers.
Choroidal quantitative assessment
An overview of the synthetically enhanced quantitative choroidal image evaluation approaches is displayed in Fig. 5b. All the choroidal metrics were performed on SSOCT and corresponding synthetically enhanced SDOCT images to evaluate whether the improved generated visualization of the choroid was in agreement with the reference SSOCT. We considered choroidal thickness (CT), choroidal area (CA), and choroidal volume (CV) as morphological metrics, and the choroidal vascularity index (CVI) as vascularity metric.
The choroidal region was delineated from the outer surface of the RPE to the CSI. The mean CT, CA, and CV for SSOCT and synthetically enhanced SDOCT images were 0.25 ± 0.08 mm and 0.24 ± 0.08 mm, 0.70 ± 0.18 mm2, and 0.71 ± 0.19 mm2, 1.69 ± 0.54 mm3 and 1.65 ± 0.51 mm3, with similar values across normal, glaucoma and DR states. Figure 6 and Supplementary Table 3 illustrate the details of the choroid morphological metrics measurement distributions. Overall, Pearson’s r and ICC scores were 0.97 [95% CI 0.96–0.98] and 0.99 [95% CI 0.98–0.99] for CT, 0.97 [95% CI 0.97–0.98] and 0.98 [95% CI 0.98–0.99] for CA, 0.95 [95% CI 0.92–0.98] and 0.97 [95% CI 0.96–0.98] for CV. All the correlations were statistically significant with P values < 0.001. MAE values were found to be 0.02, 0.04, and 0.10 for CT, CA, and CV, respectively. Overall significant correlations among individual normal, glaucoma and DR states were observed and are depicted in Table 3. Figure 6 further shows the morphological choroidal metrics statistical agreements from the synthetically enhanced SDOCT images with respect to the SSOCT measurements, along with scatter plots and Bland-Altman agreement plots. These results indicate that synthetically enhanced SDOCT images effectively generated clinically plausible synthetic choroids from the original SDOCT with no significant difference from SSOCT in terms of morphological choroidal metrics.
We defined the CVI as the ratio of vascular luminal area (LA) to CA and presented it as a percentage The mean CVI for SSOCT and synthetically enhanced SDOCT images was 0.65 ± 0.08 and 0.66 ± 0.08, with similar values across normal, glaucoma and DR states. Figure 6 and Supplementary Table 3 illustrate the details of the vascularity metric measurements. Overall, CVI Pearson’s r and ICC scores were 0.87 [95% CI 0.83–0.91] and 0.93 [95% CI 0.91–0.95]. All the correlations were statistically significant with P values < 0.001. MAE value for CVI was found to be 0.03, and overall significant correlations also among individual normal, glaucoma and DR states were observed (Table 3). These findings indicate that synthetically enhanced SDOCT images effectively generated plausible synthetic choroidal vasculature from the original SDOCT hidden choroidal vessels, with no significant difference from SSOCT in terms of CVI (Fig. 6).
Discussion
The present work shows that deep structures of the posterior pole of the eye can be visualized in SDOCT images by learning from SSOCT images. This may change the way SDOCT images are used in clinics, particularly in diseases affecting the choroid. Specifically, we demonstrated (1) that the choroid visualization was significantly enhanced, as quantified by the high correlations choroidal metrics values of synthetically enhanced SDOCT data with respect to the measurements from SSOCT, and (2) that a generative DL method applied to SDOCT images can generate synthetically enhanced SDOCT data indiscernible from the reference SSOCT, as determined by the low discriminative accuracy of clinical experts. This technology improved choroidal quantitative assessments in conventional SDOCT used in clinics.
We propose the utilization of a generative DL model based on a GAN architecture30 to enable the majority of clinicians using SDOCT to benefit from enhanced choroidal assessment without requiring expensive and specialized equipment. Synthetically DL-generated images can be manipulated to adjust image quality and increase diversity of datasets, overcoming implications for privacy laws and data sharing31,32. Generative DL models have been recently deployed to reduce image variability across OCT devices, resulting in successful improvement in image quality from time-domain OCT scans to approach that of SDOCT33, and have contributed to reducing differences among SDOCT images derived from different vendors34. The DL architectures developed for these tasks are based on CycleGAN35, a type of GAN that allows learning a suitable translation function between the different image domains without requiring paired samples and using cycle consistency loss to help generate realistic and coherent results. However, these networks consisted of complex ensembles of CycleGANs comprising several discriminator and generator models and considering different spatial windows, eventually requiring steps of data pairing and meticulous pre-processing. Given that our dataset consisted of well-paired OCT data from two different devices, collected from the same patients during the same date of visit, we adopted a Pix2Pix GAN architecture, suitable for tasks where direct correspondences between input and output images are available. Pix2Pix was designed with a conditional GAN architecture allowing for precise control over the mapping between input and output images and is characterized by superior computational simplicity, as well as more stable training dynamics36. Nevertheless, there is a scarcity of studies specifically targeting the translation of SDOCT to SSOCT and the application of generative DL models to enhance choroidal visibility is an area that remained unexplored. Hence, our contribution was crucial to understanding the generative DL potential for cross-modality choroidal enhancement.
We included paired macula scans from normal subjects, as well as glaucoma and DR. Examining the choroid in normal subjects, is essential to establish baseline measurements and understand the physiological variations to recognize subtle changes that might indicate early signs of ocular diseases, allowing for timely intervention and prevention37. In glaucoma, studies examining the macular choroidal circulation are limited with conflicting results38, and allowing better visualization of macular choroidal structures may be crucial to assess the involvement in the pathogenesis of the disease, offering valuable insights into glaucoma progression and aiding in the monitoring of treatment efficacy. In DR, studying the choroid is of paramount importance as alterations in its structure and blood flow have been associated with disease severity and progression: enhanced choroidal imaging can aid in identifying early vascular changes and guide personalized therapeutic strategies to manage DR effectively39.
Through a visual comparison of SDOCT and DL-generated images, evident improvements in choroidal visualization were observed: previously obscured structures due to increased noise levels become clearly visible in the synthetically enhanced SDOCT images from both cross-sectional (Figs. 2, 3, and 4) and enface (Supplementary Fig. 3) projections. Since the CSI was visible in the synthetically enhanced SDOCT image, a visual comparison with SSOCT choroidal area could be performed, revealing a match between the vascular structures in both modalities. To assess the qualitative plausibility and authenticity of the synthetically enhanced SDOCT images, a subset of SSOCT and synthetically enhanced SDOCT data were presented for blind evaluation to three clinicians. The subset comprises 50% of DR data to emphasize the evaluation of images where the disease exhibits more pronounced manifestations. Sensitivities and specificities of clinical expert 3 (senior consultant) were higher in the task involving the grading based on pairs of real-synthetic images in the DR subset (Supplementary Table 2). Generally, clinicians could not accurately discriminate the synthetically enhanced SDOCT images from the real data (Table 2). Therefore, the synthetically generated SSOCT images closely resembled the SSOCT images.
Through a quantitative analysis, we determined that retinal structure was not compromised: with a Pearson’s r of 0.95 and an ICC score of 0.97 with respect to the original SDOCT, we demonstrated that the DL model generated images with effectively preserved retinal thickness (Table 3). Through further quantitative assessment of the synthetically generated SSOCT images, we found that the choroidal metrics values match the high correlation of the choroidal metrics with the SSOCT data. Choroidal metrics are objective biomarkers that can facilitate reproducible and reliable measurements, allowing the documentation of the effects of ageing on the choroid as well as pathological processes and response to pharmacological intervention. Several studies have proposed CT and CA as markers to assess disease conditions37 reporting them as indicators of ocular9,40,41 and systemic health42,43,44,45,46,47. We obtained a strong significant agreement between synthetically enhanced SDOCT and SSOCT measurements from Pearson’s r of 0.97 and ICC up to 0.99: this indicates that the DL model generated images with effectively enhanced choroidal thickness and area (Fig. 6, Table 3). Similar findings were discovered when assessing the choroidal morphology from a volumetric perspective. The DL model used for CV calculation was previously trained on PlexElite scans of non-flattened images: some outliers in the correlation plots were expected given the diversity of our images set (Fig. 6). Nevertheless, excellent agreement performance was reached (Table 3, Fig. 6).
On the other hand, CVI allows vascular analyses of the choroid10,37,48,49 and has been assessed in studies on retinitis pigmentosa50, central serous chorioretinopathy (CSC)51,52, branch retinal vein occlusion53, retinal dystrophies54, and Alzheimer’s disease55. We analyzed CVI to gain further insights beyond choroidal morphology and assess the proportion of vascular LA within the choroid. Our study enabled precise CVI measurements from synthetically enhanced SDOCT data and demonstrated that they exhibited no significant difference compared to measurements from SSOCT. These measurements, which were not attainable from the original SDOCT data, showed a high Pearson’s r of 0.87 and an ICC score of 0.93, highlighting the reliability and consistency of our approach (Table 3, Fig. 6). Additionally, novel automated techniques to enhance the efficiency of CVI estimation, facilitating a faster calculation process yet providing accurate measurements, will be explored56,57.
In this study, we present an approach to translate SDOCT images into choroid-enhanced images using a state-of-the-art DL image synthesis technique, trained on a diverse dataset including normal, glaucoma, and DR cases. While SDOCT data is readily available in clinics, SSOCT scans are not routinely performed and are only available to few specialized referral centers. Our proposed methodology yielded robust and flexible results, as evidenced by the strong statistical agreements of choroidal metrics. Although training a generative DL model may pose computational challenges, this limitation becomes negligible in practice since the algorithm generated synthetically enhanced SDOCT data offline. Our study was constrained by the availability of OCT scans from only two modalities, Cirrus and PlexElite, obtained as paired data in the clinical study. The scarcity of paired data involving different modalities and OCT devices from different manufacturers limits the opportunity to investigate improvements in OCT scans from one modality based on data from another. As a result, investigations focusing on enhancing OCT scans using data from different modalities during the same visit remain uncommon in the existing literature. Strategies for data collection and collaboration between different clinical centers would be crucial to gather an adequate dataset for training and validation. Collaborative efforts would facilitate the creation of a larger and more diverse database, improving the DL model’s accuracy and applicability.
Future work will aim to extend our approach to generate enhanced choroidal data from eyes with various other diseases that impact the choroid, such as CSC, PCV, and retinal vascular diseases. Additionally, we will focus on studying choroidal changes and exploring how our synthetically enhanced SDOCT data can improve the assessment of these changes compared to standard clinical data. By including a broader range of ocular pathologies, we can assess the generalizability and robustness of our synthetic enhancement technique in diverse clinical scenarios. Beyond ophthalmology, our DL approach could enhance OCT image quality across various medical fields, such as dermatology, where high-resolution imaging is pivotal for diagnosing skin conditions. Extending the potential of our image-to-image methodology to diverse medical imaging modalities and applications, including multimodal imaging settings, could contribute to the broader landscape of medical DL research and facilitate advancements in diagnostic imaging and patient care. For instance, in magnetic resonance imaging, where scan quality and contrast are crucial for accurate diagnosis, our framework could generate enhanced images and aid in the detection and characterization of abnormalities from conventional machines operating at low magnetic field strengths.
To ease the requirements for paired data and preserve the fine disease feature details when considering scans from eyes with chorioretinal conditions, the integration of diffusion models58 in the enhanced image generation process will be explored. Furthermore, unsupervised domain adaptation59 recent advances may also offer an opportunity to overcome generalizability limitations when dealing with unseen or out-of-distribution data60. We will focus on the adoption of our approach in real-world scenarios, considering rigorous validation and standardization to enhance the credibility and acceptance of the synthetic data in clinical decision-making, providing user-friendly interfaces and a computationally efficient solution.
Methods
This study included data from normal, glaucoma, and DR participants derived from a clinical study performed at the Singapore Eye Research Institute in Singapore, a single-center tertiary eye care institution. All studies complied with the tenets of the Declaration of Helsinki and were approved by the SingHealth Centralized Institutional Review Board. Written informed consent was obtained from all the participants.
Study participants
Study participants were enrolled from the Singapore Imaging Eye Network, a clinical cross-sectional study, and collected between 2018 and 2021 from Chinese, Malay, and Indian patients aged above 21 years. Supplementary Table 1 describes the demographics of the study participants. Normal eyes were defined as individuals free from clinically relevant eye conditions such as glaucoma/-suspect/self-report glaucoma61,62 and retinopathies63. Glaucomatous eyes were defined as having a pathological optic disc appearance with a corresponding glaucoma hemifield test outside normal limits; primary open-angle glaucoma, primary angle-closure glaucoma, normal tension glaucoma, and ocular hypertension were included62. DR was defined as present if any characteristic lesion as defined by the Early Treatment Diabetic Retinopathy Study severity scale was found64. Patients with co-diagnosis of clinically relevant eye diseases such as retinal diseases and AMD65, or significant media opacity, were excluded from recruitment as they might interfere with the aim of the study. We excluded participants for poor image quality and incomplete or missing data. We finally included a total of 453 patients (253 healthy participants, 122 glaucoma patients, and 78 diabetic retinopathy patients).
OCT imaging—spectral-domain OCT
The OCT imaging was first performed using an SDOCT (Cirrus 5000, Carl Zeiss Meditec, Dublin, California, USA) characterized by a light source wavelength of 840 nm, scanning speed of 68,000 A-scan/s, axial and transverse resolution in tissue of 5 μm and 15 μm respectively. Each SDOCT data volume consisted of 245 A-scans and 245 B-scans with a field of view of 3 mm × 3 mm × 2 mm.
OCT imaging—swept-source OCT
Subsequently, with a time gap in the order of minutes, the second imaging was conducted using a prototype SSOCT system (PlexElite 9000, Zeiss Meditec, Dublin, California, USA) with a central wavelength scanning laser of 1,050 nm, system operation speed of 100,000 to 200,000 A-scan/s, and axial and lateral resolutions in tissue of 6.3 µm and 20 µm respectivelyEach SSOCT data volume consists of 300 A-scans and 300 B-scans covering a 3 mm x 3 mm x 3 mm volume.
OCT data pairing
A review software (Zeiss Meditec, Dublin, California, USA) was used to extract the B-scans. We included in our study paired OCT data acquired by deploying a 3 mm x 3 mm scanning protocol centered at the fovea. The proprietary signal strength extracted from the two OCT devices, documented using a scale from 1 to 10, was utilized as a quality index for preliminary data inclusion/exclusion. Only OCT with signal strength 6 and above were considered according to the manufacturer’s recommendation. Subsequently, one trained ophthalmic technician scanned all the participants and manually reviewed the quality of the scans. Scans were excluded from further analysis if one or more of the following criteria were met: poor clarity images, weak or inconsistent signal intensity across the scan caused by obstacles, and excessive motion artifacts. The SDOCT-SSOCT image pairing was performed at eye level. If the SDOCT and SSOCT scans for both patient eyes met the eligibility criteria above mentioned, both eyes were included in the study; if only one patient eye met the eligibility criteria, only the eligible eye was included in the study.
Retinal thickness
The thickness of the retina was manually measured at the center of the fovea. To determine the foveal pit, we scanned through several B-scans to establish the thinnest retina layer, and the neighboring two B-scans were averaged for the segmentation of the choroidal-scleral boundary66. The subfoveal retinal thickness was marked from the inner limiting membrane perpendicular to the outer surface of the RPE. The thickness, measured with a customized caliper function (MATLAB R2020b, MathWorks, MA, USA) by an OCT expert (V.B.), was presented in a pixel unit and subsequently converted to millimeters via axial digital sampling to match the scanning field of view.
Choroidal thickness
The thickness of the choroid was manually measured at the center of the fovea, determining the foveal pit with the procedure previously described for retinal thickness66. The CT was perpendicularly marked from the outer surface of the RPE to the CSI and manually measured with the same function utilized for retinal thickness by the OCT expert (V.B.). Similarly, CT was converted to millimeters via axial digital sampling.
Choroidal area
The area of the choroid was computed from the central foveal OCT scan of each volume after image brightness and contrast adjustment to allow a more precise selection of CA boundaries, using Fiji software (Image J 1.54b; http://imagej.nih.gov/ij/). To calculate CA, we used the RPE-Bruch’s membrane estimation derived from the pre-processing step and manually delineated the CSI using a polygon tool. The pixel area was converted to millimeters2 via axial and transverse digital sampling to match the scanning field of view. This step was manually performed to ensure precise CA segmentation, as it is directly followed by the CVI estimation.
Choroidal volume
The volume of the choroid was automatically calculated for each eye. The 3D CV measure was performed with a DL model67 which utilizes a U-Net-based architecture68 fused with a multi-task learning approach to segment the choroid from three-dimensional OCT aggregating the spatial context from adjacent cross-sectional slices. The resulting pixel volume was converted to millimeters3 via volumetric digital sampling to match the scanning field of view.
Choroidal vascularity index
The vascularity index was computed using Fiji software after the segmentation of the CA from the central foveal OCT scan of each volume. We defined CVI as the ratio of vascular LA to CA and presented it as a percentage37. To calculate LA, image binarization was performed using Niblack’s auto-local thresholding technique69,70, and a color threshold tool was applied to automatically delineate the LA (dark pixels). Finally, CVI was computed by dividing LA by CA10,37.
OCT images paired pre-processing
After the SDOCT-SSOCT eyes pairing step, the images underwent pre-processing to be deployed in the generative DL model (Fig. 1a). The detailed data pre-processing framework can be found in Supplementary Fig. 1. First, cropping and 3D Lanczos interpolations71,72 were performed to match volume size and field of view between SDOCT and SSOCT scans. Lanczos interpolation technique uses a sinc function as a convolution kernel to achieve high-quality resampling while minimizing aliasing artifacts and preserving fine details. Second, a de-speckling algorithm based on anisotropic diffusion filtering was applied to enhance the quality of the scans and an automated RPE location estimation was used to flatten the retina and align the corresponding OCT pairs. Third, a customized intensity-based enface registration was performed for each SDOCT-SSOCT pair to further fine-align the field of view. In Supplementary Fig. 2 we describe the registration accuracy analysis. A further step of cropping and field of view adjustments was applied to obtain the final dataset. Each OCT eye pair finally consisted of 256 scans per volume and 256×256 pixels images, covering a field of view of 2.6 mm × 2.6 mm × 2 mm.
Generative DL model development
Out of the 735 eyes satisfying the inclusion criteria, we used 589 eyes for the training of the DL model (150,784 images) and 146 independent eyes to test the DL model (37,376 images) and generate the synthetically enhanced SDOCT data. There was strictly no overlap with patients between the train and test set. We used a generative DL model based on Pix2Pix architecture30 for image synthesis to automatically translate SDOCT scans to choroidal-enhanced images based on SSOCT. The model learned deep anatomical features from SSOCT and applied choroid properties to SDOCT images. During the training process, SDOCT and SSOCT images were loaded pairwise into the model. Pix2Pix is a generative adversarial network (GAN) designed for image-to-image translation and requires paired and well-pixel-wise aligned images. Briefly, the DL architecture is comprised of a generator model which takes as input SDOCT images and creates new plausible choroidal-enhanced SDOCT images, and a discriminator model that classifies images as real (SSOCT) or fake (synthetic), to determine whether the synthetic data is acceptable transformation of the SSOCT image (Fig. 1b). The two models are trained simultaneously in an adversarial process where the generator seeks to better fool the discriminator and the discriminator tries to better identify the counterfeit images30. Specifically, the generator used in this work is a Resnet-9 architecture which makes use of residual connections73 and is composed of three encoding blocks, nine residual blocks and three decoding blocks. Each encoding block consists of a convolution, followed by instance norm and Relu layers and each of the residual blocks follows the convolution-InstanceNorm-ReLU-Dropout-convolution-InstanceNorm residual connection structure. The discriminator consists of 5 layers and uses a patch-wise method that only penalizes structure at the scale of patches. While most complex discriminators in GAN architectures utilize the whole image for establishing a synthetic or real (0 or 1) value, our PatchGAN tries to classify if each N × N patch in an image is real or synthetic with the advantage to be applied to arbitrarily large images, utilize fewer training parameters, and run faster. In this work, we use a patch size of 70 \(\times\) 70. Supplementary Fig. 4. shows the diagrams of the generator and discriminator with the details of each block. One single generative DL model was trained for normal, glaucoma, and DR. data. The development of the generative DL model was done in Python (Python Software Foundation, DE, USA) and trained for approximately 24 hours and 15 epochs using a GeForce RTX 2080 Ti GPU; Adam optimizer with a learning rate of −0.0001 was set. After training was completed, an independent SDOCT dataset consisting of the same disease distribution as the training dataset was used to generate the synthetically enhanced SDOCT data (Fig. 1c).
Clinician image evaluation
We evaluated the qualitative plausibility and authenticity of the synthetically enhanced SDOCT images providing a subset of SSOCT and corresponding synthetically enhanced SDOCT data to 3 ophthalmologists for visual evaluation and manual grading (Expert 1: R.J., ophthalmologist resident; Expert 2: J.S., senior staff registrar with >5 years of experience in ophthalmology; Expert 3: G.S.W.T, senior retina consultant with >15 years of experience in ophthalmology). First, from the 146 eye pairs in the independent test set, a total of 100 randomly selected single images (50 SSOCT and 50 synthetically enhanced SDOCT images) were presented to the clinicians. The experts were asked to report whether they believed each image was real or synthetic. Second, a total of 100 randomly selected SSOCT-synthetic image pairs were inspected: the clinical experts were asked to report which of the 2 images in each pair was real (SSOCT). For the two tasks, a total of 30 images from normal eyes, 25 glaucoma, and 50 DR were used. Expert majority predictions and overall task predictions for all images were also calculated. Individual experts’ predictions and the experts’ majority predictions were compared with the ground truth. The images were prepared on a digital grading form and the clinicians were allowed to review the images at a setting and time of their convenience. No prior information regarding the data distribution was given to avoid any bias.
Retinal and choroidal metrics evaluation
Retinal and choroidal metrics were computed for the independent test set data and extracted as mentioned in a previous section. Retinal thickness measurements were manually performed on SDOCT images and corresponding synthetically enhanced SDOCT images to assess whether the synthetic data effectively preserved the anatomical structure of the retina layers; statistical agreement and error between the two sets of measurements were subsequently computed. All the choroidal metrics were performed on SSOCT and corresponding synthetically enhanced SDOCT images to evaluate whether the improved generated visualization of the choroid was in agreement with the reference SSOCT. Statistical agreement and error between the two sets of CT, CA, CV, and CVI measurements were subsequently computed. Figure 5 shows how the retinal and choroidal metrics are extracted respectively from SDOCT, SSOCT and the corresponding synthetic data.
Statistical analysis
Evaluation metrics for clinical experts’ ability to discern between synthetic and real SSOCT images were accuracy, sensitivity, and specificity. We quantified the retinal thickness preservation of the synthetically enhanced SDOCT data over SDOCT images and the choroidal enhancement of the synthetically enhanced SDOCT data over SSOCT images by comparing the agreement of CT, CA, CV, and CVI with scatter plots and Bland-Altman plots. We computed Pearson’s correlation coefficients (r), ICC scores, and MAE values to quantify the agreement with real OCT and synthetically enhanced SDOCT data. Confidence interval values of 95% were generated using bootstrap (5000 iterations) and P values were calculated using F-test with scores less than 0.001 considered statistically significant. All the statistical analysis was done using Python and the scikit-learn library.
Reporting summary
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
Data availability
The de-identified patient data and any data that support the findings of this study may be shared on reasonable request to the corresponding author, subject to approval from the SingHealth Centralised Institutional Review Board.
Code availability
Any custom code used to implement the deep learning model detailed in this paper and all data analysis scripts will be made available upon reasonable request to the corresponding author and establishment of data sharing agreement between institutions.
References
Nickla, D. L. & Wallman, J. The multifunctional choroid. Prog. Retinal Eye Res. 29, 144–168 (2010).
Spaide, R. F. The Choroid. In Pathologic Myopia (eds. Spaide, R. F., Ohno-Matsui, K. & Yannuzzi, L. A.) 139–159 (Springer International Publishing, Cham, 2021).
Tan, B. et al. Combining retinal and choroidal microvascular metrics improves discriminative power for diabetic retinopathy. Br. J. Ophthalmol. 107, 993–999 (2023).
Koh, L. H. L., Agrawal, R., Khandelwal, N., Sai Charan, L. & Chhablani, J. Choroidal vascular changes in age‐related macular degeneration. Acta Ophthalmol. 95, e597–e601 (2017).
Chung, S. E., Kang, S. W., Lee, J. H. & Kim, Y. T. Choroidal thickness in polypoidal choroidal vasculopathy and exudative age-related macular degeneration. Ophthalmology 118, 840–845 (2011).
Kitaya, N. et al. Features of abnormal choroidal circulation in central serous chorioretinopathy. Br. J. Ophthalmol. 87, 709–712 (2003).
Grossniklaus, H. E. & Green, W. R. Choroidal neovascularization. Am. J. Ophthalmol. 137, 496–503 (2004).
Ikuno, Y. & Tano, Y. Retinal and choroidal biometry in highly myopic eyes with spectral-domain optical coherence tomography. Investig. Ophthalmol. Vis. Sci. 50, 3876–3880 (2009).
Gupta, P. et al. Choroidal thickness and high myopia: a case–control study of young C hinese men in Singapore. Acta Ophthalmol. 93, e585–e592 (2015).
Betzler, B. K. et al. Choroidal vascularity index: a step towards software as a medical device. Br. J. Ophthalmol. 106, 149–155 (2022).
Invernizzi, A. et al. Imaging the choroid: from indocyanine green angiography to optical coherence tomography angiography. Asia-Pac. J. Ophthalmol. 9, 335–348 (2020).
Sobottka, B., Schlote, T., Krumpaszky, H. G. & Kreissig, I. Choroidal metastases and choroidal melanomas: comparison of ultrasonographic findings. Br. J. Ophthalmol. 82, 159–161 (1998).
Huang, D. et al. Optical coherence tomography. Science 254, 1178–1181 (1991).
Spaide, R. F., Koizumi, H. & Pozonni, M. C. Enhanced depth imaging spectral-domain optical coherence tomography. Am. J. Ophthalmol. 146, 496–500 (2008).
Margolis, R. & Spaide, R. F. A pilot study of enhanced depth imaging optical coherence tomography of the choroid in normal eyes. Am. J. Ophthalmol. 147, 811–815 (2009).
Drexler W., Fujimoto J. G. Optical coherence tomography: technology and applications. Springer Science & Business Media; (2008).
Costa, P. et al. End-to-end adversarial retinal image synthesis. IEEE Trans. Med. Imaging 37, 781–791 (2017).
Burlina, P. M., Joshi, N., Pacheco, K. D., Liu, T. A. & Bressler, N. M. Assessment of deep generative models for high-resolution synthetic retinal image generation of age-related macular degeneration. JAMA Ophthalmol. 137, 258–264 (2019).
Liu, Y. et al. Prediction of OCT images of short-term response to anti-VEGF treatment for neovascular age-related macular degeneration using generative adversarial network. Br. J. Ophthalmol. 104, 1735–1740 (2020).
Bellemo V., Burlina P., Yong L., Wong T. Y., Ting D. S. W. Generative adversarial networks (GANs) for retinal fundus image synthesis. Springer: 289-302; 2019.
Lee, H., Kim, S., Kim, M. A., Chung, H. & Kim, H. C. Post-treatment prediction of optical coherence tomography using a conditional generative adversarial network in age-related macular degeneration. Retina 41, 572–580 (2021).
Zheng, C. et al. Assessment of generative adversarial networks model for synthetic optical coherence tomography images of retinal disorders. Transl. Vis. Sci. Technol. 9, 29–29 (2020).
Wang, Z. et al. Generative adversarial networks in ophthalmology: What are these and how can they be used? Curr. Opin. Ophthalmol. 32, 459 (2021).
Kumar, A. J. S. et al. Evaluation of generative adversarial networks for high-resolution synthetic image generation of Circumpapillary optical coherence tomography images for glaucoma. JAMA Ophthalmol. 140, 974–981 (2022).
Wong, D. et al. Evaluation of generated synthetic OCT images in deep‐learning models for glaucoma detection. Acta Ophthalmologica 100, https://doi.org/10.1111/j.1755-3768.2022.0131 (2022).
Zheng, G. et al. Deep learning algorithms to segment and quantify the choroidal thickness and vasculature in swept-source optical coherence tomography images. J. Innovat. Opt. Health Sci. 14, 2140002 (2021).
Vupparaboina, K. K., Nizampatnam, S., Chhablani, J., Richhariya, A. & Jana, S. Automated estimation of choroidal thickness distribution and volume based on OCT images of posterior visual section. Comput. Med. Imaging Graph. 46, 315–327 (2015).
Chen M., Wang J., Oguz I., VanderBeek B. L., Gee J. C. Automated segmentation of the choroid in EDI-OCT images with retinal pathology using convolution neural networks. Springer:177-184; 2017
Vupparaboina, K. K. et al. Automated choroid layer segmentation based on wide-field ss-oct images using deep residual encoder-decoder architecture. Investig. Ophthalmol. Vis. Sci. 62, 2162–2162 (2021).
Isola P., Zhu J.-Y., Zhou T. & Efros A. A. Image-to-image translation with conditional adversarial networks. Preprint at https://doi.org/10.48550/arXiv.1611.07004, 1125–1134 (2017).
Chen, J. S. et al. Deepfakes in ophthalmology: Applications and realism of synthetic retinal images from generative adversarial networks. Ophthalmol. Sci. 1, 100079 (2021).
Coyner, A. S. et al. Synthetic medical images for robust, privacy-preserving training of artificial intelligence: Application to retinopathy of prematurity diagnosis. Ophthalmol. Sci. 2, 100126 (2022).
Lazaridis, G. et al. OCT signal enhancement with deep learning. Ophthalmol. Glaucoma 4, 295–304 (2021).
Romo-Bucheli, D. et al. Reducing image variability across OCT devices with unsupervised unpaired learning for improved segmentation of retina. Biomed. Opt. express 11, 346–363 (2020).
Zhu, J.-Y., Park, T., Isola, P. & Efros, A. A. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. Preprint at http://arxiv.org/abs/1703.10593 (2020).
Pang, Y., Lin, J., Qin, T. & Chen, Z. Image-to-image translation: Methods and applications. IEEE Trans. Multimed. 24, 3859–3881 (2021).
Agrawal, R. et al. Choroidal vascularity index as a measure of vascular status of the choroid: measurements in healthy eyes from a population-based study. Sci. Rep. 6, 21090 (2016).
Lun, K. et al. Investigating the macular choriocapillaris in early primary open-angle glaucoma using swept-source optical coherence tomography angiography. Front. Med. 9, 999167 (2022).
Adhi, M., Brewer, E., Waheed, N. K. & Duker, J. S. Analysis of morphological features and vascular layers of choroid in diabetic retinopathy using spectral-domain optical coherence tomography. JAMA Ophthalmol. 131, 1267–1274 (2013).
Lindner, M. et al. Choroidal thickness in geographic atrophy secondary to age-related macular degeneration. Investig. Ophthalmol. Vis. Sci. 56, 875–882 (2015).
Young, M., Fallah, N. & Forooghian, F. Choroidal degeneration in birdshot chorioretinopathy. Retina 35, 798–802 (2015).
Kim, J. T., Lee, D. H., Joe, S. G., Kim, J.-G. & Yoon, Y. H. Changes in choroidal thickness in relation to the severity of retinopathy and macular edema in type 2 diabetic patients. Investig. Ophthalmol. Vis. Sci. 54, 3378–3384 (2013).
Wong, R. L., Zhao, P. & Lai, W. W. Choroidal thickness in relation to hypercholesterolemia on enhanced depth imaging optical coherence tomography. Retina 33, 423–428 (2013).
Ahn, S. J., Woo, S. J. & Park, K. H. Retinal and choroidal changes with severe hypertension and their association with visual outcome. Investig. Ophthalmol. Vis. Sci. 55, 7775–7785 (2014).
Sızmaz, S. et al. The effect of smoking on choroidal thickness measured by optical coherence tomography. Br. J. Ophthalmol. 97, 601–604 (2013).
Ünsal, E. et al. Choroidal thickness in patients with diabetic retinopathy. Clin. Ophthalmol. 8, 637–642 (2014).
Lee, H. K., Lim, J. W. & Shin, M. C. Comparison of choroidal thickness in patients with diabetes by spectral-domain optical coherence tomography. Korean J. Ophthalmol. 27, 433–439 (2013).
Iovino, C. et al. Choroidal vascularity index: an in-depth analysis of this novel optical coherence tomography parameter. J. Clin. Med. 9, 595 (2020).
Agrawal, R. et al. Exploring choroidal angioarchitecture in health and disease using choroidal vascularity index. Prog. Retinal Eye Res. 77, 100829 (2020).
Abdolrahimzadeh, S., Di Pippo, M., Ciancimino, C., Di Staso, F. & Lotery, A. J. Choroidal vascularity index and choroidal thickness: potential biomarkers in retinitis pigmentosa. Eye 37, 1766–1773 (2023).
Agrawal, R. et al. Choroidal vascularity index in central serous chorioretinopathy. Retina 36, 1646–1651 (2016).
Imamura, Y., Fujiwara, T., Margolis, R. & Spaide, R. F. Enhanced depth imaging optical coherence tomography of the choroid in central serous chorioretinopathy. Retina 29, 1469–1473 (2009).
Pant, P. et al. Longitudinal Assessment of the Choroidal Vascularity Index in Eyes with Branch Retinal Vein Occlusion-Associated Cystoid Macular Edema. Ophthalmol Ther 12, 2103–2115 (2023).
Wei, X. et al. Choroidal structural analysis and vascularity index in retinal dystrophies. Acta Ophthalmol. 97, e116–e121 (2019).
Robbins, C. B. et al. Subfoveal choroidal thickness and choroidal vascularity index on spectral‐domain optical coherence tomography in Alzheimer’s disease: Neuroimaging/New imaging methods. Alzheimer’s Dement. 16, e042040 (2020).
Muller, J., Alonso-Caneiro, D., Read, S. A., Vincent, S. J. & Collins, M. J. Application of deep learning methods for binarization of the choroid in optical coherence tomography images. Transl. Vis. Sci. Technol. 11, 23 (2022).
Ibrahim, M. N. et al. Improved 3D Modeling of choroidal Haller’s sublayer vasculature based on swept-source OCT scans using Phansalkar thresholding. Investigative Ophthalmol. Vis. Sci. 64, 1131 (2023).
Ho, J., Jain, A. & Abbeel, P. Denoising diffusion probabilistic models. Adv. Neural Inf. Process. Syst. 33, 6840–6851 (2020).
Ganin Y., Lempitsky V. Unsupervised domain adaptation by backpropagation. PMLR:1180-1189 (2015).
Li, R., Jiao, Q., Cao, W., Wong, H.-S. & Wu, S. Model adaptation: Unsupervised domain adaptation without source data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 9641–9650 (2020).
Mills, R. P. et al. Categorizing the stage of glaucoma from pre-diagnosis to end-stage disease. Am. J. Ophthalmol. 141, 24–30 (2006).
Spaeth, G. L. & Shields, M. B. The stages of glaucoma. Am. J. Ophthalmol. 141, 147–148 (2006).
Wong, T. Y. et al. Prevalence and risk factors for diabetic retinopathy: the Singapore Malay Eye Study. Ophthalmology 115, 1869–1875 (2008).
Ting, D. S. W., Cheung, G. C. M. & Wong, T. Y. Diabetic retinopathy: global prevalence, major risk factors, screening practices and public health challenges: a review. Clin. Exp. Ophthalmol. 44, 260–277 (2016).
Kawasaki, R. et al. Prevalence of age-related macular degeneration in a Malay population: the Singapore Malay Eye Study. Ophthalmology 115, 1735–1741 (2008).
Lin, E. et al. Are choriocapillaris flow void features robust to diurnal variations? A swept-source optical coherence tomography angiography (OCTA) study. Sci. Rep. 10, 11249 (2020).
Cahyo, D. A. et al. Multi-task learning approach for volumetric segmentation and reconstruction in 3D OCT images. Biomed. Opt. Expr. 12, 7348–7360 (2021).
Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for biomedical image segmentation. Springer: 234-241 (2015).
Sonoda, S. et al. Luminal and stromal areas of choroid determined by binarization method of optical coherence tomographic images. Am. J. Ophthalmol. 159, 1123–1131.e1 (2015).
Sonoda, S. et al. Choroidal structure in normal eyes and after photodynamic therapy determined by binarization of optical coherence tomographic images. Investig. Ophthalmol. Vis. Sci. 55, 3893–3899 (2014).
Lanczos, C. A precision approximation of the gamma function. J. Soc. Ind. Appl. Math. Ser. B: Numer. Anal. 1, 86–96 (1964).
Moraes, T., Amorim, P., Da Silva, J. V. & Pedrini, H. Medical image interpolation based on 3D Lanczos filtering. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 8, 294–300 (2020).
He, K., Zhang, X., Ren, S. & Sun, J. Deep Residual Learning for Image Recognition. Preprint at http://arxiv.org/abs/1512.03385 (2015).
Acknowledgements
This work was funded by grants from the National Medical Research Council (CG/C010A/2017_SERI; OFLCG/004c/2018-00; MOH-000249-00; MOH-000647-00; MOH-001001-00; MOH-001015-00; MOH-000500-00; MOH-000707-00; MOH-001072-06; MOH-001286-00), National Research Foundation Singapore (NRF2019-THE002-0006 and NRF-CRP24-2020-0001), A*STAR (A20H4b0141), the Singapore Eye Research Institute & Nanyang Technological University (SERI-NTU Advanced Ocular Engineering (STANCE) Program), and the SERI-Lee Foundation (LF1019-1) Singapore.
Author information
Authors and Affiliations
Contributions
V.B. designed the study, conducted the experiments required for validating the idea of the project (data collection, data pre-processing, deep learning model development, main outcome measures generation, data post-processing, choroidal metrics measurement, analysis of the results, and statistical analysis), wrote the manuscript and designed its figures. A.K.D. co-designed the study, developed the deep learning model and reviewed the results and manuscript. S.S. facilitated the choroidal volume measurements. S.J., J.R., and T.G. assessed the synthetic data. CJ, WD, TB, LX, XX, AR, and TD were involved in the ideation of the project and reviewed the results and manuscript. YL and SL provided the initial project direction, co-designed the study, provided research funding, and reviewed the results and manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Bellemo, V., Kumar Das, A., Sreng, S. et al. Optical coherence tomography choroidal enhancement using generative deep learning. npj Digit. Med. 7, 115 (2024). https://doi.org/10.1038/s41746-024-01119-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41746-024-01119-3