Various factors can affect repeatability of optical coherence tomography angiography (OCTA) measurements, and they have not been studied sufficiently. We aimed to investigate the factors associated with the repeatability of automated superficial retinal vessel density (VD) and foveal avascular zone (FAZ) metrics acquired from OCTA. A total of 141 normal eyes from 141 healthy subjects were included, and two consecutive macular 6 × 6-mm angiography scans were performed. VD, perfusion density (PD), and FAZ of the superficial capillary plexus were calculated automatically. Reproducibility was assessed based on intraclass correlations (ICCs) and coefficients of variation (CVs). VD (ICC: 0.824, CV: 3.898) and PD (ICC: 0.845, CV: 4.042) over the entire 6-mm scan area showed better repeatability than VD (ICC: 0.752, CV: 17.470) and PD (ICC: 0.752, CV: 18.552) in the 1-mm scan, and with respect to the obtained FAZ metrics (ICC < 0.75, CV > 10.0%). Regression analyses showed that two factors, signal strength (p = 0.004) and average VD over the total 6-mm scan area (p < 0.001), were significantly correlated with the CV of the VD. Signal strength was associated with the repeatability of OCTA measurements and should be considered in the analysis of retinal VD and FAZ.
Optical coherence tomography angiography (OCTA) is a rapidly evolving technique that provides a fast, noninvasive assessment of retinal and choroidal microvascular flow1. OCTA can be used to differentiate and visualize the microvasculature of various retinal and choroidal layers at different depths, which is difficult to achieve using conventional fluorescein or indocyanine green angiography2. OCTA also allows for quantitative evaluation of retinal and choroidal blood flow using several vascular metrics, such as vessel density (VD), perfusion density (PD), and foveal avascular zone (FAZ)3,4. Numerous studies have reported that these vascular metrics may be useful for monitoring and detecting age-related macular degeneration5, diabetic retinopathy6, retinal vein occlusion7, and glaucoma8.
Confidence in blood flow metrics, as measured by OCTA during clinical assessments and for research purposes, requires demonstration of repeatability, particularly between subjects. In addition, OCTA measurements can be affected by several factors, including ocular anatomy, data acquisition methods, and image quality. Although several studies have demonstrated high repeatability of OCTA measurements, not only in normal eyes but also in those with disease9,10,11, the factors associated with repeatability have not been studied sufficiently. The ability to control these factors would allow clinicians to obtain more reliable blood flow metrics with OCTA evaluation.
Therefore, the purpose of the current study was to determine the repeatability of automatic OCTA measurements in healthy subjects by identifying the factors that have a significant effect on the repeatability of OCTA measurements.
This study initially included of 164 eyes from 164 subjects, and 23 eyes were excluded due to segmentation error or motion artifact. Finally, a total of 141 normal eyes from 83 men and 58 women (mean age, 46.1 ± 17.1 years) were examined in this study. The mean visual acuity of the subjects was −0.03 ± 0.08 logMAR, and the mean spherical equivalent and, intraocular pressure, and axial length were −1.32 ± 2.71 D, 15.4 ± 3.0 mmHg, and 24.2 ± 1.2 mm, respectively (Table 1).
Repeatability of OCTA measurements
There was no statistically significant difference in the VD, PD, or FAZ values obtained by two consecutive 6-mm scans performed with a 5-min interval (Table 2). ICC values achieved a better than ‘good’ grade for all parameters, but differed among measurement areas. VD and PD showed excellent reproducibility, while FAZ was classified as good. The CV was less than 10% for VD and PD for the 6-mm scan of the entire area; however, CV > 10% for VD and PD for 1-mm scans and the FAZ.
Correlation analyses: factors associated with OCTA measurement repeatability
Pearson’s correlation was used to analyze the relationship between various factors and the repeatability of VD OCTA measurements in 6-mm scans (Table 3). Age, BCVA, IOP, spherical equivalent, axial length, and CMT were not significantly associated with the absolute differences or CVs of the two 6-mm scans (all p > 0.05). The VD of the 6-mm scan showed a significant negative correlation with the absolute differences and CVs of VD and PD for the total scan area (all p < 0.001), and were also significantly correlated with the CV of the FAZ (r = −0.326, p < 0.001). In addition, signal strength showed statistically significant negative correlations with absolute differences and CVs of VD, PD, and FAZ (all p < 0.01), indicating that the higher the signal strength, the better the repeatability. In multivariate linear regression analyses, signal strength (partial r = 0.246, p = 0.004) and VD from the 6-mm scans (partial r = −0.577, p < 0.001) showed a significant association with the CV of VD (Table 4).
Effect of signal strength
To confirm the effect of signal strength, we compared the OCTA measurements of 25 subjects for whom signal strength differed between the two consecutive scans (signal strength values of 8 and 9). Significant differences were observed in the measured parameters between the two signal strengths, with the exception of FAZ (Table 5).
To investigate the effect of signal strength on measurement reproducibility, we compared the CV values of subjects for whom signal strength was the same for the two consecutive scans, with both having a value of either 9 (n = 25) or 10 (n = 63). Both the VD and PD values for the 6-mm scan, and FAZ metrics, showed lower CV values in the group with a signal strength of 10 for both scans (Fig. 1).
In previous studies, OCTA repeatability and reproducibility were generally good in both normal and diseased eyes. You et al. analyzed the intra- and inter-visit reproducibility of vascular density of superficial macula under various conditions [analysis range, retina state (normal or not), etc.] in normal patients and patients with retinal disease12. Although the repeatability and reproducibility were better in the normal group than in those with retinal disease, the CV value was <5% in most cases and showed good repeatability in both groups.
Lei et al. also analyzed the repeatability and reproducibility of measurements of the vascular density and PD of the superficial macula in normal and retinal disease eyes11. They performed three consecutive measurements using the same OCTA machine and analyzed the repeatability and reproducibility. The CV was less than 6% for both VD and PD repeatability and reproducibility, under various conditions. The ICC was high for both repeatability (0.82–0.98) and reproducibility (0.62–0.95).
In an earlier study, we analyzed the repeatability of OCTA VD measurements of the superficial macula in 134 patients with various retinal diseases that could cause deformation of the retinal structure9. Although previous studies have reported that pigment epithelium detachment and/or drusen interfere with the accuracy of VD measurements, our previous study of retinal disease cases showed good reproducibility, where the ICC value ranged from 0.737–0.905 and CV from 5.60–8.86% in13.
Despite the good repeatability and reproducibility of OCTA measurements in normal and diseased eyes reported in many studies, certain factors influence the readings, including the scan location, scan area (3 × 3-mm vs. 6 × 6-mm scans), signal strength, and retinal abnormalities. Therefore, we investigated factors that might affect the repeatability of OCTA measurements. Unlike previous studies, we analyzed OCTA results for normal eyes, and found that the signal strength had significant negative correlations with the absolute difference and CV of two measurements of VD and PD, based on 6-mm scans, and the FAZ. In other words, the higher the signal strength, the lower the CV value, and the better the repeatability (Figs 2 and 3). These results are consistent with those of Lei et al. showing that signal strength is a significant factor in the reproducibility of both VD and PD readings11. Signal strength showed a strong correlation with VD and PD in one of our previous studies14. In addition, Yu et al. reported that signal strength reduction which was induced by beam attenuation and defocus significantly affected OCTA measurements, and noted that signal strength should be considered in the interpretation of OCTA measurements15.
In this study, during the calculation of the VD and PD, repeated B-scans were acquired from the same tissue location, and motion contrast was created by calculating the decorrelation of repeated B-scans. A thresholding algorithm was applied to the obtained unthresholded OCTA B-scans to remove invalid pixels associated with low or noisy OCTA pixels, and thus yield a thresholded OCT-B scan image16. During this process, the size of the vessel is an important factor, as large vessels are less affected by signal strength; however, signals from small vessels may be attenuated or undetected17. The macula has only small vessels, and signal strength therein thus correlates strongly with VD and PD reproducibility.
In this study, reproducibility indexed by ICC and CV for normal subjects was generally good; however, low reproducibility was also seen occasionally. The ICC values indicated excellent reproducibility in terms of VD and PD, but FAZ achieved only a ‘good’ classification. The CV values of VD and PD for the 6-mm scans showed good reproducibility (<10%). However, with regard to the PD in 1-mm scans of the central foveal area and the FAZ, the CV values exceeded 10%. In the report of You et al.12, superficial VD reproducibility was found to be lower in the central foveal area, because the retinal vessels are located only in the periphery of the central measurement area and the size and shape of the FAZ varies from person to person. This is similar to the results in the current study.
We suggest that if this study had used 3 × 3-mm OCTA scans, higher reproducibility of VD and PD values of the superficial macula would have been obtained. The 3 × 3-mm scan has better resolution than the 6 × 6-mm scan, with a scan interval of 12.2 µm compared to 17.1 µm4. This difference in scan density is expected to improve the repeatability of 3 × 3-mm OCTA scans. In addition, 3 × 3-mm scans measure approximately 60,000 points, while 6 × 6-mm scans measure approximately 120,000 points; as such, the 6 × 6-mm scan takes about twice as long, rendering it more susceptible to motion artifacts.
This study had the limitation of a retrospective, observational design. Previous studies included only a small number of patients, despite being concerned with both normal and diseased eyes. As our study included only normal patients, the results are meaningful in that the assessment of the reproducibility of OCTA was not affected by structural changes in the retina. Second, because 6 × 6 mm scan was used for analysis in this study, we could not confirm how the SS affected the analyses of other areas, such as the optic disc, or other scan protocols, such as the 3 × 3 mm angiography scans. Third, we included subjects with mild cataract in this study. However, mild cataracts can also affect image quality. Finally, Li et al.18. reported that the elongation of axial length lowered the repeatability of OCTA measurements, whereas no significant association between axial length and repeatability was found in our study. We excluded subjects with high myopia from this study and could not identify any trends of OCTA repeatability in the long eye. This should be clarified in future studies.
In conclusion, signal strength significantly affected OCTA repeatability in this study. Therefore, signal strength should be a key consideration in OCTA analyses of the retinal microvasculature and FAZ, where it is necessary to achieve the highest possible signal strength.
This was a retrospective observational study. Patient medical data recorded at the Retina Clinic of Chungnam National University Hospital (Daejeon, Republic of Korea) were evaluated. No enrolled patient had participated in any other studies. The study protocol was approved by the institutional review board of Chungnam National University Hospital and adhered to the tenets of the Declaration of Helsinki. The requirement for obtaining informed patient consent was waived due to the retrospective nature of the study.
Among the healthy normal patients who visited our clinic for various reasons (health screening checkups, workup for ocular disease, etc.), those who met the study inclusion and exclusion criteria were included. All subjects underwent comprehensive ophthalmic examinations including a slit-lamp examination, best-corrected visual acuity (BCVA), intraocular pressure (IOP) measurements, dilated fundus examination, fundus photography, measurement of axial length using the IOLMaster® (Carl Zeiss Meditec, Jena, Germany), and of OCT and OCTA using the Zeiss Cirrus 5000 system (Carl Zeiss Meditec, Dublin, CA, USA). The inclusion criteria included age of 20–79 years, BCVA > 20/25 (Snellen), spherical equivalent within ±6 diopters (D), high-quality fundus image, absence of glaucomatous and other optic neuropathies, absence of retinal nerve fiber layer defects based on fundus photography, and no previous intraocular surgeries or procedures. Exclusion criteria included the presence of diabetes or hypertension, a history of retinal neuro-ophthalmic disease and glaucoma, a history of ocular trauma, BCVA < 20/25, IOP > 21 mmHg, spherical equivalent > + 6.0 D or < −6.0 D, axial length ≥ 26.0 mm, and the presence of segmentation errors or motion artifacts in the angiography scan. Considering the possibility of deterioration of image quality, nuclear sclerosis (according to the Lens Opacity Classification System III19) grade 3 and above were excluded from the study. If both eyes met the inclusion criteria, one eye was randomly selected.
The Zeiss Cirrus HD-OCT 5000 with AngioPlex™ (Carl Zeiss Meditec) was used to acquire microvasculature images of macular areas. All eyes underwent macular angiography imaging (two 6 × 6-mm scans at 5-min intervals) with the pupils dilated. The Zeiss Cirrus instrument operated at a central wavelength of 840 nm and a speed of 68,000 A-scans per second. There were 350 A-scans in each B-scan along the horizontal and vertical dimensions4. The Optical Micro Angiography Complex (OMAGc) was used to analyze changes in intensity and phase within the sequential B-scans performed at the same position20,21, to produce en face microvascular images. The vascular images of the superficial capillary plexus, which spanned the region from the internal limiting membrane to the inner plexiform layer, and the deep capillary plexus, which extended from the inner nuclear layer to the outer plexiform layer, were displayed automatically. The AngioPlex™ system incorporates FastTrac™ retinal-tracking technology to minimize motion artifacts.
All scans were analyzed using Cirrus OCTA software (AngioPlex™, version 10.0). The measurement area of the 6 × 6-mm scan was divided into nine subfields according to the Early Treatment of Diabetic Retinopathy study. VD (defined as the total length of the perfused vasculature per unit area in the region of measurement) and PD (defined as the total area of perfused vasculature per unit area in the region of measurement) of each subfield were measured automatically, as well as the FAZ area, perimeter, and circularity (defined as 4πA/P, where A and P represent the area and perimeter, respectively)22. All OCTA scans were performed by the same experienced examiner, and all scans were reviewed individually by two investigators, for loss of fixation, segmentation errors, and motion artifacts; substandard scans were excluded. The central macular thickness (CMT) was also measured using a 512 × 128 macular cube combination scan mode.
Repeatability was assessed based on intraclass correlations (ICCs) and coefficients of variation (CV). The within-subject standard deviation (Sw) was calculated as the square root of the within-subject variance23. CV was calculated as CV = Sw/average of the measurements) × 100; CV < 10% indicated good reproducibility. The ICC is a statistical parameter that summarizes the reproducibility of a given group of subjects. It is based on variance component analyses and indicates the variance attributable to real differences between subjects as a proportion of the total variation24. ICC values were classified as follows: poor (ICC < 0.40), fair (0.40 ≤ ICC < 0.60), good (0.60 ≤ ICC < 0.75), or excellent (ICC ≥ 0.75)25.
The VD, PD, and FAZ for each of two consecutive scans were compared using the paired t-test, and the ICC and CV values were calculated. To identify factors that affect OCTA measurement repeatability, Pearson’s correlation and linear regression analyses of the absolute difference and CV of the VD value of the two scans (for the 6-mm scans of the entire area) were performed.
All statistical analyses were conducted using SPSS statistical software for Windows (version 18.0; SPSS Inc., Chicago, IL, USA) and GraphPad PrismTM (version 6.0; GraphPad Software, San Diego, CA, USA). Snellen BCVA results were converted into the logarithm of the minimum angle of resolution value (logMAR). Continuous variables are presented as the mean ± standard deviation. Differences were considered significant at p < 0.05.
Data supporting the findings of the current study are available from the corresponding author on reasonable request.
Spaide, R. F., Klancnik, J. M. Jr. & Cooney, M. J. Retinal vascular layers imaged by fluorescein angiography and optical coherence tomography angiography. JAMA Ophthalmol 133, 45–50, https://doi.org/10.1001/jamaophthalmol.2014.3616 (2015).
Savastano, M. C., Lumbroso, B. & Rispoli, M. I. N. Vivo Characterization of Retinal Vascularization Morphology Using Optical Coherence Tomography Angiography. Retina 35, 2196–2203, https://doi.org/10.1097/iae.0000000000000635 (2015).
Kim, A. Y. et al. Quantifying Microvascular Density and Morphology in Diabetic Retinopathy Using Spectral-Domain Optical Coherence Tomography Angiography. Invest Ophthalmol Vis Sci 57, Oct362–370, https://doi.org/10.1167/iovs.15-18904 (2016).
Rosenfeld, P. J. et al. ZEISS Angioplex Spectral Domain Optical Coherence Tomography Angiography: Technical Aspects. Dev Ophthalmol 56, 18–29, https://doi.org/10.1159/000442773 (2016).
Toto, L. et al. Association between outer retinal alterations and microvascular changes in intermediate stage age-related macular degeneration: an optical coherence tomography angiography study. Br J Ophthalmol 101, 774–779, https://doi.org/10.1136/bjophthalmol-2016-309160 (2017).
Hirano, T. et al. Quantifying vascular density and morphology using different swept-source optical coherence tomography angiographic scan patterns in diabetic retinopathy. Br J Ophthalmol, https://doi.org/10.1136/bjophthalmol-2018-311942 (2018).
Wang, Q. et al. Optical coherence tomography angiography in retinal vein occlusions. Graefe’s Archive for Clinical and Experimental Ophthalmology, 1–8 (2018).
Yarmohammadi, A. et al. Relationship between Optical Coherence Tomography Angiography Vessel Density and Severity of Visual Field Loss in Glaucoma. Ophthalmology 123, 2498–2508, https://doi.org/10.1016/j.ophtha.2016.08.041 (2016).
Lee, M. W., Kim, K. M., Lim, H. B., Jo, Y. J. & Kim, J. Y. Repeatability of vessel density measurements using optical coherence tomography angiography in retinal diseases. Br J Ophthalmol, https://doi.org/10.1136/bjophthalmol-2018-312516 (2018).
Venugopal, J. P. et al. Repeatability of vessel density measurements of optical coherence tomography angiography in normal and glaucoma eyes. British Journal of Ophthalmology, bjophthalmol, 2017-310637 (2017).
Lei, J. et al. Repeatability and Reproducibility of Superficial Macular Retinal Vessel Density Measurements Using Optical Coherence Tomography Angiography En Face Images. JAMA Ophthalmol 135, 1092–1098, https://doi.org/10.1001/jamaophthalmol.2017.3431 (2017).
You, Q. et al. Reproducibility of Vessel Density Measurement With Optical Coherence Tomography Angiography in Eyes With and Without Retinopathy. Retina 37, 1475–1482, https://doi.org/10.1097/iae.0000000000001407 (2017).
Spaide, R. F., Fujimoto, J. G. & Waheed, N. K. Image Artifacts in Optical Coherence Tomography Angiography. Retina 35, 2163–2180, https://doi.org/10.1097/iae.0000000000000765 (2015).
Lim, H. B., Kim, Y. W., Kim, J. M., Jo, Y. J. & Kim, J. Y. The Importance of Signal Strength in Quantitative Assessment of Retinal Vessel Density Using Optical Coherence Tomography Angiography. Sci Rep 8, 12897, https://doi.org/10.1038/s41598-018-31321-9 (2018).
Yu, J. J. et al. Signal Strength Reduction Effects in OCT Angiography. Ophthalmol Retina, https://doi.org/10.1016/j.oret.2019.04.029 (2019).
Cole, E. D. et al. The Definition, Rationale, and Effects of Thresholding in OCT Angiography. Ophthalmol Retina 1, 435–447, https://doi.org/10.1016/j.oret.2017.01.019 (2017).
Spaide, R. F., Fujimoto, J. G., Waheed, N. K., Sadda, S. R. & Staurenghi, G. Optical coherence tomography angiography. Prog Retin Eye Res 64, 1–55, https://doi.org/10.1016/j.preteyeres.2017.11.003 (2018).
Li, M. et al. The repeatability of superficial retinal vessel density measurements in eyes with long axial length using optical coherence tomography angiography. BMC Ophthalmol 18, 326, https://doi.org/10.1186/s12886-018-0992-y (2018).
Chylack, L. T. Jr. et al. The Lens Opacities Classification System III. The Longitudinal Study of Cataract Study Group. Arch Ophthalmol 111, 831–836, https://doi.org/10.1001/archopht.1993.01090060119035 (1993).
Zhang, A., Zhang, Q., Chen, C. L. & Wang, R. K. Methods and algorithms for optical coherence tomography-based angiography: a review and comparison. J Biomed Opt 20, 100901, https://doi.org/10.1117/1.jbo.20.10.100901 (2015).
An, L., Johnstone, M. & Wang, R. K. Optical microangiography provides correlation between microstructure and microvasculature of optic nerve head in human subjects. J Biomed Opt 17, 116018, https://doi.org/10.1117/1.jbo.17.11.116018 (2012).
Wang, R. K. et al. Three dimensional optical angiography. Opt Express 15, 4083–4097 (2007).
Bland, J. M. & Altman, D. G. Measurement error. Bmj 313, 744 (1996).
Mwanza, J. C. et al. Reproducibility of peripapillary retinal nerve fiber layer thickness and optic nerve head parameters measured with cirrus HD-OCT in glaucomatous eyes. Invest Ophthalmol Vis Sci 51, 5724–5730, https://doi.org/10.1167/iovs.10-5222 (2010).
Cicchetti, D. V. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological assessment 6, 284 (1994).
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Lee, T.H., Lim, H., Nam, K.Y. et al. Factors Affecting Repeatability of Assessment of the Retinal Microvasculature Using Optical Coherence Tomography Angiography in Healthy Subjects. Sci Rep 9, 16291 (2019) doi:10.1038/s41598-019-52782-6