Introduction

Optical coherence tomography angiography (OCTA) is a rapidly evolving technique that provides a fast, noninvasive assessment of retinal and choroidal microvascular flow1. OCTA can be used to differentiate and visualize the microvasculature of various retinal and choroidal layers at different depths, which is difficult to achieve using conventional fluorescein or indocyanine green angiography2. OCTA also allows for quantitative evaluation of retinal and choroidal blood flow using several vascular metrics, such as vessel density (VD), perfusion density (PD), and foveal avascular zone (FAZ)3,4. Numerous studies have reported that these vascular metrics may be useful for monitoring and detecting age-related macular degeneration5, diabetic retinopathy6, retinal vein occlusion7, and glaucoma8.

Confidence in blood flow metrics, as measured by OCTA during clinical assessments and for research purposes, requires demonstration of repeatability, particularly between subjects. In addition, OCTA measurements can be affected by several factors, including ocular anatomy, data acquisition methods, and image quality. Although several studies have demonstrated high repeatability of OCTA measurements, not only in normal eyes but also in those with disease9,10,11, the factors associated with repeatability have not been studied sufficiently. The ability to control these factors would allow clinicians to obtain more reliable blood flow metrics with OCTA evaluation.

Therefore, the purpose of the current study was to determine the repeatability of automatic OCTA measurements in healthy subjects by identifying the factors that have a significant effect on the repeatability of OCTA measurements.

Results

Demographics

This study initially included of 164 eyes from 164 subjects, and 23 eyes were excluded due to segmentation error or motion artifact. Finally, a total of 141 normal eyes from 83 men and 58 women (mean age, 46.1 ± 17.1 years) were examined in this study. The mean visual acuity of the subjects was −0.03 ± 0.08 logMAR, and the mean spherical equivalent and, intraocular pressure, and axial length were −1.32 ± 2.71 D, 15.4 ± 3.0 mmHg, and 24.2 ± 1.2 mm, respectively (Table 1).

Table 1 Demographics.
Full size table

Repeatability of OCTA measurements

There was no statistically significant difference in the VD, PD, or FAZ values obtained by two consecutive 6-mm scans performed with a 5-min interval (Table 2). ICC values achieved a better than ‘good’ grade for all parameters, but differed among measurement areas. VD and PD showed excellent reproducibility, while FAZ was classified as good. The CV was less than 10% for VD and PD for the 6-mm scan of the entire area; however, CV > 10% for VD and PD for 1-mm scans and the FAZ.

Table 2 The repeatability of two consecutive angiography measurements: vascular density, perfusion density, and foveal avascular zone metrics.
Full size table

Correlation analyses: factors associated with OCTA measurement repeatability

Pearson’s correlation was used to analyze the relationship between various factors and the repeatability of VD OCTA measurements in 6-mm scans (Table 3). Age, BCVA, IOP, spherical equivalent, axial length, and CMT were not significantly associated with the absolute differences or CVs of the two 6-mm scans (all p > 0.05). The VD of the 6-mm scan showed a significant negative correlation with the absolute differences and CVs of VD and PD for the total scan area (all p < 0.001), and were also significantly correlated with the CV of the FAZ (r = −0.326, p < 0.001). In addition, signal strength showed statistically significant negative correlations with absolute differences and CVs of VD, PD, and FAZ (all p < 0.01), indicating that the higher the signal strength, the better the repeatability. In multivariate linear regression analyses, signal strength (partial r = 0.246, p = 0.004) and VD from the 6-mm scans (partial r = −0.577, p < 0.001) showed a significant association with the CV of VD (Table 4).

Table 3 Pearson’s correlation analysis results of various clinical factors and repeatability parameters of optical coherence tomography angiography measurements
Full size table
Table 4 Multivariate linear regression analysis of significant factors in Pearson’s correlation analysis and repeatability parameters of vessel density for the 6-mm scan.
Full size table

Effect of signal strength

To confirm the effect of signal strength, we compared the OCTA measurements of 25 subjects for whom signal strength differed between the two consecutive scans (signal strength values of 8 and 9). Significant differences were observed in the measured parameters between the two signal strengths, with the exception of FAZ (Table 5).

Table 5 Comparison of vascular density, perfusion density, and foveal avascular zone metrics between two consecutive macular angiography scans (n = 25).
Full size table

To investigate the effect of signal strength on measurement reproducibility, we compared the CV values of subjects for whom signal strength was the same for the two consecutive scans, with both having a value of either 9 (n = 25) or 10 (n = 63). Both the VD and PD values for the 6-mm scan, and FAZ metrics, showed lower CV values in the group with a signal strength of 10 for both scans (Fig. 1).

Figure 1
figure 1

Comparison of the coefficients of variation of subjects showing the same signal strength across two scans, of 9 (n = 25) or 10 (n = 63). All parameters of the signal strength 10 group, including VD, PD, and FAZ metrics, showed better repeatability than those of the signal strength 9 group (all p < 0.05).

Full size image

Discussion

In previous studies, OCTA repeatability and reproducibility were generally good in both normal and diseased eyes. You et al. analyzed the intra- and inter-visit reproducibility of vascular density of superficial macula under various conditions [analysis range, retina state (normal or not), etc.] in normal patients and patients with retinal disease12. Although the repeatability and reproducibility were better in the normal group than in those with retinal disease, the CV value was <5% in most cases and showed good repeatability in both groups.

Lei et al. also analyzed the repeatability and reproducibility of measurements of the vascular density and PD of the superficial macula in normal and retinal disease eyes11. They performed three consecutive measurements using the same OCTA machine and analyzed the repeatability and reproducibility. The CV was less than 6% for both VD and PD repeatability and reproducibility, under various conditions. The ICC was high for both repeatability (0.82–0.98) and reproducibility (0.62–0.95).

In an earlier study, we analyzed the repeatability of OCTA VD measurements of the superficial macula in 134 patients with various retinal diseases that could cause deformation of the retinal structure9. Although previous studies have reported that pigment epithelium detachment and/or drusen interfere with the accuracy of VD measurements, our previous study of retinal disease cases showed good reproducibility, where the ICC value ranged from 0.737–0.905 and CV from 5.60–8.86% in13.

Despite the good repeatability and reproducibility of OCTA measurements in normal and diseased eyes reported in many studies, certain factors influence the readings, including the scan location, scan area (3 × 3-mm vs. 6 × 6-mm scans), signal strength, and retinal abnormalities. Therefore, we investigated factors that might affect the repeatability of OCTA measurements. Unlike previous studies, we analyzed OCTA results for normal eyes, and found that the signal strength had significant negative correlations with the absolute difference and CV of two measurements of VD and PD, based on 6-mm scans, and the FAZ. In other words, the higher the signal strength, the lower the CV value, and the better the repeatability (Figs 2 and 3). These results are consistent with those of Lei et al. showing that signal strength is a significant factor in the reproducibility of both VD and PD readings11. Signal strength showed a strong correlation with VD and PD in one of our previous studies14. In addition, Yu et al. reported that signal strength reduction which was induced by beam attenuation and defocus significantly affected OCTA measurements, and noted that signal strength should be considered in the interpretation of OCTA measurements15.

Figure 2
figure 2

Optical coherence tomography angiography (OCTA) images of a 60-year-old male subject with nuclear sclerosis grade 1. Signal strength of the upper and lower row images which scanned consecutively was 8, and both OCTA images showed significant difference values [vessel density in the 6 mm full area: 13.5 mm−1 (A) vs. 16.1 mm−1 (D); perfusion density in the 6 mm full area: 0.322 (B) vs. 0.386 (E); foveal avascular zone area: 0.10 mm2 (C) vs. 0.13 mm2 (F)].

Full size image
Figure 3
figure 3

Optical coherence tomography angiography (OCTA) images of a 68-year-old male subject with pseudophakia. Signal strength of the upper and lower row images which scanned consecutively was 10, and both OCTA images showed similar values [vessel density in the 6 mm full area: 16.2 mm−1 (A) vs. 16.9 mm−1 (D), perfusion density in the 6 mm full area: 0.396 (B) vs. 0.412 (E); foveal avascular zone area: 0.45 mm2 (C) vs. 0.46 mm2 (F)].

Full size image

In this study, during the calculation of the VD and PD, repeated B-scans were acquired from the same tissue location, and motion contrast was created by calculating the decorrelation of repeated B-scans. A thresholding algorithm was applied to the obtained unthresholded OCTA B-scans to remove invalid pixels associated with low or noisy OCTA pixels, and thus yield a thresholded OCT-B scan image16. During this process, the size of the vessel is an important factor, as large vessels are less affected by signal strength; however, signals from small vessels may be attenuated or undetected17. The macula has only small vessels, and signal strength therein thus correlates strongly with VD and PD reproducibility.

In this study, reproducibility indexed by ICC and CV for normal subjects was generally good; however, low reproducibility was also seen occasionally. The ICC values indicated excellent reproducibility in terms of VD and PD, but FAZ achieved only a ‘good’ classification. The CV values of VD and PD for the 6-mm scans showed good reproducibility (<10%). However, with regard to the PD in 1-mm scans of the central foveal area and the FAZ, the CV values exceeded 10%. In the report of You et al.12, superficial VD reproducibility was found to be lower in the central foveal area, because the retinal vessels are located only in the periphery of the central measurement area and the size and shape of the FAZ varies from person to person. This is similar to the results in the current study.

We suggest that if this study had used 3 × 3-mm OCTA scans, higher reproducibility of VD and PD values of the superficial macula would have been obtained. The 3 × 3-mm scan has better resolution than the 6 × 6-mm scan, with a scan interval of 12.2 µm compared to 17.1 µm4. This difference in scan density is expected to improve the repeatability of 3 × 3-mm OCTA scans. In addition, 3 × 3-mm scans measure approximately 60,000 points, while 6 × 6-mm scans measure approximately 120,000 points; as such, the 6 × 6-mm scan takes about twice as long, rendering it more susceptible to motion artifacts.

This study had the limitation of a retrospective, observational design. Previous studies included only a small number of patients, despite being concerned with both normal and diseased eyes. As our study included only normal patients, the results are meaningful in that the assessment of the reproducibility of OCTA was not affected by structural changes in the retina. Second, because 6 × 6 mm scan was used for analysis in this study, we could not confirm how the SS affected the analyses of other areas, such as the optic disc, or other scan protocols, such as the 3 × 3 mm angiography scans. Third, we included subjects with mild cataract in this study. However, mild cataracts can also affect image quality. Finally, Li et al.18. reported that the elongation of axial length lowered the repeatability of OCTA measurements, whereas no significant association between axial length and repeatability was found in our study. We excluded subjects with high myopia from this study and could not identify any trends of OCTA repeatability in the long eye. This should be clarified in future studies.

In conclusion, signal strength significantly affected OCTA repeatability in this study. Therefore, signal strength should be a key consideration in OCTA analyses of the retinal microvasculature and FAZ, where it is necessary to achieve the highest possible signal strength.

Methods

Subjects

This was a retrospective observational study. Patient medical data recorded at the Retina Clinic of Chungnam National University Hospital (Daejeon, Republic of Korea) were evaluated. No enrolled patient had participated in any other studies. The study protocol was approved by the institutional review board of Chungnam National University Hospital and adhered to the tenets of the Declaration of Helsinki. The requirement for obtaining informed patient consent was waived due to the retrospective nature of the study.

Among the healthy normal patients who visited our clinic for various reasons (health screening checkups, workup for ocular disease, etc.), those who met the study inclusion and exclusion criteria were included. All subjects underwent comprehensive ophthalmic examinations including a slit-lamp examination, best-corrected visual acuity (BCVA), intraocular pressure (IOP) measurements, dilated fundus examination, fundus photography, measurement of axial length using the IOLMaster® (Carl Zeiss Meditec, Jena, Germany), and of OCT and OCTA using the Zeiss Cirrus 5000 system (Carl Zeiss Meditec, Dublin, CA, USA). The inclusion criteria included age of 20–79 years, BCVA > 20/25 (Snellen), spherical equivalent within ±6 diopters (D), high-quality fundus image, absence of glaucomatous and other optic neuropathies, absence of retinal nerve fiber layer defects based on fundus photography, and no previous intraocular surgeries or procedures. Exclusion criteria included the presence of diabetes or hypertension, a history of retinal neuro-ophthalmic disease and glaucoma, a history of ocular trauma, BCVA < 20/25, IOP > 21 mmHg, spherical equivalent >  + 6.0 D or < −6.0 D, axial length ≥ 26.0 mm, and the presence of segmentation errors or motion artifacts in the angiography scan. Considering the possibility of deterioration of image quality, nuclear sclerosis (according to the Lens Opacity Classification System III19) grade 3 and above were excluded from the study. If both eyes met the inclusion criteria, one eye was randomly selected.

OCTA

The Zeiss Cirrus HD-OCT 5000 with AngioPlex™ (Carl Zeiss Meditec) was used to acquire microvasculature images of macular areas. All eyes underwent macular angiography imaging (two 6 × 6-mm scans at 5-min intervals) with the pupils dilated. The Zeiss Cirrus instrument operated at a central wavelength of 840 nm and a speed of 68,000 A-scans per second. There were 350 A-scans in each B-scan along the horizontal and vertical dimensions4. The Optical Micro Angiography Complex (OMAGc) was used to analyze changes in intensity and phase within the sequential B-scans performed at the same position20,21, to produce en face microvascular images. The vascular images of the superficial capillary plexus, which spanned the region from the internal limiting membrane to the inner plexiform layer, and the deep capillary plexus, which extended from the inner nuclear layer to the outer plexiform layer, were displayed automatically. The AngioPlex™ system incorporates FastTrac™ retinal-tracking technology to minimize motion artifacts.

All scans were analyzed using Cirrus OCTA software (AngioPlex™, version 10.0). The measurement area of the 6 × 6-mm scan was divided into nine subfields according to the Early Treatment of Diabetic Retinopathy study. VD (defined as the total length of the perfused vasculature per unit area in the region of measurement) and PD (defined as the total area of perfused vasculature per unit area in the region of measurement) of each subfield were measured automatically, as well as the FAZ area, perimeter, and circularity (defined as 4πA/P, where A and P represent the area and perimeter, respectively)22. All OCTA scans were performed by the same experienced examiner, and all scans were reviewed individually by two investigators, for loss of fixation, segmentation errors, and motion artifacts; substandard scans were excluded. The central macular thickness (CMT) was also measured using a 512 × 128 macular cube combination scan mode.

Repeatability assessment

Repeatability was assessed based on intraclass correlations (ICCs) and coefficients of variation (CV). The within-subject standard deviation (Sw) was calculated as the square root of the within-subject variance23. CV was calculated as CV = Sw/average of the measurements) × 100; CV < 10% indicated good reproducibility. The ICC is a statistical parameter that summarizes the reproducibility of a given group of subjects. It is based on variance component analyses and indicates the variance attributable to real differences between subjects as a proportion of the total variation24. ICC values were classified as follows: poor (ICC < 0.40), fair (0.40 ≤ ICC < 0.60), good (0.60 ≤ ICC < 0.75), or excellent (ICC ≥ 0.75)25.

Statistical analyses

The VD, PD, and FAZ for each of two consecutive scans were compared using the paired t-test, and the ICC and CV values were calculated. To identify factors that affect OCTA measurement repeatability, Pearson’s correlation and linear regression analyses of the absolute difference and CV of the VD value of the two scans (for the 6-mm scans of the entire area) were performed.

All statistical analyses were conducted using SPSS statistical software for Windows (version 18.0; SPSS Inc., Chicago, IL, USA) and GraphPad PrismTM (version 6.0; GraphPad Software, San Diego, CA, USA). Snellen BCVA results were converted into the logarithm of the minimum angle of resolution value (logMAR). Continuous variables are presented as the mean ± standard deviation. Differences were considered significant at p < 0.05.