Agreement in Measures of Macular Perfusion between Optical Coherence Tomography Angiography Machines

We evaluated the agreements in foveal avascular zone (FAZ) area and vessel density (VD) parameters (within the superficial capillary plexus region), between two widely used optical coherence tomography angiography machines. Participants who attended the Singapore Malay Eye Study III between 29th March and 6th August 2018, were enrolled in this study. Participants underwent fovea-centered 6×6-mm macular cube scan, using both AngioVue and Cirrus HDOCT machines. Scans were analyzed automatically using built-in review software of each machine. 177 eyes (95 participants) without retinal diseases were included for final analysis. Mean FAZ area was 0.38 ± 0.11 mm2 and 0.30 ± 0.10 mm2, based on AngioVue and Cirrus HDOCT, respectively. Mean parafoveal VD was 0.50 ± 0.04 in Angiovue, and 0.43 ± 0.04 in Cirrus HDOCT. Cirrus HDOCT measurements were consistently lower than those by AngioVue, with a mean difference of −0.08 (95% limits of agreement [LOA], −0.30–0.13) mm2 for FAZ area, and −0.07 (95% LOA, −0.17–0.03) for parafoveal VD. Intraclass correlation coefficients for FAZ area and parafoveal VD were 0.33 and 0.07, respectively. Our data suggest that agreements between AngioVue and Cirrus HDOCT machines were poor to fair, thus alternating use between these two machines may not be recommended especially for follow up evaluations.

octA image analysis. All scans were analyzed and measured automatically within the superficial capillary plexus (SCP) region, using the respective machines' built-in review software. In the AngioVue (software version 2016.2.0.35), the SCP region was automatically segmented between an inner boundary 3 µm below the internal limiting membrane (ILM), and an outer boundary 15 µm below the inner plexiform layer (IPL) (Fig. 1A). In the Cirrus HDOCT (AngioPlex software, version10.0), the inner boundary of the SCP was set at the ILM, whereas the outer boundary was set at the level marked as 70% of the thickness between ILM and the outer plexiform layer (OPL) (Fig. 1D).
Quantitative analysis of the foveal avascular zone (FAZ) area within the SCP region was performed automatically using the AngioVue (Fig. 1B) and Cirrus software (Fig. 1E). The vessel density (VD) was defined as the total area of perfused vasculature per unit area in a region of measurement, which was also automatically calculated by AngioVue (Fig. 1C) and Cirrus HDOCT (Fig. 1F) review software. Foveal VD was measured from the central subfield zone 1 (1 mm diameter); while parafoveal VD was measured and averaged from the inner ring area (3 mm diameter) formed by subfields zone 2 to 5. The location and dimension of subfields 1 to 5 were the same between the built-in review software of AngioVue and Cirrus HDOCT. clinical measurements and questionnaire. All participants underwent standardized systemic and ophthalmic examinations. Non-fasting venous blood samples were collected and analyzed for biochemical testing of serum glycated hemoglobin (HbA1c), glucose and total cholesterol. Diabetes was defined as either random glucose ≥11.1 mmol/L, HbA1c ≥ 6.5%, use of diabetic medication, or self-reported history. Hypertension was defined as either systolic blood pressure (BP) ≥ 140 mmHg, diastolic BP ≥ 90 mmHg, antihypertensive drugs usage, or self-reported history of hypertension. Hyperlipidemia was defined as either total cholesterol ≥6.2 mmol/L or use of lipid lowering medication. Spherical equivalent (SE) was calculated as the spherical value plus half of the negative cylinder value. Axial length (AL) was measured using non-contact partial coherence interferometry (IOL Master V3.01, Carl Zeiss Meditec AG, Jena, Germany). www.nature.com/scientificreports www.nature.com/scientificreports/ A detailed interviewer-administered questionnaire was used to collect information including medication use, history of systemic and ocular disease, history of retinal surgery and laser treatment, as well as smoking status (classified as current and non-current smokers).
Statistical analysis. Data from both eyes were included in the analysis if available. We first described the distribution of the FAZ area and VD parameters of each machine. The correlations between the parameters measured from both machines, were calculated using Pearson correlation coefficient. The agreement in FAZ area and VD measurements between both machines, was evaluated using intraclass correlation coefficients (ICC, based on absolute agreement model) 11 and Bland-Altman plot 12 .
ICC values of 0.81 to 1.00 indicate almost perfect agreement, values of 0.61 to 0.80 indicate good agreement, values of 0.41 to 0.60 indicate moderate agreement, Values less than 0.40 indicate poor to fair agreement 13 . In Bland-Altman plot, the difference of two measurements (from both machines) was plotted against the average of two measurements. Proportional bias was investigated by testing whether the slope of the least squares regression line for difference of two measurements against the average of two measurements significantly differed from zero. Meanwhile, systemic bias was investigated by comparing the mean difference values with zero using one-sample t test 14 . The presence of proportional bias indicates that the differences between both machines are not constant throughout the range of measurements, whereas systemic bias indicates the differences are fixed throughout all measurements.
All statistical analyses were performed using Stata 13.0 (StataCorp LP, College Station, TX). The P value (2-sided) for significance was set at less than 0.05.

Results
Of the 223 eyes from the original 147 participants who underwent OCTA scans with the AngioVue and Cirrus HDOCT, 46 eyes were excluded (Fig. 2) due to poor signal strength, significant media opacity (6 eyes, e.g. dense cataract and large floater), retinal diseases (24 eyes, due to diabetic retinopathy, age-related macular degeneration, epi-retinal membrane, pigment epithelial detachment, wet age-related macular degeneration), and segmentation error made by the built-in review software (7 eyes). Thus, leaving 177 eyes from 95 participants in the final analysis. The mean age of included participants was 62.6 ± 6.8 years, and 51 (53.7%) of them were female. All participants were Malays. Table 1 shows the systemic and ocular characteristics of study participants. 73 (76.8%) participants had hypertension, 48 (50.5%) had diabetes, and 68 (71.6%) participants had hyperlipidemia. 18 (19.0%) were current smokers. The mean spherical equivalent was 0.27 ± 1.74 diopter, and the mean axial length was 23.52 ± 1.00 mm.
The distribution and correlations between OCTA parameters in the SCP are presented in Table 2. The mean FAZ area was 0.38 ± 0.11 mm 2 in Angiovue, and 0.30 ± 0.10 mm 2 in Cirrus HDOCT. The mean foveal VD was 0.29 ± 0.05 in Angiovue, and 0.20 ± 0.07 in Cirrus HDOCT. The mean parafoveal VD was 0.50 ± 0.04 and 0.43 ± 0.04 in the AngioVue and Cirrus HDOCT, respectively. Significant linear correlations albeit small effect estimates were observed between the parameters from both machines, this included the FAZ area (r = 0.4282, P < 0.001), foveal VD (r = 0.4985, P < 0.001) and parafoveal VD (r = 0.2022, P = 0.007). Table 3 further shows the agreement in parameters between AngioVue and Cirrus HDOCT. The ICC values of FAZ area, foveal VD were 0.33 and 0.22, respectively; while parafoveal VD was 0.07, indicating poor to fair agreement between these measurements. Measurements from the Cirrus HDOCT were consistently lower than those by AngioVue, with mean differences of −0.08 mm 2 (95% limits of agreement [LOA], −0.30 to 0.13) in FAZ area (Fig. 3A), −0.09 (95% LOA, −0.21 to 0.03) in the foveal VD (Fig. 3B), and −0.07 (95% LOA, −0.17 to 0.03) in the parafoveal VD (Fig. 3C). Systemic bias was found in the measurements of the FAZ area, foveal VD, parafoveal www.nature.com/scientificreports www.nature.com/scientificreports/ VD, as well as VD in all parafoveal subfields (all P < 0.001). On the other hand, the VD differences between both machines for measurements of foveal VD, nasal, superior and temporal macular subfields significantly correlated with the average of both machines' measurements (all P ≤ 0.029) and thus indicated presence of proportional bias.
Further sensitivity analyses which randomly selected one study eye from each participant showed largely similar results as the original results which used both eyes (Supplementary Tables 1 & 2).

Discussion
In this study, we evaluated the agreement in OCTA parameters (within the SCP region) between two commonly used OCTA modalities. Our results demonstrated that the measurements of the FAZ and VD parameters from the Cirrus HDOCT were consistently lower than AngioVue, and the agreements in these parameters between the two machines were generally poor to fair. These data suggest that alternating use between the two machines may not be recommended, especially in multi-center clinical trials and long-term follow up visits.
The FAZ is the central area of macula without blood vessels, but surrounded by a continuous network of capillaries 15 . It provides clinically useful information for macular ischemia 16 . It is also a significant predictor for visual acuity in DR, retinal vein occlusion 17 and glaucoma 18 . The advent of OCTA provided new ways of quantifying the FAZ area 16 , In our study, the agreement in the FAZ area between the two OCTA machines was poor. Consistent with our finding, previous studies also reported significant differences in the FAZ area measurement from different machines 8,15,19 . Two factors might contribute to this poor agreement. Firstly, the segmentation of the FAZ boundary in each machine is different 16 . For instance, in AngioVue, automated segmentation of the SCP was set at an inner boundary at 3 µm below the ILM and outer boundary at 15 µm below the IPL. In Cirrus HDOCT, the SCP region's inner boundary was set at the ILM, whereas the outer boundary was an estimated boundary of the IPL, which was marked at the 'level' where it was 70% of the thickness between ILM and the OPL. Secondly, the density of B-scans in the same scan was different between the two machines as well. In the 3 × 3-mm area, the AngioVue captures 304 B-scans, while the Cirrus HDOCT captures 245 B-scans. This might have also resulted in the different measurements observed between the two machines 15 . The overall poor agreement of FAZ area indicates that the results from the two machines cannot be interpreted interchangeably.
Quantification of the macular VD using OCTA is a potential imaging marker that may be deployed in clinical practice in the future. In our study, the agreement between the two machines for parafoveal VD measurement was poor to fair. This was consistent with most previous studies 8, 15 . The poor agreement of the VD may be partially due to the different segmentation boundaries of retinal layer in different OCTA machines (as described above  www.nature.com/scientificreports www.nature.com/scientificreports/ in Methods). The slightly different 'slabbed' layer might result in different VD measurement 15 . On the other hand, two previous studies reported good reproducibility of the VD measurement across different machines 19,20 . However, these studies had very small sample size (n ≤ 24 eyes), and might be subjected to bias.
The strengths of our study include the stringent quality control performed on all OCTA scans. All B-scans were checked for alignment and segmentation errors. Images with segmentation errors, poor signal strength or retinal diseases which might potentially affect the FAZ and VD measurements, were excluded. This robust data preparation process further substantiates the validity of our results. However, a few limitations should be noted. First, we only compared parameters within the SCP region between AngioVue and Cirrus HDOCT. This was because measurements in other retinal layers were not available from the Cirrus AngioPlex review software when this study was conducted. Second, even though we only evaluated parameters of the same measurement area from both machines, it cannot be entirely ruled out that the scans might not overlap perfectly due to the potential slight difference in fixation pattern of study eye from one machine to another, especially given that the 'registered fixation pattern' in Angiovue could not be transferred to Cirrus HDOCT, and vice versa. Nevertheless, as all scans were thoroughly checked for scan alignment (i.e. centred at fovea), we expect this limitation to have minimal impact on the overall findings. Lastly, this study was restricted to eyes without retinal diseases, therefore generalizability of our results to eyes with pathologies may be limited.
In conclusion, in eyes without retinal diseases, between AngioVue and Cirrus HDOCT, the agreements in the FAZ and VD parameters were poor to fair. Our findings suggest that alternating use between these two machines may not be recommended.