Introduction

Glaucoma is a progressive optic neuropathy characterized by a specific pattern of damage in the optic nerve head (ONH) morphology and retinal nerve fiber layer (RNFL) thickness.1, 2, 3, 4 It has been demonstrated that various imaging technologies are capable of detecting these structural changes in glaucomatous and GS eyes.4, 5, 6

Most nerve head parameter measurements generated by the current spectral-domain optical coherence tomography (SDOCT) and previous generation of time-domain optical coherence tomography (TDOCT) devices rely on accurate identification of the disc margin. Rim volume, optic disc area, and cup-to-disc ratio measurements are just a few parameters that require disc margin delineation. This essential landmark can be automatically identified by software or manually demarcated by the operator. Aside from directly affecting ONH parameters, devices such as the RTVue-100 SDOCT (RTVue-100; Optovue Inc., Fremont, CA, USA) superimpose the RNFL measurement circle on the center of the optic disc, so any displacement of the disc margin may indirectly affect the RNFL thickness measurements as well.

The neural canal opening (ie Bruch’s membrane opening) is the anatomic structure that colocalizes with the optic disc margin.7 Current SDOCT devices are not capable of distinguishing Bruch’s membrane (BM) from the retinal pigment epithelium (RPE) layer,8 so the inner edge of the bright band that represents the retinal pigment epithelium/Bruch’s membrane (RPE/BM) complex, referred to as ‘RPE/BM end points’, is used to identify the disc margin.9 Evidence from studies employing TDOCT has shown good agreement between manual and automated definitions of disc margin delineation, although these studies note that failure of proper disc margin recognition does occur.10, 11 Similarly, studies employing SDOCT have shown good agreement between automated and manual disc delineation methods;12, 13 however, statistically significant differences in ONH and RNFL parameters have been reported.13

The purpose of this study was to investigate the impact of manual vs automated RPE/BM endpoint demarcation on the measurements of ONH and all sectoral RNFL thickness parameters in a group of normal, GS, and glaucoma patients participating in the Advanced Imaging for Glaucoma Study (AIGS) that demonstrate a wide range of glaucomatous damage.

Materials and methods

Study population

Participants consisted of perimetric glaucoma (PG), GS and preperimetric glaucoma (GSPPG) cases, and normal patients who were prospectively enrolled in the AIGS study at Bascom Palmer Eye Institute, Palm Beach Gardens, FL. Informed consent was obtained from all subjects using a consent form approved by the Institutional Review Board for Human Research of the University of Miami Miller School of Medicine, which was in agreement with the provisions of the Declaration of Helsinki. The inclusion criteria consisted of a spherical equivalent refractive error between −8.00 and +4.00 diopters, best-corrected visual acuity equal to or better than 20/40, age between 40 and 79 years at enrollment, reliable standard automated perimetry (SAP; ≤15% fixation losses and ≤33% false positives and false negatives), and no prior history of intraocular surgery except for uncomplicated cataract extraction. Subjects with peripapillary atrophy extending to 1.7 mm from the center of the optic disc or those who had poor-quality OCT images were excluded.

PG patients had glaucomatous optic nerve damage and/or abnormal SAP defined as abnormal glaucoma hemifield test (GHT) (ie, outside the normal limits) and pattern standard deviation (PSD) outside the 95% confidence interval. Glaucomatous optic nerve damage was defined as neuroretinal rim narrowing, notching, excavation, or RNFL defect. Patients with SAP abnormality had at least one confirmatory visual field examination. GSPPG was defined as ocular hypertension (intraocular pressure (IOP)≥24 mm Hg) with normal optic discs and normal SAP (normal GHT and PSD>0.05), ONH/nerve fiber layer defect, progression with normal SAP, or a diagnosis of glaucoma in the fellow eye. Normal subjects had no history of ocular disease except cataract, intraocular pressure (IOP)≤21 mm Hg, normal optic disc appearance based on clinical stereoscopic examination and review of stereoscopic disc photographs, and normal SAP.

Clinical measurements

All patients underwent a baseline examination consisting of a complete ophthalmic examination including slit-lamp biomicroscopy, gonioscopy, Goldmann applanation tonometry, ultrasound pachymetry, dilated fundus examination, stereoscopic photography of the optic disc, SAP, and SDOCT imaging.

SDOCT imaging and analysis

A commercially available RTVue-100 SDOCT (RTVue-100; Optovue Inc., version 6.1.0.21) was used to measure the RNFL and ONH parameters. A laser diode with 840±10 nm wavelength captures 26 000 A-scans/second, with a frame rate of 256–4096 A-scans/frame and 5 μm axial and 15 μm transverse resolution, and, similar to other SDOCT technologies, uses fast Fourier transformation of collected frequencies to improve the signal-to-noise ratio.14, 15, 16, 17

The ONH protocol was used to obtain both the ONH and RNFL measurements. RNFL thickness measurements were taken along 13 circular B-scans with diameters of 1.3–4.9 mm manually positioned on the optic disc to create a peripapillary RNFL thickness map. The RNFL thickness measurement was generated from a 3.45-mm-diameter circle and is calculated as the difference in distance between the internal limiting membrane and the outer edge of the inner plexiform layer.17 The ONH scan provided a three-dimensional (3D) view of the ONH, and was created using 12 radial B-scans with a fixed length of 3.7 mm for ONH shape analysis. The RPE/BM end points were determined automatically. Twenty-four RPE/BM end points defined the shape of the disc margin.18, 19 The software allows the operator to manually correct the locations of four RPE/BM end points on two perpendicular B-scans. The software would then extrapolate the other points using these four manually adjusted points. The center of the four points was used as the center of the optic disc. The RNFL measurement circle in the ONH scan protocol was centered on the center of the optic disc obtained from the locations of RPE/BM end points. Therefore, improper identification of RPE/BM end point(s) may directly affect the ONH parameters and indirectly affect the RNFL measurements.

The RTVue-100 device calculates nine nerve head parameters: rim volume, nerve head volume, cup volume, average cup-to-disc (C/D) area ratio, horizontal C/D ratio, vertical C/D ratio, optic disc area, rim area, and cup area. The RNFL parameters include the average RNFL thickness, eight sectoral measurements, and four quadrant measurements. The eight sectors are temporal-upper (TU), superior-temporal (ST), superior-nasal (SN), nasal-upper, nasal-lower (NL), inferior-nasal (IN), inferior-temporal (IT), and temporal-lower (TL). The quadrants are generated from the average of two corresponding sectors; the superior-temporal quadrant (STQ) is the average of ST and TU sectors, the superior-nasal quadrant (SNQ) is the average of the SN and NU sectors, the inferior-nasal quadrant (INQ) is the average of the IN and NL sectors, and the inferior-temporal quadrant (ITQ) is the average of the IT and TL sectors.

All scans were taken without pharmacologic pupil dilation. To focus the image, the operator used the ‘auto all’ function, which optimizes the focus adjustment, polarization adjustment, and Z position, to obtain a clear and centered image. An Optovue Inc.-authorized technician calibrated the device 1 month prior to beginning the study. A single trained operator (SMI) acquired three sequential scans while the patient maintained the same head position on the chinrest without moving. Images that were obtained during eye movement, or were unfocused, poorly centered, had segmentation failure, or had a signal strength index<40 were discarded and were retaken to meet the quality criteria. An algorithm called Symmetry Analysis automatically generated an average of three consecutive exams. The operator saved and documented the automated measurements. The RPE/BM end points on each scan were reanalyzed and manually modified to their subjective best fit by selecting the ‘modify RPE anchor points’ option in the Analysis Mode. The three modified scans were averaged using the same Symmetry Analysis algorithm and digital copies of the reports were generated.

Three-dimensional images of the optic discs were examined to determine the degree of optic disc tilt in both the horizontal and vertical meridians. Measurements were centered on the optic disc and tilted orientation was defined as an angle ≥20° between an oblique line connecting the two opposite RPE/BM end points and the horizontal line transecting one of the two RPE/BM end points.20

Statistical analysis

Statistical analysis was performed using JMP 8.0.2 (SAS Inc., Cary, NC, USA), and IBM SPSS 20.0 (IBM, Armonk, NY, USA). Analysis of variance with Bonferroni post-hoc test for multiple comparisons and chi-square test were used. A probability value of 0.001 was considered as statistically significant to account for multiple comparisons.

The coefficient of variation (COV) was calculated as the standard deviation divided by the mean. Low COV values indicate lower variability and better repeatability of the diagnostic test. The concordance correlation coefficient (CCC) was calculated as a measure of the agreement between automated and manual demarcation, with higher values indicating strong agreement between the two measurement variables;21 <0.20 is considered poor, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 good, and 0.81–1.00 very good.22 Bland–Altman plots were generated to compare the automated and manual RPE/BM endpoint demarcations, and to illustrate the difference between the automated and manual measurements vs their mean, and their limits of agreement (mean difference±1.96 SD).

Results

Ninety-nine eyes of 50 patients (31 female, 19 male, age 68±10 years) consisting of 36 PG, 56 GSPPG, and 7 normal eyes were included in the analysis. Table 1 describes the clinical characteristics of the study population. No subjects had extensive peripapillary atrophy extending beyond 1.7 mm from the center of the optic disc. A total of 3 eyes had tilted optic disc with an angle of orientation of ≥20°. A total of 297 scans were deemed acceptable based on the inclusion/exclusion criteria. An experienced operator (SMI) performed an independent examination of all accepted images for the accuracy of the automated placement of the RPE/BM end points. In all 276 (92.9%) scans had at least one RPE/BM end point modified, resulting in recalculation of measurement values. Figure 1 demonstrates a single ONH scan with automated and manual RPE/BM endpoint demarcation and shows the misplacement of the RPE/BM end points by the automated algorithm.

Table 1 Clinical characteristics of the study population (n=99)
Full size table
Figure 1
figure 1

Optic nerve head scan results after automated and manual RPE/BM endpoint delineation methods. RPE/BM end points were manually adjusted to more accurately demarcate RPE/BM end points (yellow circle).

Full size image

Table 2 compares the automated and manual techniques for the measurements of RNFL and ONH parameters. Overall, the RNFL thickness measurements were similar (P>0.05) between the two methods of demarcation, except the inferior-nasal sector that showed borderline difference between the two methods (P=0.04). However, this difference did not remain statistically significant after Bonferroni correction.

Table 2 Automated versus manual RPE/BM endpoint demarcation values
Full size table

The Bland–Altman plot of the difference between the two methods vs the mean of the two methods for average RNFL thickness measurements (Figure 2a) demonstrates that average RNFL thickness values were within the limits of agreement of the two techniques for the two methods. For the ONH measurements, the C/D ratio and rim area showed significant differences between the two methods (P<0.001). The Bland–Altman plot of the difference vs the mean of the two methods for the rim area (Figure 2b) showed that the rim area measurements were greater with the automated demarcation method compared with the manual method (P<0.001). The overall rim volume was not significantly different between the two demarcation methods (Figure 2c; P=0.40); however, the automated method underestimated rim volumes <0.04 mm3, performed similarly to the manual method for rim volumes between 0.04 and 0.08 mm3, and overestimated rim volumes >0.08 mm3.

Figure 2
figure 2

(a–c) Bland–Altman plots comparing manual vs automated RPE/BM endpoint delineation methods for average retinal nerve fiber layer (RNFL) thickness (a), rim area (b), and rim volume (c). Average RNFL thickness values were within the limits of agreement, rim area measurements were greater with the automated demarcation method, and rim volume measurements were inconsistent across different rim volumes.

Full size image

The values for COV and CCC for the ONH parameters were as follows: cup area 17.6% and 0.88; cup volume 7.4% and 0.91; average C/D ratio 18.1% and 0.78; rim area 25.3% and 0.69; and rim volume 42.6% and 0.71, respectively. The COV and CCC values for the RNFL parameters were as follows: average 2.1% and 0.98; ITQ quadrant 8.1% and 0.79; INQ quadrant 12.6% and 0.67; SNQ quadrant 7.8% and 0.83; and STQ quadrant 7.8% and 0.88, respectively. There was an overall high agreement and a moderate–substantial concordance between the automated and manual determination methods for both ONH and RNFL parameters.

Discussion

SDOCT is a high-resolution ocular imaging technology capable of providing objective and repeatable measurements of ONH and RNFL parameters.23, 24, 25 The disc margin is anatomically complex and studies have shown that the clinically identified disc margin is often based on more than one tissue structure.26, 27, 28 It has been demonstrated that this variability can lead to a clinical disc margin assessment that may over- and/or underestimate the amount of remaining rim in any given optic disc region.27, 29 SDOCT technology allows cross-sectional visualization of the neural canal opening, which provides a distinct advantage in identifying disc margins compared to traditional stereoscopic examination. This advantage is especially apparent in cases of optic nerve hypoplasia, where clinical identification of the disc margin can be very difficult. These complexities illustrate the utility of accurate and objective disc margin demarcation and this study was undertaken to examine the impact of manual vs automated disc margin delineation on ONH and RNFL parameters obtained using SDOCT.

Two studies have examined the impact of manual vs automated demarcation of RPE/BM end points on ONH measurements using TDOCT.10, 11 Schuman et al10 used two OCT versions to demonstrate a high degree of correlation between automated and manual disc margin identification for disc area, C/D area ratio, cup area, cup volume and rim volume. Using TDOCT, Iliev et al11 found only 10 of 49 disc shapes required no modification, with the majority of disc margin points positioned outside the evident edge of the RPE/choriocapillaris complex, leading to an apparent enlargement of the disc.

Two studies employing SDOCT have also examined manual and automated disc margin delineation. In the current study, we found RNFL parameters to be mainly similar; however, the IN sector found a significant difference (P<0.05) between the two demarcation methods. As we analyzed sectors rather than quadrants, it is difficult to make direct comparison with other similar studies.12, 13 However, our findings tend to agree with those of Garas and colleagues,13 in that the level and the trend of thickness differences were comparable between the two studies. Our findings differ from the results published by Mesiwala et al,12 who showed no significant differences between the demarcation methods for any measurement parameter. This inconsistency may be due to a number of factors, including, but not limited to, the inherent subjectivity in manual RPE/BM endpoint demarcation and software version available at the time of the study. We used a more current software version, which utilized an ONH protocol, rather than the older nerve head map 4mm (NHM4) protocol used in the study by Garas et al13 and Mesiwala et al.12 The ONH and NHM4 protocols use the same scan pattern for ONH topography; however, the ONH protocol yields a larger data set consisting of seven additional circle scans for RNFL thickness measurement compared with the previous software version. Additionally, the significant change found in the IN sector may be attributed to an apparent increased variability of measurements in the nasal region.23, 24, 30 Although not fully understood, Knighton and Qian31 noticed that the nasal reflectance was particularly sensitive to aperture location, and this may account for the high measurement variability in the nasal RNFL region.

For ONH parameters, we found the rim area and all C/D ratios to be significantly different (P<0.05) between the demarcation methods. Overall, Garas et al13 also found significant differences between the manual and automated demarcation methods for several ONH parameters. It is difficult to know whether these findings differ due to the subjectivity inherent in manual disc margin demarcation, method of statistical analysis, patient population, disease severity, image quality, or a combination of these factors.

When we examined the ONH data in more detail, we found that with lower rim volumes (≤0.04 mm3) the automated method produced significantly (P<0.05) smaller measurement values than the manual method, while at higher rim volumes (>0.08 mm3) the automated method produced significantly greater measurement values than the manual method. Although it is not completely understood why this occurs, there are several reasons why nerves with small rim volume may be subject to underestimation by the automated method. SDOCT depends on varying signal intensities to identify specific structures. The RPE/BM complex algorithm may inaccurately identify the RPE/BM end point when this point is in close proximity to the bright reflectance produced by the cup border, when the border tissue of Elschnig/Bruch’s membrane extends past the RPE, or when the signal intensity is diminished from shadows generated by overlying vessel, floaters, or vitreous opacities. The reason for overestimation of large rim volume is more difficult to explain, when, theoretically, the RPE/BM end point should be easily identified when further from the cup border, although border tissue variability and artifact such as shadows may also significantly affect RPE/BM endpoint identification in this situation as well.

Our study has potential limitations. We included a relatively small number of PG, GSPPG, and normal eyes and did not have a second operator for inter-operator comparison of manually identified RPE/BM end points. Our participants consisted of all AIGS subjects with study visits between July and September 2011. Due to the asymmetric sample size between groups, we did not make inter-group comparisons. The repeatability (COV) of ONH and RNFL parameters has been studied previously and has been reported to be between 1.3% and 5.69%, respectively;25 however, the repeatability of the manual RPE/BM endpoint demarcation was not examined in this study. Finally, few glaucoma subjects had advanced disease and only three eyes had tilted discs, so the results of this study may not be generalizable to these populations. Future studies may be warranted to examine what ocular characteristics lead to automated RPE/BM endpoint misplacement, to what degree this change affects clinical interpretation of SDOCT reports, to examine disc delineation in experienced vs less experienced operators, and to compare optic disc assessment by SDOCT vs expert evaluation of optic disc photographs.

In conclusion, there was a high agreement and a moderate–substantial concordance between automated and manual determination of RPE/BM end points for RNFL parameters. The greater variation in ONH parameters compared with RNFL thickness parameters represents additional evidence that RNFL thickness parameters are more robust than ONH parameters for the consistent follow-up of glaucoma patients. Clinicians should be aware of the potential variability in these measurements and operators need to inspect images for the accurate placement of the RPE/BM end points during the analysis of the image.