Introduction

The diagnosis and management of glaucoma depends on evaluation of the optic nerve head (ONH), thus rendering a reproducible method of assessing the ONH indispensable to the ophthalmologist. In the clinical setting, slit-lamp biomicroscopic estimation of the optic cup-to-disc ratio (C/D ratio) remains the most frequently used method of ONH evaluation in glaucoma or suspected glaucoma. Newer and more objective techniques with excellent reproducibility, such as stereoscopic optic disc imaging1 and confocal scanning laser ophthalmoscopy,2 are not routinely employed in the typical clinical setting. Consequently, the consistency of clinically documented C/D ratio measurements, and other ONH parameters relevant to the glaucoma evaluation, is important.

Several studies have aimed to examine the intra- and inter-observer variability in the subjective estimation of the C/D ratio.3, 4, 5 Good intraobserver agreement (the consistency of an observer's evaluation recorded during separate sessions) is crucial in the monitoring of glaucoma in all settings, whereas interobserver agreement (the consistency of separate observers' evaluations) becomes important where the glaucoma patient, or suspect, may be evaluated by a different observer on successive visits.

Kirwan et al6 investigated whether pharmacological mydriasis influenced interobserver agreement in ONH evaluation, and concluded that examination without such pupillary dilation markedly impairs interobserver agreement of the C/D ratio. In that paper, each patient was randomly assigned to pupillary dilatation or not, and then examined by both observers on a single visit. Consequently, intraobserver agreement was not assessed.

We report the results of a study designed to investigate whether pharmacological mydriasis influences firstly, interobserver agreement and secondly, within-observer agreement before and after pupil dilation, in the evaluation of the ONH in the context of the patient with glaucoma or suspected glaucoma.

Materials and methods

Baseline agreement

The first part of this study was designed to give baseline inter- and intra-observer agreement data for two observers. In order to estimate this, both observers graded monoscopic optic disc photographs of 29 eyes on two separate occasions. These photographs were taken from patients sent from a general clinic for ‘baseline optic disc photography’. Each examiner had a minimum of 5 years' experience in clinical ophthalmology. Each commented on the following ONH parameters: horizontal C/D ratio; vertical C/D ratio, and gave a judgement on whether or not the optic disc was pathologically cupped. Each observer was masked to clinical data, his previous responses, and the responses of the other observer. The observers were encouraged to estimate the C/D ratio to the nearest 0.05. No grading aids such as graticules were used by either examiner.

Influence of pharmacological mydriasis

In the second part of the study, both observers evaluated the above parameters in 53 eyes of 53 patients on two occasions, the pupil being dilated for only one of these visits. In addition to the above parameters, the observers also commented on the presence or absence of the following variables: laminar dots; disc margin haemorrhage; disc saucering; disc notching (or localised thinning of the neuroretinal rim), and peripapillary atrophy (PPA). Subjects were enrolled in this study after giving written and informed consent to participate and the Tenets of the Declaration of Helsinki were adhered to. Exclusion criteria comprised previous ocular surgery in the study eye, and any optic disc anomaly such as a tilted or highly myopic disc.

One eye (the study eye) of each subject was randomly assigned to receive pharmacological mydriasis (1% tropicamide) on either the first or the second visit. The subjects were examined by both examiners on a single occasion with mydriatic state (undilated or dilated) determined by randomisation and then re-examined up to 2 weeks later in the opposite mydriatic state, that is, without mydriasis if mydriasis had been used before, or with mydriasis if that had not been used for the initial examination. The same slit-lamp (Haag-Streit®) and the same indirect biomicroscopic 90 dioptre lens (nontinted, Volk®) were used by both observers on each occasion. An independent study coordinator recorded the pupil diameter using the slit-lamp on both clinic visits (undilated and dilated), and randomly selected the study eye based on the toss of a coin.

One observer (SB), who was experienced in cataract grading for research purposes, graded the lens in each study eye under conditions of pharmacological mydriasis using the Lens Opacification Cataract System (LOCSIII™). Each observer was masked to previous estimates by either observer, and to the clinical details.

Statistical analysis

Bland–Altman plots were generated to examine agreement between and within observers for the estimation of the C/D ratio by plotting the difference between the readings of two observers against mean values, or between readings for each observer under dilated and undilated conditions against mean values, respectively. The F-test was used to determine whether the variances were significantly different.

The unweighted kappa (κ) was used to evaluate agreement, between and within observers, in the assessment of the presence or absence of: laminar dots; disc haemorrhage; disc saucering; disc notching; PPA, and glaucomatous damage. The kappa statistic is used with categorical data and expresses the agreement between two measures corrected for chance agreement. The value of kappa ranges from −1 (complete disagreement) to 0 (chance agreement) to +1 (perfect agreement). Agreement was classified according to the kappa values by which it was represented, as follows: −1.0–0.0 (poor); 0.01–0.20 (negligible); 0.21–0.40 (passable); 0.41–0.60 (moderate); 0.61–0.80 (good); 0.81–1.0 (excellent). 95% Confidence intervals are presented for all kappa values. Differences were considered to be statistically significant when the 95% confidence intervals of κ of the two groups of observers did not overlap. Sample size was estimated using a proportions method, and gave sufficient power to detect a reduction in good agreement from 90 to 65%, with an alpha error of 5% and a beta error of 20%. To assist comparison with previous work, a good degree of agreement was defined as a difference of 0.1 or less for C/D ratio estimation. The proportions achieving such levels of agreement, between and within observers, were compared using the χ2-test. PEPI version 4.0 was used for kappa calculations. SAS version 8 was used for other calculations.

Results

Baseline agreement

Intraobserver agreement of C/D ratio estimates of 29 optic disc photographs, recorded on two separate occasions showed that observers were statistically similar in terms of intersessional (ie difference in results from a single observer on two occasions) consistency of VCDR and HCDR estimation (P=0.0741 and 0.2589, respectively). Each observer showed a high proportion of vertical and horizontal C/D ratio estimates (86–90%) with ⩽0.1 intersessional discrepancy. There was no statistically significant difference between observers in terms of the proportions of ONHs examined where the VCDR and HCDR estimates did not differ by >0.1 (P=0.68 and 1.0 respectively). Intraobserver agreement regarding the pathological nature of the ONH was good for observer 1 (κ=0.75) and moderate for observer 2 (κ=0.52). Interobserver agreement was moderate for categorisation of the ONH as pathological (κ=0.46).

Influence of pharmacological mydriasis

The male : female ratio for the 53 volunteers enrolled in the study was 28 : 25, with a mean (±SD) age of 70±15 years (range 27–88 years). Two additional patients were excluded from the study as their fundal view was insufficient to allow grading of the C/D ratio, due to the presence of very dense cataracts. The majority of the patients (52 of 53) in this study were Irish Caucasians. The mean (±SD) pupil diameter, before and after mydriasis, was 2.77±0.81 mm (range 0.9–4.5) and 6.56±1.08 mm (range 3.5–8.0), respectively. Mean (±SD) of LOCS III grading of nuclear opalescence, posterior subcapsular cataract and cortical cataract was 2.0±0.9, 0.9±0.5, and 1.0±0.6, respectively. As judged by the LOCS III and clinical grading, most patients had mild degree of cataract, usually of the nuclear type. The mean (±SD) and median estimate of vertical and horizontal C/D ratios before and after dilation ranged from 0.53 to 0.59 (±0.24–0.26) and 0.5 to 0.7, respectively, for both observers. The mean vertical and horizontal C/D ratio was marginally greater (by a value of 0.03–0.04) after dilation, as judged by each observer. The differences in the C/D ratio estimates with and without pharmacological mydriasis were not statistically significant.

The results for agreement within (before and after dilation) and between (inter-) observers for the assessment of the vertical and horizontal C/D ratio are shown in Table 1. Within-observer agreement across mydriatic states was similar for each observer with regard to VCDR and HCDR. Interobserver agreement for VCDR and HCDR were statistically similar in the dilated and undilated states. Bland–Altman plots for VCDR estimates, between and within observers, in the dilated and undilated states, are displayed in Figures 1 and 2. Interobserver agreement was also examined in both mydriatic states by estimating the proportion of examinations achieving high levels of intersessional agreement (difference in C/D ratio ⩽0.1) for the vertical and horizontal C/D ratio (Table 2). There was no statistically significant difference between proportions in either mydriatic state.

Table 1 Within-observer (before and after dilation) and interobserver agreement of C/D ratio estimates
Full size table
Figure 1
figure 1

(a) Bland–Altman plot of interobserver agreement for VCDR without mydriasis. (b) Bland–Altman plot of interobserver agreement for VCDR with mydriasis. Obs=observer.

Full size image
Figure 2
figure 2

(a) Bland–Altman plots of within-observer agreement before and after mydriasis (Observer 1). (b) Bland–Altman plots of within-observer agreement before and after mydriasis (Observer 2).

Full size image
Table 2 Proportions of eyes examined clinically, for each observer, where high levels of intersessional agreement for vertical (VCDR) and horizontal (HCDR) cup/disc ratio estimates (intersessional difference of ⩽0.1) were achieved
Full size table

Table 3 shows the kappa agreement statistics for each of the different variables examined. Agreement was moderate to good for assessment of the ONH for the presence or absence of: laminar dots (κ=0.41–0.45 for interobserver agreement and κ=0.58–0.69 for within-observer agreement across mydriatic states); PPA (κ=0.36–0.45 for interobserver agreement and κ=0.4–0.54 for within-observer agreement); and pathological cupping (κ=0.59–0.68 for interobserver agreement and κ=0.46–0.48 for within-observer agreement). For several other ONH variables such as the presence or absence of disc margin haemorrhages, saucering, or disc notching, the agreement was not as good.

Table 3 Interobserver and within-observer (before and after dilation) agreement (κ) for ONH parameters other than C/D ratio
Full size table

Discussion

The current study has compared interobserver agreement and within-observer agreement (before and after mydriasis) of ONH evaluation under conditions of pharmacological mydriasis and nonmydriasis.

Baseline agreement based on grading optic disc photographs

First, we demonstrated good intraobserver agreement in assessing vertical and horizontal C/D ratios using monoscopic optic disc photographs. More than 86% of examinations within observers across two sessions demonstrated high levels of agreement when measuring both the vertical and horizontal C/D ratio. Intraobserver agreement was moderate to good for assessing the pathologic nature of the ONH and interobserver agreement was moderate for the same parameter. Intraobserver agreement was therefore noted to be better than interobserver agreement when deciding whether or not the optic disc was pathologically cupped. These findings confirm that both examiners showed similar levels of intraobserver agreement when assessing the optic disc for the purposes of determining the influence of pharmacological mydriasis on ONH evaluation, and parallel a common observation from other studies, which demonstrate that the reliability in assessing the ONH is higher within observers than between observers.7, 8

Influence of mydriasis on clinical grading of the ONH

We have shown that the instillation of mydriatic drops prior to examination of the ONH did not result in higher levels of agreement either within or between observers, and that pharmacological mydriasis did not significantly influence vertical or horizontal C/D ratio estimation; nor did it influence the proportion of C/D ratio comparisons with a high level of agreement. Our Bland–Altman figures indicate a good degree of interobserver and within-observer agreement when assessing the vertical C/D ratio, and suggest that routine pharmacological mydriasis is not essential in monitoring the glaucoma suspect or patient in the presence of a satisfactory view of the ONH. These findings differ from those of previous investigators.6 Our study group was made up almost entirely of patients from the same ethnic group, which may help explain our higher number of C/D ratio estimates with ⩽0.1 difference noted during undilated examinations (85% compared with 72–77% in Kirwan's paper). Additionally, lens opacity among our volunteers was mild, but was not evaluated by Kirwan et al, again possibly contributing to the discrepancy in the findings of the two studies in this respect because grading the C/D ratio through an undilated pupil in the presence of more significant cataract may have accounted for their lower levels of interobserver agreement. The proportion of examinations showing high level of agreement in our study in dilated patients (81–89%) was, however, lower than Kirwan's study (97%). Two possible reasons for this discrepancy exist. Firstly, several patients in their study were given phenylephrine 10%, if judged necessary clinically, and secondly, two of their three graders had glaucoma subspecialty training and were therefore experienced graders, whereas both observers in this study had at least 5 years of experience in ophthalmology, but neither had a subspecialty interest in glaucoma. Finally, our study population included a greater number of glaucoma suspects than Kirwan et al's sample, probably because their subjects were randomly selected from a cohort of patients attending either a general or a glaucoma clinic, whereas most of our subjects were ‘glaucoma suspect’ referrals.

The ONH parameters represented by the highest levels of agreement in our study were the presence of laminar dots, peripapillary atrophy and glaucomatous cupping. It is important to re-emphasise that the observers were able to meaningfully assess the ONH parameters in all study patients, because those with a very poor or no fundal view were excluded (n=2) prior to entering the study, as we feel that these patients should always be examined with mydriasis in order to optimise the view. Contrary to expectation, interobserver agreement regarding the pathologic nature of the ONH was higher for each mydriatic state (dilated and undilated) than within-observer agreement across mydriatic states. This finding emphasises the importance of documenting whether or not the patients' eye was dilated at the time of each examination if within-observer agreement is to be optimised.

Comment

Our data suggest that, in the presence of a fundus view sufficient to grade the C/D ratio, pharmacological dilatation does not enhance agreement within or between observers. This finding is important in the monitoring of glaucoma patients where pupillary dilatation may be contraindicated, or if the patient is unwilling to undergo pharmacological mydriasis. Of interest, another study9 has shown that when confocal scanning laser ophthalmoscope images are of good quality, topographic optic disc parameter measurements obtained with a dilated pupil are similar to those obtained with an undilated pupil.

Obviously, our study has only addressed examination of the ONH, which is most relevant for assessment of glaucoma patients and suspects. We did not aim to assess the integrity of the retinal nerve fibre layer, as this is not routinely documented by most general ophthalmologists. Mydriatic examination is certainly warranted when a comprehensive peripheral retinal examination is required. We also routinely dilate any patient in whom macular pathology is suspected. Less-experienced observers may still require the use of dilating drops to facilitate a reliable ONH examination. Avoidance of routine pharmacological mydriasis offers several advantages to patients and the healthcare provider. First, it permits patients to drive to and from the clinic, which may promote attendance. Second, clinicians will need to see the patient only once during the visit, following assessment of visual acuity and visual field, thus enhancing clinic efficiency.

In conclusion, interobserver agreement of ONH parameters is not compromised by evaluation under conditions of nonmydriasis if the clinician can achieve a satisfactory view of the optic disc in the undilated state.