Introduction

Age-related macular degeneration (AMD) is the leading cause of visual loss in the industrialised world1, 2 and is characterised by a decline in central vision.1 With increasing treatment options available in the form of anti-vascular endothelial growth factor (anti-VEGF) injections for neovascular AMD, the importance of detecting and monitoring this change is more important than ever. Current National Institute for Healthcare and Clinical Excellence guidelines published in July 2013 (http://www.nice.org.uk/guidance/TA294) and the September 2013 Royal College of Ophthalmologists guidelines (http://www.rcophth.ac.uk) state that patients with logMAR visual acuity (VA) ranging between 0.3 and 1.2 logMAR are eligible for treatment with anti-VEGF therapy. In addition, a reduction in VA of more than five letters (0.1 logMAR) is a part of the criteria used to determine re-treatment.3, 4 This necessitates that logMAR based charts are employed and the Early Treatment Diabetic Retinopathy Study (ETDRS) chart is currently considered the gold standard test for measuring and monitoring VA in AMD patients.5 However, there is a recognised difficulty in introducing and employing standardised logMAR based acuity measurements.6, 7 Difficulties in standardising testing procedures including pass–fail criteria8 and scoring methods,9 render optimum use of hard-copy logMAR charts challenging in a busy clinical environment.

Furthermore, as a result of the increased number of letters on an ETDRS chart, acuity measurements take twice as long as Snellen chart measurements with median test times of 60 vs 30 s, respectively being reported.6 In order to address this several groups have looked at the performance of ETDRS charts with different test procedures. Camparini et al10 have described a psychophysical adaptive method termed ETDRS-fast in order to achieve shorter test times. Rosser et al11 previously investigated the effect of using a three-letter per line acuity test, the reduced logMAR (RLM) charts on measurements compared with the ETDRS chart in subjects with cataract, pseudophakia, or early glaucoma. Laidlaw et al6 developed the RLM chart further by reducing the inter-letter spacing to half a letter width in their more compact RLM chart. Both studies found good agreement with gold standard acuity measurements and a halving of the test time, but at the price of a slight increase in test–retest variability (TRV). Noushad et al7 found similar TRV values using a modified three letters per line logMAR chart compared with a standard five letters per line chart (±0.10 vs ±0.08 logMAR, respectively) but with a 30% saving in testing time.

LogMAR visual acuity measurements are recognised as being far superior to Snellen chart measurements in terms of precision and repeatability, and testing with these charts would become far more familiar with increased usage.12 Over the years, a number of computerised VA measurement systems have been developed as an alternative to standard printed acuity charts in an attempt to semi-automate the measurement process13, 14, 15 and have proposed a number of advantages over the ETDRS chart. Measurement rigour is better controlled and standardized. COMPlog incorporates an automated calculation system, which can present VA test scores in any of the routinely encountered acuity formats (decimal, UK and US Snellen; decimal logMAR and number of ETDRS letters read), thus minimising training and technician based errors. It also allows the full clinically encountered VA range to be measured from one test distance (3 m) of −0.30 to 1.68 logMAR acuity. The random generation of Sloan16 letters has also been proposed an advantage by avoiding memorisation effects, which may be a limitation encountered with the ETDRS chart for patients requiring numerous VA assessments in monitoring disease progression and treatment effects. The COMPlog computerised clinical visual acuity measurement system has been developed by our group for routine clinical and research use, and has been validated as providing comparable VA measurements to the ETDRS chart in a number of ophthalmic disease states,13 with similar TRV and testing times. We have previously investigated the effect of letter separation on VA in a group of clinical subjects including normal subjects and those with a range of ocular diseases. We determined that half-letter spacing allowed letters at the poorer end of the VA range to fit on a standard size secondary viewing monitor without introducing bias compared with ETDRS chart measurements.17

In AMD, fixation characteristics18 significantly differ from those in normal subjects, with eccentric fixation often adopted using a preferred retinal locus.19, 20 This may render patients with AMD more susceptible to the influence of letter spacing and the number of letters presented per line. To our knowledge this has not been investigated with letter recognition tasks in logMAR-based distance VA measurements.

The aims of this study performed on patients with AMD were:

  1. 1)

    To investigate whether the half-letter width spacing employed in COMPlog algorithms had an adverse effect on agreement, TRV, and test time in patients with AMD compared with gold standard single-letter scoring acuity measurements taken using an ETDRS chart.

  2. 2)

    To investigate the effect of presenting COMPlog three letters per line on agreement, TRV, and test time in patients with AMD compared with both ETDRS chart and COMPlog five letters per line measurements.

Materials and methods

Fifty adult subjects (35 female and 15 male) who satisfied the inclusion criteria were invited to participate from an outpatient AMD clinic at St Thomas’ Hospital, London. The median age was 86 years (interquartile range 78–89 years). Ethical approval for this study was granted by London-Westminster Research Ethics Committee, and informed consent was obtained from each subject. We certify that all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during this research.

Each subject underwent six VA measurements. Two of these were taken using the ETDRS charts 1 and 2 (Lighthouse Low Vision Products, Long Island City, NY, USA), and the remainder using both the five and three letters per line assessment setting, tested twice, on the COMPlog system (COMPlog Clinical Vision Measurement Systems Ltd, London, UK). These measurements are referred to as ETDRS1, ETDRS2, C51, C52, C31, and C32, respectively.

The inclusion criteria for this study included a visual acuity of better than count fingers and a diagnosis of AMD as made at their clinic appointment by the clinician using fundus examination and Optical Coherence Tomography with no other reported ocular comorbidities in the test eye. Forty four subjects had a diagnosis of neovascular or wet AMD defined as the presence of fluid, exudates and/or blood in the sub-retinal space and/or in the sub-retinal pigment epithelial space in the test eye. Six had a diagnosis of non-exudative or dry AMD with either soft indistinct drusen, or hard or soft drusen plus pigmentary changes or geographic atrophy without any signs of exudative AMD. Each subject underwent timed test and retest VA measurements of the eye being treated in the AMD clinic. In order to increase the range of VA measured at the poorer end, the fellow eye was used if this had poorer than 1.20 logMAR, but better than count fingers visual acuity owing to AMD only (ie, outside the treatable range as determined at their clinic appointment).

All measurements were taken in a random sequence to control for fatigue and learning effects, with the random sequence being generated by research randomizer (http://www.randomizer.org). Subjects wore their habitual spectacle correction with their fellow eye occluded. All measurements were conducted by two trained examiners (YB and NS) and a forced choice procedure was employed where the subject was requested to guess each letter if they were unsure of the identity until the full termination criteria of a whole line read incorrectly was met for each test.8 Letter presentation and viewing times were not restricted either on the ETDRS chart or COMPlog system.

ETDRS charts 1 and 2 were used and displayed in the standard Lighthouse Low vision Products light box and were read from a distance of 4 m. The chart luminance was measured to be 181 cd/m2, which is in compliance with recommendations for the standardisation of VA measurement. The subject was requested to begin at the top of the chart and encouraged to read each letter, guessing when they were unsure until a whole line of letters were read incorrectly. Responses were marked as either correct or incorrect for each letter on specially designed proforma and the final VA score was calculated in logMAR terms with each letter being assigned a value of 0.02 logMAR. If fewer than five letters were read correctly from this test distance, testing was repeated at a distance of 1 m (a common procedure followed in clinical studies) with the score adjusted accordingly.21

COMPlog is a computerised VA measurement system consisting of a PC capable of running Microsoft Windows XP or subsequent operating systems, a 21.3-inch 1600 × 1200 pixel resolution LCD flat panel secondary monitor which was measured to have a luminance of 255 cd/m2 with Weber contrast of letters −99%, and the COMPlog software programme running within the Microsoft dotnet framework. The programme consists of an algorithm involving ‘range finding’, and ‘thresholding’ as has been described in detail in previous publications.13, 22 The range finding component essentially involves the presentation of sequentially smaller, single-crowded Sloan letters to determine the approximate threshold acuity. This is followed by the ‘thresholding’ phase in which lines of Sloan letters surrounded by a crowding box are presented. In this study, thresholding commenced four logMAR lines above the incorrect range finding score. In the event of letters being incorrectly identified on the first thresholding line, successively larger lines are presented in 0.1 logMAR steps until an entire line is correctly read at which point the programme then reverses and descends to threshold. Lines of each size, however, are presented only once. With a single viewing distance of 3 m and a 21.3-inch monitor, a letter acuity range of between −0.30 and 1.68 logMAR (100–1 ETDRS letters, 6/3 to 1.5/71 UK Snellen) can be measured without having to move the patient. Responses were entered as either correct or incorrect by the examiner. The full history of letters displayed and response given is displayed on the PC monitor visible to the examiner with the option to ‘undo’ if the incorrect response is accidentally recorded.

In this study, two different COMPlog measurement algorithms were employed. In the first, referred to as C5, single rows of five letters per line spaced half a letter width apart and surrounded at the same separation by a crowding box of one stroke width thickness were presented during the thresholding phase.17 The second, referred to as C3, had similar parameters, the only departure being the presentation of three instead of five letters per test line. The termination criterion was set at all letters on one line being incorrectly identified. Once the termination criterion had been met, the test automatically terminated with calculation and presentation of a ‘fully interpolated’ logMAR acuity score and its Snellen equivalent in selected formats. The interpolation gives credit for each individual letter correctly read.

Test times were measured using a stop clock with timing started as soon as the ETDRS chart was presented or the first letter in the ‘range finding’ part of the COMPlog test was displayed on the secondary monitor and terminated when a whole line was read incorrectly. ETDRS chart result calculation times were also included.

Statistical analysis

Bland and Altman23 methods were employed. Scatter plots were created in which the average of the two acuity scores for each test was plotted against the difference between the two scores to ensure that there was no systematic effect of score magnitude on measurement error. The normality of the distribution of the differences scores was assessed using the Shapiro–Wilk W-test. Calculations to quantify (1) bias (mean and 95% CI of the mean) between ETDRS, and both C5 and C3 assessments and (2) TRV for each test pair expressed as 95% limits for agreement (mean ±1.96 SD) were made. The Wilcoxon signed-ranks test was used to compare testing times.

Results

The median of the first ETDRS chart acuity scores (ETDRS1) was 0.34 logMAR (interquartile range 0.18–0.55 logMAR).

Bland–Altman scatter plots of the observed difference plotted against the average of the test and retest measurements were constructed to represent the graphical spread of each six pairs of measurements. These were: ETDRS1 vs ETDRS2, C51 vs C52, C31 vs C32, ETDRS1 vs C51, ETDRS1 vs C31, and C51 vs C31. The first three demonstrate the TRV of each measurement technique (Figure 1a–c). The latter three are method comparison studies (Figure 2a–c) and give information on agreement. In each case, the mean difference and upper and lower 95% limits of agreement are plotted as dotted lines. The data sets were examined using the Shapiro–Wilk W-test (P=0.052 for ETDRS1 and ETDRS2, P=0.439 for C51 and C52, and P=0.412 for C31 and C32) and on this basis, Bland and Altman summary statistics of mean bias and 95% limits of agreement were calculated and are presented in Table 1. The results suggest no clinically significant systematic or proportional bias between ETDRS and COMPlog measurements and similar TRV values.

Figure 1
figure 1

(a–c) Bland–Altman plots for test and retest measurements for: (a) ETDRS, (b) C5, and (c) C3 acuity measurements. In each instance, the mean difference and upper and lower 95% limits of agreement are plotted.

Full size image
Figure 2
figure 2

(a–c) Bland–Altman plots for method comparison studies between: (a) ETDRS and C5, (b) ETDRS and C3, and (c) C5 and C3 acuity measurements. In each instance, the mean difference and upper and lower 95% limits of agreement are plotted.

Full size image
Table 1 COMPlog and ETDRS test–retest and inter-test agreement: results from 50 subjects with AMD
Full size table

Median (interquartile range) test times for ETDRS, C5, and C3 measurements were 96 (71–125), 105 (71–143), and 65 s (54–91), respectively. A Wilcoxon signed-ranks test using a simple Bonferroni adjusted alpha level of 0.017 indicated no significant difference in test times between ETDRS chart measurements and COMPlog five letters per line measurements (Z=−1.960, P=0.051). COMPlog three letters per line VA measurements were significantly faster when compared with the gold standard ETDRS chart (Z=−5.034, P=0.000).

Discussion

The aims of this study were to investigate the effect on agreement, TRV, and test time with the gold standard ETDRS chart of employing a half-letter width spaced computerised acuity chart algorithm using both three and five letters per line in patients with AMD.

The benefits of using computerised tests compared with hard-copy test charts are recognised13, 14, 24 with the ability to have more standardised measurements. In addition, the ETDRS chart in which letters are spaced a letter-width apart, is only able to measure up to 1.00 logMAR VA from a 4 m test distance. This means that either the patient or the chart has to be moved to a shorter test distance if their VA is worse than this. In a previous study,17 we looked at the effect of letter spacing in normal subjects and subjects with a range of ocular diseases including amblyopia on a computerised test in order to be able to measure the fully encountered clinical VA range from one test distance without having an adverse effect on agreement with the gold standard ETDRS chart as a result of increased crowding. However, central fixation compromise and the adoption of eccentric fixation using a preferred retinal locus20 in patients with AMD mean that specific testing and validation in this group is required.

No systematic or proportional bias was evident between ETDRS chart measurements and either C5 or C3 measurements suggesting that the closer letter spacing and the higher luminance on the computerised test measurements has no effect on VA measurements in this group of AMD subjects. The ETDRS chart and C5 assessment were both found to have the same repeatability values with TRV scores of ±0.13 logMAR each. These values are similar to those found in a previous study comparing ETDRS and C5 (±0.12 logMAR for both tests) in the assessment of VA in 59 amblyopic children.13 The values observed in our study are also comparable to data reported in the literature, where values have been reported to a range between ±0.07 and ±0.20 logMAR.6, 9, 11, 14, 25, 26, 27, 28 As mentioned earlier, the recommendations for re-treatment with anti-VEGF therapy include a reduction in VA of more than five letters. Cousens et al29 created a simple model to predict the sensitivity to change of visual acuity measurements. They predicted a sensitivity of ~37% only in detecting a true change of more than five letters with a TRV of ±0.13 for ETDRS and C5 measurements. C3 measurements were found to induce only a slight increase of two letters in TRV (0.17 vs 0.13 logMAR).

Median test times with the C5 assessment were found to be slightly slower than ETDRS chart measurements (105 vs 96 s), although this did not reach statistical significance. However, the calculation of single letter scoring ETDRS letter scores from a hard-copy scoring sheet is prone to error and typically takes 30 s. The scoring in COMPlog is automated and immediate. The closer crowding of letters on the computerised chart may offer an explanation to this and the ETDRS chart scores were calculated by someone very familiar with logMAR scoring. This may not necessarily be the case in routine clinical practice.12 C3 measurements were found to offer half a minute saving in test times (median test time 65 vs 96 s). Other groups have demonstrated similar findings using three vs five letters per line chart designs with small increases in TRV but significant test time savings.6, 7, 11 A survey by the Royal College of Ophthalmologists and the Macular Society (AMD Services Survey 2013 Report, September 2013) identified insufficient clinic time and support staff shortages as key barriers to providing good or excellent services with 56.3% of services running extra clinics to meet demand. The shorter test times achieved with C3 measurements may therefore be desirable when considered in light of the only slight reduction in TRV compared with the manually scored ETDRS charts that have not been universally adopted in practice.30 Investigation of means of reducing test times with the COMPlog system by altering the testing algorithm is ongoing. For example, by reducing the jump back between range finding and thresholding, it may be possible to achieve even shorter test times than the ETDRS chart.

With the highly comparable reliability of C5 to the gold standard ETDRS assessment, COMPlog may provide a suitable alternative to the ETDRS chart in patients with AMD and with three letters per line testing (C3), provide a significant test time saving advantage.