Reply

Sir,

We read with interest the letter by Choudhary and Kyle concerning the use of potentially flawed methodology in the ascertainment of vision loss in the better eye in unilateral amblyopes. Their letter questions the reliability of ascertainment using active surveillance employed by the British Ophthalmological Surveillance Unit. In comparative trials this methodology has been shown to identify significantly more cases than any other method.^{1, 2}

However, it is widely recognised that in even apparently exhaustive epidemiological studies, including those undertaken through active surveillance schemes employing a well-established methodology, complete ascertainment of all eligible cases is rarely achieved.³ However, few investigators consider or report the level of ascertainment in their studies,³ and we agree that it is important. Indeed, some years ago, we reported our application of capture–recapture analysis,^{4, 5} novel in ophthalmology at the time, to assess completeness of ascertainment in a study of congenital cataract in which cases were identified through national active surveillance schemes similar to BOSU.⁶

Capture–recapture analysis is known to be the only truly valid way of estimating ascertainment. Its application requires two or more independent sources of cases, individual matching of cases reported through each source and equal probability of ascertainment of given case by each source. In the absence of a truly independent second source of cases, we were unable to apply capture–recapture analysis in our study of the incidence and causes of visual impairment arising from loss of vision in the nonamblyopic eye, published last year in the Lancet.⁷ However, we did cite the work by the BOSU Steering Committee, recently published in Eye,⁸ comprising a survey of respondents and independent modelling of observed cases and the populations served by BOSU respondents, that suggested ascertainment in our study was of the order of 70%. We speculated that individuals with treatable disorders affecting their nonamblyopic eye, such as cataract, may have been less completely ascertained than those with untreatable disorders. We emphasised that we were reporting minimum estimates of incidence.⁷

Choudhary and Kyle assert that ascertainment in our study was only 37%, and speculate that the level of ascertainment in other BOSU studies may be similarly low. Their view is based on flawed reasoning and erroneous calculation. They have applied an estimate of the prevalence of amblyopia in the general population in Belgium to the incident population of individuals certified as partially sighted or blind in Britain each year, to calculate that 450 people per year who are registered partially sighted or blind also have amblyopia. They assume that this must therefore be the number of people with visual impairment following loss of vision in their nonamblyopic eye who should have been identified in our study, instead of about 185 per year (ie 370 cases in 24 months of surveillance we reported). This is a questionable approach based on a number of unprovable assumptions: notably amblyopia is mentioned as a cause in about 100 individuals certified as blind or partially sighted each year.⁹ Even if the approach were sound, their calculation contains a number of errors. Firstly, the prevalence estimate of amblyopia of 1.5% they apply refers to the prevalence of amblyopia with acuity of LogMAR 0.3 or worse (6/12 or worse) in the amblyopic eye. However, our eligibility criterion was an acuity of worse than 0.3 (ie 6/18 or worse)⁷—the prevalence of this level of amblyopia is likely to be no more than 1%, based on recent population-based studies in the UK.¹⁰ Thus, using their approach and applying a prevalence of 1%, at most 300 individuals with amblyopia would be anticipated to be certified as partially sighted or blind each year. Critically, however, only the subset of these people with visually impairing disease affecting their nonamblyopic eye only would have been eligible for our study. As we reported,⁷ those with other disease affecting their amblyopic eye were ineligible, and if notified were excluded, as our aim was to identify only those individuals who, if they had had a more favourable outcome in their amblyopic eye, would not have been rendered visually impaired by the loss of the fellow eye. Most visually impairing diseases tend to affect both eyes. Thus it is likely that of the possible 300 individuals per year described earlier, the majority would have additional disease in their amblyopic eye, rendering them ineligible for our study—but even assuming only 50% were ineligible, then at most 150 cases per year should have been identified in our study. Finally, as we reported,⁷ we used only cases identified in a 24-month period (using data when reporting had achieved a ‘steady state’, as is conventional with BOSU studies), while Choudhary and Kyle's estimate is based a time period of 30 months.

We reiterate our view that ascertainment in our study was about 70% complete. While this is not ideal, experience from longer-established national surveillance schemes suggests that this is the order of ascertainment one might expect in the early years of such a scheme.¹¹

The critical question one should ask about any study is whether the findings are sufficiently robust and novel to be of value to clinical practice or policy. The minimum incidence of visual impairment arising from loss of vision in the nonamblyopic eye we reported⁷ is higher than previously reported and indicates it is an important public health issue. The findings have already influenced the considerations of the UK National Screening Committee about childhood screening for amblyopia.

We hope the very large number of ophthalmologists who support BOSU, including those who contributed specifically to our study, will be reassured about the quality and value of work undertaken through BOSU.

Far from employing ‘suspect methodology’, BOSU uses a well-established approach to provide a unique and powerful resource for the epidemiological study of uncommon ophthalmic disorders, which is envied outside the UK. The BOSU ensures that an evaluation of ascertainment is included in the study methodology and reported as part of the findings. In time, the studies undertaken through it can be expected to contribute a significant body of evidence on which clinical practice and policy will be based—as the example of the British Paediatric Surveillance Unit, now in its 17th year and on which BOSU is modelled, so clearly shows.¹¹ It would be a great pity if BOSU were prevented from fulfilling this potential role in ophthalmology in the UK.