Clinical Study | Published:

Can a community optometrist-based referral refinement scheme reduce false-positive glaucoma hospital referrals without compromising quality of care? The community and hospital allied network glaucoma evaluation scheme (CHANGES)

Eye volume 24, pages 881887 (2010) | Download Citation



To describe the design, activity, and quality of the referral refinement phase of a novel glaucoma shared-care scheme.


Eight Optometrists with a Specialist Interest in glaucoma (OSI) were trained to perform a community-based comprehensive glaucoma evaluation of low-risk glaucoma hospital referrals (only one/none of the following factors noted for either eye: abnormal optic disc, abnormal visual field, abnormal intraocular pressure (IOP; 22–28 mmHg or IOP asymmetry)) using equipment standardized to that of the hospital glaucoma service.


One hundred and thirty-eight (27%) of a total of 512 glaucoma-related referrals were deemed ‘low risk’. Their choice of OSI discharged 40 (35%). The consultant agreed (virtually) with the decision to discharge with 28 (70%) and disagreed with 12 (30%). Comparing findings between OSI and consultant for 99 referred patients, sensitivity, specificity, and negative predictive values for a suspicious optic disc were 78, 61, and 79%, respectively. For an IOP of >21 mmHg, they were 74, 85, and 90%, respectively. For an occludable anterior chamber angle (Van Herick's versus gonioscopy), they were 69, 88, and 94%, respectively.


This referral refinement process can reduce numbers of false-positive referrals attending the hospital glaucoma service while retaining a relatively high level of examination quality.


Healthcare in the United Kingdom is undergoing major change, particularly with regard to altering the traditional boundaries of primary and secondary care. The white paper ‘Our Health, Our Care, Our Say’1 set out objectives to develop services in settings more convenient and accessible to patients and recent policies aim to expand the range of community eye services.2, 3 Traditionally, all referrals for glaucoma from optometrists or other primary care providers are offered a hospital-based appointment. The UK government has invested in several pilot projects to assess potential alternative glaucoma pathways but there is still a paucity of objective evidence for purported advantages such as improved access and cost-effectiveness.4

The Community and Hospital Allied Network Glaucoma Evaluation Scheme (CHANGES) was launched in August 2006 and was designed to evaluate objectively alternatives to the traditional glaucoma care pathway. The scheme involves specifically trained optometrists in the triage of patients suspected as having glaucoma (referral refinement phase) and, more recently, co-management of high-risk glaucoma suspects (shared-care phase) in the community setting. False-positive referrals consume significant secondary care resources, which could be directed more usefully elsewhere.

The CHANGES scheme has not received additional government funding (unlike the other pilot projects) and hence may be a more useful model for other secondary care providers. The purpose of this article is to describe the design, activity, and quality of the referral refinement phase of the CHANGES scheme.

Materials and methods

Eight Optometrists with a Specialist Interest in glaucoma (OSI) were appointed in March 2006. The OSIs work in community-based private practices that have the same standardized basic equipment used in the hospital glaucoma service, namely a slitlamp, a Humphrey visual field analyser (Carl Zeiss Meditec, Dublin, CA, USA), applanation tonometer (Goldmann model; Haag-Streit, Bern, Switzerland), and digital photography of the optic disc (Topcon, Tokyo, Japan). They were trained in glaucoma detection by the hospital glaucoma team during the four didactic half-day sessions, which included practical examination of the optic disc. All of the OSIs attended glaucoma clinics at the hospital and most achieved a nationally recognized postgraduate certificate in glaucoma shared care (City University, London).

The referral refinement phase of the CHANGES scheme commenced in August 2006. All glaucoma-related GOS-18 (General Optical Services-18) referral letters are scrutinized by the hospital-based optometrist and categorized as either low or high risk according to a protocol. The patient was deemed low risk if only one/none (eg only a family history) of the following factors were noted on the letter for either eye: abnormal optic disc, abnormal visual field (SITA-Fast), abnormal intraocular pressure (IOP) (22–28 mmHg or IOP asymmetry). All other referrals were deemed high risk (including any reference to a shallow anterior chamber) and were offered a hospital glaucoma clinic appointment in the traditional manner. The patients with low-risk referral letters were sent an information pack that contained a copy of the original referral letter and contact information for a choice of eight OSIs, and were invited to make an appointment within a month. Patients who failed to make an appointment were reminded by telephone or by mail. The OSI then examined the low-risk patients according to a protocol and discharged or referred the patient depending on their findings. Patients were only discharged if they had all of the following features in both eyes: an IOP below 22 mmHg, a normal optic disc appearance, Van Herick's temporal limbal chamber depth5 deeper than 15% of corneal thickness, and normal visual fields.

Agreement between the examination by the OSI and by the glaucoma consultant ophthalmologist (RB) was checked for all patients referred from the OSI to the hospital and also for a sample of those discharged (one in four consecutive discharged patients per OSI was reviewed clinically by the glaucoma consultant). The documentation of the findings of every patient examined by an OSI (which included a dilated optic disc examination with 66 or 78 dioptre lens and also a 24-2 SITA Standard visual field test) was also reviewed in a ‘virtual clinic’ by the glaucoma consultant (RB) and the hospital-based optometrist (KF) with data recorded and audited using an electronic patient record adapted for this purpose (Medisoft Limited, Leeds, UK) (Figure 1).

Figure 1
Figure 1

Illustration of the referral refinement phase of the Community and Hospital Allied Network Glaucoma Evaluation Scheme (CHANGES). Figures relate to the numbers of patients involved in each step of the pathway. Solid arrows, traditional pathway; dashed arrows, new CHANGES pathway; GOS-18, General Optical Services-18; VF, visual fields; IOP, intraocular pressure.

Statistical analysis

Data were recorded on a computer database (Microsoft Excel version) for all stages of the pathway for each patient. The sensitivity, specificity, positive, and negative predictive values were calculated to show the degree of agreement between OSI and consultant findings.


Activity of the referral refinement pathway

The hospital glaucoma service received 512 GOS-18 referral letters between 25 August 2006 and 31 December 2007. These were triaged into 374 (73.0%) ‘high-risk’ letters and 138 (27.0%) ‘low-risk’ letters. The content of the letters and the resultant referral pattern for the patients are summarized in Table 1.

Table 1: The content of the GOS-18 referral letters and the resultant destination for each of the 512 patients referred between 25 August 2006 and 31 December 2007

All 138 patients whose referral letter was judged ‘low risk’ were invited by letter to make an appointment with their choice of community OSI. Hundred and twenty-one (87.7%) were seen by an OSI, whereas 16 (11.6%) patients never made an appointment (despite repeated attempts on behalf of the hospital) with an OSI, and 1 (0.7%) chose a different hospital provider. All 374 patients whose referral letter was judged high risk were sent an appointment for the hospital glaucoma service. Three hundred and forty-three (91.7%) attended their appointment that was made with one consultant glaucoma ophthalmologist (RB). Non-attenders to hospital were offered up to two further appointment dates.

On review of the documentation, of the 40 patients discharged, the consultant agreed with the decision with 22 patients (55.0%) and disagreed with 11 patients (27.5%; 10 were due to suspicious optic disc/nerve fibre layer images and one was a protocol violation). The sampling process meant that 7 of these 40 patients (17.5%) were offered appointments with the hospital consultant (one of whom would also have been referred because of disagreement). Five of the 10 patients with suspicious optic disc/nerve fibre layer images on virtual review of documentation were found on subsequent examination by the consultant to have an abnormal optic disc/nerve fibre layer. The positive predictive value (PPV) of a patient having signs related to glaucoma detected by a consultant, following the original (non-OSI) referral to the hospital was therefore 86%.

Comparison of the content of the original referral letter (GOS-18) with the findings of the OSI and with the findings of the glaucoma consultant

Agreement between the content of the original referral letter (GOS-18) received by the hospital and the findings of the OSI was also investigated (Table 2). For referral letters in which raised IOP (22 but <28 mmHg) was the only abnormality, approximately half (52.5%) of these patients were found to have an abnormal IOP by the OSI; however, 13 of these patients were found to also have an abnormal optic disc necessitating hospital referral. Concordance between original referral letter content and OSI findings was greatest for letters that noted an abnormal optic disc appearance (the OSI agreed in 63.6%).

Table 2: Agreement between the content of the original referral letter (GOS-18) and the findings of the OSI

The findings of the original referral letter (GOS-18) were also compared with the findings of the consultant glaucomatologist for the 99 patients who had been directed into the CHANGES scheme with low-risk letters and who were subsequently reviewed by the consultant in the hospital glaucoma clinic (for reasons of abnormal findings, disagreement with the OSI decision to discharge, or sampling of patients discharged). For the finding of an abnormal optic disc, sensitivity, specificity, positive, and negative predictive values were 74% (95% CI: 58–85), 68% (95% CI: 55–80), 63% (95% CI: 48–76), 78% (95% CI: 64–88), respectively, when considering the consultant findings to be the ‘gold standard’. For the finding of an IOP above 21 mmHg in either/both eyes, sensitivity, specificity, positive, and negative predictive values were 63% (95% CI: 42–80), 79% (95% CI: 68–87), 53% (95% CI: 35–70), 85% (95% CI: 74–92), respectively.

Agreement between the consultant glaucoma ophthalmologist and the OSI

Eighty-one patients examined by the OSIs were judged to be at risk of glaucoma and referred to hospital. An additional 18 patients (sampled or discharge decision overridden by the consultant) were subsequently seen by the consultant glaucomatologist in the hospital glaucoma clinic, a total of 99 patients. Agreement between the consultant and the OSI with respect to IOP (>21 mmHg in either eye), optic disc examination (suspect/normal), and anterior chamber angle assessment was measured (Table 3). In patients, where the OSI detected an IOP in either eye of >21 mmHg but the ophthalmologist detected <22 mmHg in both eyes, the difference in IOP between OSI and ophthalmologist in the eye with the higher IOP measured by the OSI was within 3–4 mmHg in 37%, within 5–6 mmHg in 50%, and within 7–8 mmHg in 13% of patients. In patients, where the hospital ophthalmologist detected an IOP in either eye of >21 mmHg but the OSI detected <22 mmHg in both eyes, the difference in IOP between OSI and ophthalmologist in the eye with the higher IOP measured by the HES was within 0–2 mmHg in 17%, within 3–4 mmHg in 32%, 5–6 mmHg in 17%, within 7–8 mmHg in 17%, and within 9–10 mmHg in 17% of patients.

Table 3: Agreement between the consultant glaucomatologist and the OSI with respect to intraocular pressure (IOP), optic disc examination, and anterior chamber angle assessment for 99 patients (81 referred to the hospital, 18 patients seen on account of disagreement after documentation review or sampling of discharges)

The period of analysis (25 August 2006–31 December 2007) was divided into two consecutive 8-month periods to assess whether agreement between the OSI and the consultant glaucomatologist had improved over time. Referral letters had been received for 42 patients between 25th August 2006 and 25th April 2007, whereas 57 patients had been referred between 26 April 2007 and 25 December 2007. The mean difference in IOP measured by the OSI compared with that measured by the hospital ophthalmologist was not statistically different between the two 8-month time periods (independent t-test, P=0.23).

Significantly fewer false positives made by the OSI (as compared to the ‘gold standard’ measurement made by the consultant glaucomatologist) were reported in the more recent 8-month period for IOP measurements only (recorded as either above 21 or 21 mmHg; χ2, P=0.015). There were no significant differences in false-positive rates for the other two features nor with false-negative rates for all the three features. A significant disagreement was detected after virtual review of a patient's optic discs resulting in dismissal of one of the OSIs early in the scheme when the OSI did not respond to offers of retraining.

The process of sampling 1 : 4 of consecutive patients discharged by each of the OSIs identified 7 patients who were subsequently examined by the hospital glaucoma service. One was found to have an IOP of more than 21 mmHg in either eye. The consultant agreed with the OSI decision to discharge with the remaining six patients. One sampling discharge patient had already been recalled on account of disagreement between the OSI and consultant in the virtual assessment.


Design and activity of the scheme

This report describes the referral refinement phase of the CHANGES scheme, one of relatively few established glaucoma shared-care models involving community-based optometrists in the UK. The concept of sharing the care of glaucoma patients or glaucoma suspects with non-ophthalmologists is not new,6 but despite a recent governmental drive to encourage such programmes, little evidence is available for the efficacy and safety of such models, hence the detailed analysis of this particular scheme.

Standardizing the equipment in the community OSI practice against that in the hospital service offers a number of advantages: avoiding the need to repeat tests if the patient attends the hospital; facilitating the ‘virtual’ scrutiny of the documented findings of the optometrist by the hospital consultant; ensuring that patients requiring long-term follow-up have had a baseline examination of sufficient quality to allow measurement of structural and functional (including automated analysis) progression of the disease. Protocol-guided data collection and digital data transfer enabled effective virtual hospital supervision of the scheme so, for example, inappropriate decisions to discharge could be detected immediately. The sampling process was an additional quality control measure.

The criteria for ‘low-risk’ referrals (only one abnormal feature documented) used in CHANGES could be deemed overly conservative compared with other schemes,7 but we wanted patient safety to be paramount in the scheme. Using these criteria, a quarter of referrals were judged low risk and, of these patients, a third were subsequently discharged by the OSI. This pathway has therefore reduced the numbers of patients attending the hospital by 8% (1/4 × 1/3). Policy makers may be discouraged by the relatively small reduction in hospital attendances afforded by this model. However, it should be noted that of the high-risk patients seen by the hospital, only 10% were discharged at their first visit. This would suggest that relaxation of the criteria to increase the numbers in the low-risk category may not significantly reduce the numbers of patients who ultimately attend the hospital. The decision to divide referrals into low and high risk seemed sensible in the circumstances of this scheme; however, there is limited evidence for the justification of this and it is recognized that comparison with earlier studies that have not used such a stratification of referrals is made difficult as a result. Another potential limitation of this study was the relatively small number of patients analysed although this is substantially higher than that reported by some other schemes.7

Amongst the low-risk referrals, false positives were commonest (54%) for those with only an abnormal visual field. The majority of optometrists referring to the hospital use a different type of visual field analyser to the Humphrey. Were the criteria for high-risk referrals to be relaxed, allowing more patients to attend the OSI first, those with abnormal visual fields in combination with moderately raised IOPs would be the best and safest group to pilot this in.

Twelve percent of patients failed to attend the OSI within a month of receipt of the original referral letter and subsequently had to be called to the hospital. Non-attendance rates were similar in the hospital glaucoma service among new high-risk referrals (8.3%). Many of the non-attenders to the OSI were patients of working age.

Quality and accuracy of data and comparisons between providers

Of all the features in the low-risk group, a suspicious optic disc was the most likely to be confirmed by the OSI (in 71% of patients) resulting in a hospital referral, whereas abnormal visual fields alone or asymmetric IOPs alone were the most likely to be discharged by the OSI, sparing a hospital visit by these patients. All referrals were from optometrists who used non-contact tonometry, and of those with only an abnormally high IOP (22–28 mmHg), approximately a third were found to have a normal IOP with Goldmann tonometry and were subsequently discharged. Past studies have highlighted differences in IOP measurement between non-contact ‘airpuff’ and Goldmann applanation tonometry.8, 9

The comparison of abnormal features detected by the referring optometrist and the findings of the OSI among the low-risk referrals show that additional abnormal features are often detected by the OSI. The PPV for this new pathway was 86%. This means that 14% of referrals were false positives, which is lower than that reported by other studies (20–65%),10, 11, 12, 13, 14, 15, 16 although one should be cautious about such comparisons as this PPV relates to the agreement of the original referring optometrist's finding of at least one abnormal glaucoma-related sign (IOP, disc, visual fields, anterior chamber angle) when compared with a consultant ophthalmologist specializing in glaucoma also finding at least one glaucoma-related sign (ie, a justifiable referral), rather than the detection of glaucomatous optic neuropathy per se. Our low false-positive rate may be a result of the referral refinement process that involved triaging of the referral letters into high and low risk with the result that the prevalence of signs associated with glaucoma would be higher in patients with ‘high-risk’ referral letters and also higher among the group who were subjected to comprehensive examination by the OSI (the ‘low-risk’ referral letters). There may be other factors relating to the overall performance of optometrists in detection of glaucoma in the local catchment area of this hospital. The number of false-positive referrals to the hospital may be reduced further if non-OSI optometrists perform a more comprehensive initial examination or introduce specific referral guidelines, although another study17 that attempted to do the latter found this had little effect on the proportion of false referrals. Of 40 patients discharged by the OSI, 6 (15%) were falsely discharged (‘false negatives’). These were detected by the virtual review of written and imaging data by the consultant. Efforts are ongoing to reduce this rate by targeting the source of error, the majority of cases being false interpretation of the optic nerve and retinal nerve fibre layer characteristics.

The current system of payment for optometrists performing a General Optical Services National Health Service ‘sight test’ does not include payment for the re-checking of visual fields or IOP (unless this is part of a locally agreed scheme) so there is little incentive for referral refinement.

For IOP, referral refinement does seem to be effective by reducing false-positive IOP findings to the hospital. It could be debated that this could be achieved without a shared-care scheme, by simply improving tonometry in all optometrist practices. For disc assessment, sensitivity of agreement between consultant and OSI was marginally higher than between consultant and GOS-18 referral letter, but the specificity was worse. This leads one to consider whether the OSI is really adding more information in terms of referral refinement with the optic disc examination.

The good agreement between consultant and OSI was an important finding with high negative predictive values implying low numbers of false negative findings, which would pose a risk to patient safety. However, it must be recognized that PPVs reported for this subset of patients referred to the consultant will generally be higher than if one considered all hospital referrals that would occur in a traditional pathway. This is because the OSI had performed a comprehensive glaucoma evaluation and discharged patients with no suspicion of glaucoma thus resulting in a referred set of patients with a higher prevalence of glaucoma or abnormal parameters. The agreement between an OSI judgement of a shallow anterior chamber (Van Herick's temporal limbal chamber depth5 shallower than 15% of corneal thickness) and a consultant judgement of an occludable angle on gonioscopy was not dissimilar to values obtained in another study involving Mongolian subjects (sensitivity, 84%, specificity, 86%).5 This is an important consideration in that most community-based optometrists are not trained in gonioscopy.


The CHANGES glaucoma scheme has successfully involved optometrists based in this particular community. This report shows that a referral refinement process can reduce numbers of false-positive referrals attending the hospital glaucoma service. Importantly, this modest reduction was achieved with relatively low risk to the patient as shown in this report by the finding of high PPVs for the key features of the glaucoma examination (optic disc, IOP, anterior chamber angle) when comparing the findings of OSIs and the consultant glaucoma ophthalmologist in this particular clinical setting, although the risk involved in a traditional glaucoma care pathway is difficult to quantify and will certainly vary depending on the clinical setting involved. The next key question to be answered is whether cost effectiveness can be objectively demonstrated.


  1. 1.

    Department of Health, United Kingdom Government. Our Health, Our Care, Our Say 2006 (accessed from ).

  2. 2.

    Department of Health, United Kingdom Government. Commissioning Toolkit for Community Based Eye Care Services 2007 (accessed from ).

  3. 3.

    Department of Health, United Kingdom Government. .

  4. 4.

    Department of Health, United Kingdom Government. .

  5. 5.

    , , , , , et al. Detection of gonioscopically occludable angles and primary angle closure glaucoma by estimation of limbal chamber depth in Asians: modified grading scheme. Br J Ophthalmol 2000; 84: 186–192.

  6. 6.

    . Care to share? Br J Ophthalmol 2000; 84(7): 677–678.

  7. 7.

    , , , . Community refinement of glaucoma referrals. Eye 2003; 17: 21–26.

  8. 8.

    , , , , , . A comparison of four methods of tonometry: method agreement and interobserver variability. Br J Ophthalmol 2005; 89(7): 847–850.

  9. 9.

    , , , . Comparative study of three non-contact tonometers and the Goldmann tonometer. Ophthalmologica 1994; 208(3): 115–118.

  10. 10.

    , , . Referral patterns to an ophthalmic outpatient-clinic by general-practitioners and ophthalmic opticians and the role of these professionals in screening for ocular disease. Br Med J 1988; 297: 1162–1167.

  11. 11.

    , . Referral pattern of true and suspected glaucoma to an adult ophthalmic outpatient clinic. Trans Ophthalmol Soc UK 1983; 103: 284–287.

  12. 12.

    , . Positive predictive value of optometrist-initiated referrals for glaucoma. Ophthalmic Physiol Opt 1999; 19: 62–67.

  13. 13.

    . The changing pattern of glaucoma referrals by optometrists. Eye 1998; 12: 854–857.

  14. 14.

    , . The diagnostic outcome of new glaucoma referrals. Ophthalmic Physiol Opt 1997; 17: 3–6.

  15. 15.

    , . Accuracy of referral to a glaucoma clinic. Ophthalmic Physiol Opt 1997; 17: 7–11.

  16. 16.

    , . Efficiency of referral for suspected glaucoma. Br Med J 1991; 302: 998–1000.

  17. 17.

    , . Do locally agreed guidelines for optometrists concerning the referral of glaucoma suspects influence referral practice? Eye 2001; 15: 458–463.

Download references


The authors thank the eight optometrists with a specialist interest in glaucoma (Sue Martynski, Caroline Hurst, Alison Hammond, Lesley Hodgson, Sally Bushby, Angela Wimpenny, John Kidd, Keziah Latham) and the community eye services department at Hinchingbrooke Hospital (Alison Hammond and Bridget Wilson). Karen Hayton and Gene Dunbar of the former Huntingdon Primary Care Trust are thanked for their collaborative efforts.

Financial support from Allergan Inc., (Irvine, CA, USA) is gratefully acknowledged in the form of an unrestricted grant for a Glaucoma Fellowship (LC) and also in the form of a Diagnostic Innovations in Glaucoma Care award to the CHANGES scheme in 2007.

Author information


  1. Glaucoma Service, Department of Ophthalmology, Hinchingbrooke Hospital, Huntingdon, Cambridgeshire, UK

    • R R A Bourne
    • , K A French
    • , L Chang
    • , A D Borman
    • , M Hingorani
    •  & W D Newsom
  2. Glaucoma Service, Moorfields Eye Hospital, London, UK

    • R R A Bourne
    •  & L Chang
  3. Vision and Eye Research Unit, Postgraduate Medical Institute, Anglia Ruskin University, Cambridge, UK

    • R R A Bourne


  1. Search for R R A Bourne in:

  2. Search for K A French in:

  3. Search for L Chang in:

  4. Search for A D Borman in:

  5. Search for M Hingorani in:

  6. Search for W D Newsom in:

Competing interests

The authors declare no conflict of interest.

Corresponding author

Correspondence to R R A Bourne.

About this article

Publication history






Further reading