Introduction

This is the second in a series of three studies assessing the accuracy of diagnostic algorithms for each of the three most commonly encountered ophthalmic presentations: red eye(s),1 visual loss, and diplopia. As discussed in the previous paper,1 our three algorithms highlight those key aspects of both the history and the examination which allow the inexperienced examiner to come to the likely diagnosis. Algorithms are, therefore, simply the user-friendly version of the diagnostic and/or treatment thought processes of more experienced clinicians.

Algorithms are being used more and more in medicine; junior doctors are being asked to provide cross cover over a greater range of specialties than before. Current examples in healthcare include diagnosing pulmonary embolus,2 differentiating types of tachycardia3 and a number of algorithms published by the Resuscitation Council (UK) such as Basic and Advanced Life Support.4 Diagnostic algorithms have been suggested to reduce time, effort and bias,5 and cost of care and errors.6 It has been shown that in situations where algorithms have been introduced, experienced healthcare workers continue to demonstrate autonomy.6

The causes of visual loss range from benign; refractive error, to life threatening; posterior visual pathway lesions. Most patients are initially seen in primary care, by GPs, A&E doctors or optometrists who have varying levels of experience in dealing with acute ophthalmic patients. Unfortunately, most UK doctors including GPs and A&E doctors have had between 0 and 2 weeks ophthalmology teaching,7 during their undergraduate training, leaving them inexperienced and wary of dealing with acute eye problems.8 In many A&E departments nurse practitioners are taking on more responsibility and independently managing patients, without any formal eye training or experience. Optometrists are becoming increasingly involved in ophthalmic primary care in the community and are the first port of call for many patients with an eye emergency problem, particularly in Scotland.9 These groups of inexperienced clinicians (with respect to acute ophthalmological problems), including junior ophthalmologists, may benefit from simple clinical algorithms. Algorithms are always a compromise between having enough detail to cover the most commonly encountered diagnoses whilst remaining simple enough to use.

The aim of this prospective study was to assess the diagnostic accuracy of the Edinburgh Visual Loss Algorithm, published in ‘Ophthalmology Pocket Tutor’,10 which was created by Dr Mark Wright, Consultant Ophthalmologist and Lead for Undergraduate Ophthalmology Education at Edinburgh University.

Materials and methods

Patients presenting to the Edinburgh Eye Pavilion with visual loss, usually having been referred from primary care (optometry or general practice) or less commonly from secondary care (non ophthalmology hospital setting) were assessed using the Edinburgh Visual Loss Algorithm (Figure 1) by one of three groups;

  • One fourth year medical student (CL; following the week-long clinical attachment in ophthalmology).

  • Junior ophthalmology trainees (<2 years of ophthalmology experience).

  • Optometrists (based in LOTT; Lothian Optometry Treat & Teach clinic).

Figure 1
figure 1

The Edinburgh Visual Loss Diagnostic Algorithm.

Full size image

This data was collected in order to assess the accuracy of the ‘algorithm-assisted’ diagnosis. An experienced ophthalmologist, defined as having a minimum of 2 years full-time training, then independently examined the patient to arrive at a ‘gold-standard’ diagnosis. The degree of concordance between the algorithm-assisted diagnosis and the gold standard was then assessed.

The source and diagnosis of each referral from primary care or another hospital specialty was also noted to estimate the ‘pre-algorithm’ diagnostic accuracy.

Ethical approval was granted through the University of Edinburgh College of Medicine, in conjunction with the South East Scotland Research Ethics Service.

Results

Seventy completed forms were returned during the recruitment period of February to December 2014. Two were excluded; both were filled in incorrectly by the algorithm user, leaving 68 cases available for analysis.

Profile of referrer

The breakdown of referrer by type was; Optician 62%, GP 12%, A+E Doctor 4%, doctors from other hospital specialties 4%, nurse practitioners in minor injuries units 3% and diabetic retinopathy screeners 2%. Six percent of patients self-presented without a referral and referral information was unavailable in 7%.

Pre-algorithm accuracy of referrals with visual loss

Out of these 68 cases, a ‘pre-algorithm’ referral diagnosis was unavailable in 9; in 5 the initial referral was unavailable or lost and 4 patients self-presented to the acute eye clinic without being referred from the community. This left 59 cases which allowed a direct comparison to be made between the diagnoses made by non-experts with and without the visual loss algorithm. In 16 of the 59 cases the referrer simply described the symptoms and signs but did not include a diagnosis. For the purposes of the analysis these referrals were classified as incorrect diagnoses. This was the case for referrals from; community optometrist in 5 out of 42 patients, by doctors and nurse practitioners in 10 out of 14 patients and by diabetic retinopathy screeners in 1 out of 1 patient. The overall pre-algorithm accuracy of referral of patients with visual loss was 51% (30/59). Optometrists were the most accurate referrers making the correct diagnosis 67% (28/42) of the time. GPs were correct in 13% (1/8) of referrals, A&E doctors in 33% (1/3). There were no correct referral diagnoses from; doctors from other hospital specialties (0/3), nurse practitioners in minor injuries units (0/2), and diabetic retinopathy screeners (0/1).

Of these 59 cases, no attempt at the diagnosis was made by the community optometrist in 5 out of 42 patients, by doctors and nurse practitioners in 10 out of 14 patients and by diabetic retinopathy screeners in 1 out of 1 patient. For the purposes of the analysis these were classified as incorrect diagnoses.

The pre-algorithm accuracy of referral of patients with visual loss was 51% (30/59). Optometrists were the most accurate referrers making the correct diagnosis 67% (28/42) of the time. GPs were correct in 13% (1/8) of referrals, A&E doctors in 33% (1/3), and doctors from other hospital specialties were correct in 0% (0/3). Nurse practitioners in minor injuries units were correct in 0% (0/2) of referrals and diabetic retinopathy screeners referred one patient without knowing the diagnosis 0% (0/1).

Profile of algorithm user

The breakdown of algorithm user by type was; medical student 45%, optometrist 18%, and junior ophthalmology trainee 37%.

Diagnostic accuracy of the algorithm-assisted diagnoses

Overall, the algorithm-assisted diagnosis was correct 84% (57/68) of the time. The ‘gold-standard’ ophthalmologists made seven out a possible eight diagnoses contained within the algorithm; there were no ‘chiasmal’ diagnoses. The algorithm correctly diagnosed: retina in 71% of cases (5/7), macula in 86% (25/29), peripheral retina in 100% (2/2), optic nerve in 71% (5/7), media opacity in 89% (16/18), post chiasmal in 100% (4/4), and refractive error in 0% (0/1); see Figure 2. The accuracy of diagnosis was broadly similar for each algorithm user; medical student 81%, ophthalmology trainee (ST2 and below) 84% and optometrist 92%. Sensitivity and specificity of the algorithm for the different diagnoses can be seen in Figure 3.

Figure 2
figure 2

Graph to show the number of patients with each eye condition and the accuracy of the Edinburgh Visual Loss Algorithm in correctly diagnosing these patients.

Full size image
Figure 3
figure 3

Graph to show the sensitivity and specificity of the Edinburgh Visual Loss Algorithm for each diagnosis.

Full size image

Analysis of the incorrect algorithm-assisted diagnoses

The algorithm-assisted diagnosis was incorrect in 16% (11/68) of the cases. The two sources of diagnostic error were either defects in the algorithm itself; 9% (6/68) or failure of the inexperienced clinician to elicit a clinical sign; 7% (5/68).

Six incorrect diagnoses were made due to problems with the algorithm

Three of these errors were due to dual/pre-existing pathology. One patient in whom the dominant cause of their visual loss was macula (wet age-related macular degeneration AMD) also had cataract resulting in a hazy retinal view and consequently an incorrect algorithm-assisted diagnosis of media opacity. Another patient with new wet AMD had an incorrect algorithm-assisted diagnosis of a post-chiasmal cause of visual loss due to a bilateral, although central, visual field defect secondary to longstanding bilateral dry AMD. The last patient in this group had dual pathology; new onset media opacity caused by idiopathic vitreous haemorrhage and pre-existing non-arteritic anterior ischaemic optic neuropathy (NAION). This patient was incorrectly diagnosed as ‘retina’, because of a relative afferent pupillary defect (RAPD) related to the NAION.

Two patients were classified incorrectly due to the absence of an RAPD despite extensive retinal pathology. One of these patients had a macula-off retinal detachment and the other had a non-ischaemic central retinal vein occlusion (CRVO).

The last patient in this group had a dislocated intraocular lens implant, causing a large refractive error that was not fully corrected by a pin hole. As a result this was diagnosed incorrectly as a media opacity.

Five incorrect diagnoses were made due to failure to elicit a clinical sign

Two patients were wrongly diagnosed because the non-specialist did not detect an RAPD resulting in a diagnosis of macula or media opacity, when the gold-standard diagnosis was optic nerve. In one case an RAPD was falsely detected by the non-specialist, the patient had wet AMD. In two cases an error occurred when the non-specialist incorrectly determined whether there was a clear or hazy retinal view on fundoscopy resulting in one incorrect diagnosis each of media opacity and macula.

Comparison of diagnoses made by non-experts with and without the algorithm

In the 59 cases where a ‘pre-algorithm’ referral diagnosis was available, diagnostic accuracy improved from 51 to 86% when using the algorithm. This is shown, grouped by referral source, in Table 1.

Table 1 Table showing the pre-algorithm accuracy of each referral (grouped by referral source) compared with diagnosis made in the same patient by a non-expert using the Edinburgh Visual Loss Algorithm
Full size table

Discussion

The baseline (pre-algorithm) diagnostic accuracy in this study from all referrers was 51%. The only other study that provided figures on referrals from GPs and ophthalmic opticians specifically relating to visual loss found marginally higher diagnostic accuracies, ranging from 51–72% depending on the source of referral.11 These figures demonstrate a need for a diagnostic aid when non-ophthalmologists assess patients who present with visual loss. Our study showed an overall algorithm-assisted diagnostic accuracy of 84%. Its simplicity of use is confirmed by our finding that the most inexperienced user; a medical student with only 14 h of undergraduate ophthalmology teaching, demonstrated a very high level of diagnostic accuracy; 81%.

The ideal triage system facilitates the patient being assessed by the most appropriate person at the first visit. The algorithm correctly diagnosed the presence of a posterior visual pathway lesion (‘post-chiasmal’) in 100% (4/4) patients. The most appropriate triage decision for this group of four patients may have been to organize neuro-imaging prior to the first ophthalmic consultation or indeed redirection to a neurovascular clinic if a vascular cause was thought likely.

The ideal algorithm should allow the user to demonstrate very high sensitivity with respect to potentially the most serious conditions; posterior visual pathway lesions, post chiasmal in 100% (4/4), and also to have high levels of specificity for the more minor conditions; media opacity in 89% (16/18).

The incorrect identification of RAPDs was involved in 5 out of 11 wrong diagnoses. RAPDs can be difficult to elicit, particularly for less experienced examiners.12 In order to improve inexperienced examiners skill in eliciting an RAPD we have demonstrated this clinical technique at https://www.eemec.med.ed.ac.uk/pages/resources/mw-ophthalmology-page. Only one error resulted from difficulty of an inexperienced examiner using a direct ophthalmoscope. The difficulty in performing accurate direct ophthalmoscopy by different specialties has been described elsewhere.13 In our study this difficulty contributed to a diagnosis of ‘don’t know’ as seen in 11 out of 18 referrals from doctors and nurses. This was taken into account when designing the algorithm by simplifying fundoscopy findings, for example, retinal appearance either ‘clear’ or ‘hazy’.

In reaching an algorithm assisted diagnosis of a retinal problem we have assumed that both the peripheral and central retina (macula) are involved. Significant retinal pathology with an estimated 30–50% of nerve axons need to damaged before an RAPD is evident. In our study two patients with ‘retinal’ pathology did not have an RAPD. One of these had a retinal detachment and the other had a non-ischaemic CRVO, whilst both these patients had involvement of the peripheral and central retina neither had significant retinal ischaemia and therefore no accompanying RAPD.

The algorithm failed to reach the correct anatomical diagnosis in three cases when there was dual/pre-existing eye disease. Deciding which of two pathologies, for example, cataract and AMD is the dominant one relies upon clinical experience and confirms the limitations of our algorithm where more complex decision making is required.

During the course of the study a problem relating to the urgency of referrals in patients with media opacity became evident. The most common cause of media opacity is cataract, which causes visual loss of gradual onset and can be referred routinely as per the algorithm. However, vitreous haemorrhage also results in media opacity but typically with sudden onset and always requires prompt assessment. We will update the algorithm to reflect this.

One of the educational benefits to the inexperienced clinician of using the algorithm when assessing patients who present with visual loss is that it acts as a clinical prompt reminding the novice of each of the pertinent components of the examination and the ideal sequence in which to perform them.

The main limitation of the study was the lack of patients with a diagnosis of ‘chiasmal’ disease. This is a very infrequently encountered scenario and hence difficult to recruit patients to.

Conclusion

This is the first time the diagnostic accuracy of any visual loss algorithm has been assessed. The baseline diagnostic accuracy of non-ophthalmologists rose from 51 to 84% when patients were assessed using The Edinburgh Visual Loss Diagnostic Algorithm. This algorithm significantly improves the diagnostic accuracy of referrals to the hospital eye service, regardless of the user’s previous ophthalmic experience. We hope we have demonstrated its potential as a learning tool for inexperienced clinicians. A number of open access learning tools including downloadable copies of five diagnostic algorithms and narrated lectures accompanying the algorithms are available at https://www.eemec.med.ed.ac.uk/pages/resources/mw-ophthalmology-page.