Introduction

The Royal College of Ophthalmologists' (RCOphth) charter includes the following objectives: ‘to advance the science and practice of ophthalmology; educate medical practitioners in the science and practice of ophthalmology [and] to maintain proper standards in the practice of ophthalmology for the benefit of the public’.1 In relation to cataract surgery, it has advanced these aims by organising two National Cataract Surgery Surveys in 19902, 3, 4 and 1997–985, 6 to examine variations in the organisation of services and clinical outcomes. Both surveys established benchmark standards, by which departments and individual surgeons could judge their performance. In 2003, the RCOphth proposed a Cataract National Dataset (CND) to be collected electronically as a by-product of routine care using electronic medical record (EMR) systems. It was designed to support excellent clinical care and facilitate detailed audit of clinical outcomes on a local and national scale. In 2005/6 Connecting for Health's clinical engagement arm commissioned a ‘Cataract Do Once and Share’ project to refine the CND7 and pilot its extraction from existing EMR systems used in the English NHS. Studies related to this and previous electronic data extractions have updated benchmark standards in the United Kingdom and internationally.8, 9, 10, 11, 12 The RCOphth 2004 Cataract Surgery Guidelines have subsequently been amended to recommend that the CND is collected electronically as part of routine clinical care to enable detailed audit of all cases.13

Posterior capsular rupture (PCR) with or without vitreous loss is the most common intra-operative complication during cataract surgery and is widely regarded as the benchmark complication to judge surgical quality.6, 9, 12, 14 This measure has high validity for cataract surgeons and is important as it is associated with the need for additional surgical procedures, a greater number of follow-up visits, increased frequency of postoperative complications and may adversely affect the final visual outcome.15 PCR rates are likely to form a key clinical indicator for revalidation of ophthalmologists in the new regulatory environment that is currently being developed in the United Kingdom.16

Many large studies have documented mean rates of PCR in cohorts of patients,6, 8, 9, 14 or learning curves for trainees17 or experienced surgeons learning phacoemulsification.18 Very few studies, however, have analysed inter-surgeon variation of this benchmark complication and all such studies involved junior surgeons19during training rather than fully trained surgeons who deliver the majority of cataract surgery in the NHS.9

The primary aim of this audit is to describe the overall rates and variations of PCR within each grade of surgeon in the United Kingdom using a large data set extracted as part of the piloting of the CND.

Materials and methods

The methods used in this large prospective cross-sectional survey audit have been described in detail in the first paper in the series.9 Briefly, NHS Trusts using EMR systems for cataract surgery were invited to submit data for analysis. Following appropriate consent, data were extracted and anonymised such that no patients, hospitals, or health-care professionals were identifiable. Extracted data were assessed for conformity with the CND in terms of format and completeness by the Royal College of Ophthalmologists Dataset Working Group Lead (JMS). Data from 12 Trusts using a single EMR system (Medisoft Ophthalmology, Medisoft Limited, Leeds, UK) were deemed of sufficient quality for analysis, and these data form the basis for this series of reports. A local ethics committee confirmed that ethics approval was not required, as this study was an audit with no patients, hospitals, or health-care workers identifiable.

To balance completeness, speed, and accuracy of data entry, the Medisoft EMR allows many cataract surgical fields to be defaulted for each surgeon's technique but a compulsory choice must be made each time the operative complication field is completed. In keeping with our other reports in this series, posterior capsular rupture with or without vitreous loss is analysed as the benchmark complication.

Statistical methods

Funnel plots are used to display the data, as they are the method of choice for institutional and individual comparison.20 They plot an estimate of underlying quality, in this case posterior capsule rupture for each surgeon, against an easily interpretable measure of its precision, in this case the total number of cases performed by each surgeon. They allow rapid visual comparison of data from multiple individuals. No confidence, or ‘control’ limits have been applied, as these are crude complication rates that have not been adjusted for case mix complexity.

Results

Data were extracted on 55 567 cataract operations performed at 12 NHS trusts by 406 surgeons between November 2001 and July 2006. Overall, 86.0% of operations were performed between January 2004 and July 2006, when the data were extracted. No individual surgeon contributed more than 4.6% of operations and no hospital contributed more than 20% of all operations.

Data on the presence or absence of a surgical complication were available for all 55 567 cases. The grade of 404 of the 406 surgeons who contributed to the study was available for 55 515 cases (99.9%) and are used for analysis in this study.

Funnel plots of the percentage of operations complicated by PCR vs the number of operations performed by each surgeon are shown in Figures 1, 2, 3, 4. Figure 1 shows data for all surgeons, Figure 2 the data for independent surgeons (consultants, staff grades, and associate specialists), Figure 3 the data for more senior trainees (specialist registrars and fellows), and Figure 4 shows the data for the most junior grade of surgeons in their first 2–3 years of training (senior house officers (SHO) at the time of data collection).

Figure 1
figure 1

Funnel plot for all surgeons: percentage of operations complicated by PCR vs number of operations performed by each surgeon. (Data for eight surgeons with PCR in more than 20% of their operations (range 22–100%) are not shown on the above graph in order that all the graphs have the same Y axis scale. These surgeons had contributed between 1 and 29 operations to the dataset and all were trainees.) (N=406 surgeons). Overall PCR rate=1.92%.

Full size image
Figure 2
figure 2

Funnel plot for independent surgeons: percentage of operations complicated by PCR vs number of operations performed by each consultant surgeon. (N=137 surgeons). Overall PCR rate=1.41%.

Full size image
Figure 3
figure 3

Funnel plot for senior trainee surgeons (specialist registrars and fellows): percentage of operations complicated by PCR vs number of operations performed by each senior trainee surgeon. (Data for four surgeons with PCR in 20% or more of their operations (range 24–100%) are not shown on the above graph in order that all the graphs have the same Y axis scale. These surgeons had each contributed between 1 and 29 operations to the data set.) (N=188 surgeons). Overall PCR rate=2.48%.

Full size image
Figure 4
figure 4

Funnel plot for junior trainees (SHO) surgeons: percentage of operations complicated by PCR vs number of operations performed by each SHO surgeon. (Data for four surgeons with PCR in more than 20% of their operations (range 22–50%) are not shown on the above graph in order that all the graphs have the same Y axis scale. These surgeons had each contributed between 2 and 9 operations to the data set.) (N=79 surgeons). Overall PCR rate=5.1%.

Full size image

The overall rates of PCR for all surgeons, independent surgeons, senior trainees, and junior trainees or SHOs were 1.92, 1.41, 2.48, and 5.10%, respectively.

The range of PCR within each group for surgeons who contributed more than 50 cases to the data were as follows: 0–15.1% for all surgeons; 0–15.1% for independent surgeons; 0–11.9% for senior trainees, and 0–11.9% for SHOs.

Discussion

This is the first study to harness the power of EMR systems to extract data collected as a by-product of routine clinical care on a large cohort of patients in order to describe variation between surgeons in the benchmark complication of PCR during cataract surgery.

The data in this study were of high quality with key variables completed for 99.9–100% of cases. The size and quality of this set of data allows an accurate description of the degree of variation in PCR rates for different grades of surgeon. Provided the sample is representative of surgical practice in the United Kingdom, all grades of surgeon now have a reliable context in which to judge their own performance. These data contrast with previous studies, which only presented figures for overall PCR rates without any measures of variation between surgeons,6, 8, 9 with the exception of a few small studies that focused on trainees alone.19 Our previous report9 has demonstrated that the data collected are likely to be typical of all cataract surgery in the United Kingdom, as the basic demographic data on age, sex, and frequency of day case are similar to the National Hospital Episode Statistics during the same time period.21

As expected, the study demonstrates that the rate of PCR is much lower for independent surgeons (consultants, staff grades, and associate specialists, Figure 2) than it is for senior trainees (specialist registrars and fellows, Figure 3) and the most junior trainee surgeons (SHOs, Figure 4). It is of note that the spread around the overall rate on the funnel plots widens as the number of cases decreases. This effect is predominantly because of sampling error and does not necessarily imply that these surgeons are under performing. With small samples, it becomes impossible to make any meaningful interpretation of the data.

The overall PCR rate of 5.10% for junior trainees (SHOs) was, as expected, highest. The overall rate of 2.48% for senior trainees (specialist registrars and fellows) in this study was comparable with that reported in a recent study that analysed UK trainees log books19 (mean 2.29%, range 0.6–6.9%). The upper end of this range may appear alarming, but with small sample sizes these variations in rates may well be due to sampling error, an effect which is well illustrated by our funnel plots. These rates are, however, lower than those in many US residency programmes, but this is to be expected as US training programmes are shorter and trainees on an average perform far fewer surgeries during training compared with the United Kingdom.22, 23, 24, 25

The General Medical Council in the United Kingdom is changing the way fully qualified doctors are regulated by introducing a requirement to have the licence to practice renewed every 5 years.16 The process of revalidation is currently being developed but will undoubtedly depend on detailed audit of each doctor's range of clinical practice. Cataract surgery is the most commonly performed surgical procedure in the United Kingdom and is performed by the vast majority of ophthalmologists. There is wide agreement among ophthalmologists that PCR can be used as a measure of surgical quality. PCR is therefore considered by many as a key quality indicator for the revalidation of cataract surgeons. In the United Kingdom, there has been a trend for improvement from the mean benchmark rate of 4.4% defined in the 1997 National Survey6 to 2.68% in the first Pilot National Electronic Cataract Surgery Survey13 and 1.92% in the current survey of 55 567 cases.9 None of these studies, however, make any reference to the variation between surgeons. This rapid change in mean rate over time of one of the key clinical indicators for ophthalmology demonstrates the need to regularly update clinical benchmarks.

When seeking to judge if any individual surgeon's PCR rate is outside acceptable limits, it is important to account for case mix by analysing all the ocular and systemic factors that increase the risk of PCR in their personal cohort of patients. From our previous analysis, the probability of PCR occurrence can be predicted in relation to a range of known preoperative risk factors and the grade of surgeon.12 This paper makes no comment as to whether any individual's PCR rate is wide of acceptable boundaries. Further work will develop a methodology for case-mix adjustment based on comparisons between the predicted probability of a complication arising according to the regression model defined in our earlier paper12 and whether or not a complication did or did not actually arise.

In conclusion, the researchers believe that regular analysis of variations in key clinical indicators for each grade of surgeon will be essential to inform revalidation and measure the quality of training in cataract surgery. To avoid misrepresentations, it is essential to develop a robust method of risk adjustment to account for the complexity of the case mix of individual surgeons.