Abstract
Aim
To apply the control chart method to retrospectively analyse the variation in postcataract surgery endophthalmitis.
Methods
The endophthalmitis data between 01/07/97 and 30/06/07 (10 years) at the Royal Gwent Hospital were analysed using the control chart method. Certain criteria related to the control chart were employed to detect unusual variations and early outbreaks of endophthalmitis.
Results
A total of 21 032 cases of cataract were performed in that period, with a mean annual incidence of 0.16%. The control chart could provide a reliable method to detect an early outbreak of endophthalmitis before it was suspected and acted on. The chart provided visual information of the altering trends of endophthalmitis cases and allows unusual variations to be detected early.
Conclusion
The control chart is potentially a sensitive method to detect anomaly in the cases of postcataract surgery endophthalmitis. Its control limits are dynamic and factor in recent performance to detect any unusual variation.
Similar content being viewed by others
Introduction
Cataract surgery is one of the most performed surgical procedures in the United Kingdom. The recent British Ophthalmological Surveillance Unit study reported an overall UK postcataract surgery endophthalmitis incidence rate of 0.14% or 1 in 700 operations,1 which is very similar to the rate found in a systematic review of 215 published studies.2 Although this condition is rare, it has a poor prognosis, with only 53% of cases achieving a final visual acuity of 20/40 or better.3
The rate of postcataract surgery endophthalmitis may be consistent for a period of time. However, the exact number is not expected to be identical every month but varies around a long-term mean value. The variation is inherent in the process itself, which is reasonably stable and likely to be predictable within certain limits. With a special cause or ‘unnatural’ variation, one or more points vary in an unpredictable manner. This is a ‘signal’ that the process has changed. Various methods such as the Monte-Carlo4 and binomial theorem probability5 have been described to detect these ‘signals’ or what we more commonly described as early outbreaks.
The control chart is a method that has been used widely in healthcare at operational, clinical, strategic, and public policy level6, 7, 8, 9, 10 to effectively detect unusual variations in a system. We feel that this method has a useful role to detect variations in the rates of endophthalmitis and early outbreaks in postcataract surgery endophthalmitis.
The control chart has control limits and certain guidelines to detect unusual variations in a system. Any point that falls outside the upper and lower control limits is deemed unusual. These limits are usually set at 3 standard deviations (SD) However, for a situation with unacceptably serious outcome, the control limit may be tightened to 2 SD.11 The obvious disadvantage of this action is an increase in false-positive rate. The probability of a point falling beyond either control limit set at this level by chance is approximately 0.05. Other guidelines recommended11 for suggesting an unusual variation in a control chart are as follows:
-
1)
2 out of 3 consecutive points between the control limit and one SD below this level.
-
2)
A run. This occurs when seven or more consecutive points fall on either side of the centre line.
-
3)
A trend. When six consecutive points move steadily upwards or downwards.
We used this approach to retrospectively analyse our departmental postcataract surgery endophthalmitis over a 10-year study period. We also discussed and compared the different possible statistical methods available at present.
Materials and methods
All cataract extractions from 01/07/97 to 30//06/07 (10 years) at the Royal Gwent Hospital were analysed. Patients who underwent combined procedures (eg, trabeculectomy and penetrating keratoplasty) were excluded. Patient details were obtained from theatre logbooks. Patients' records were requested for those diagnosed with postoperative endophthalmitis.
Acute-onset endophthalmitis was defined as any patient with a clinical diagnosis of presumed infectious endophthalmitis occurring within 6 weeks of cataract surgery.7
The annual incidence for each year was calculated. The control chart method was used to analyse the data to investigate the pattern of unusual variation and the possibility of an outbreak retrospectively.
Results
During the study period, a total of 21 032 cataract operations were performed in the Royal Gwent Hospital. There were 35 cases of acute-onset endophthalmitis. There were 21 female and 14 male patients. The mean age was 79.7 years (range: 69–93). The annual incidences (1 July to 30 June) of postcataract surgery endophthalmitis during the study period were 0.17, 0.16, 0.21, 0.15, 0.19, 0.40, 0.09, 0.05, 0.10, and 0.08% respectively. The mean annual incidence over the 10-year period was 0.16%.
A line graph was plotted for the number of non-infected cases between endophthalmitis cases against each endophthalmitis case (see Figure 1). There was only 34 points for 35 cases of endophthalmitis since the first entry for the number of non-infected cases was taken after the first endophthalmitis case. We used the control chart method to detect a rare event and set the control limits accordingly (see Appendix for control limit calculation).
The points at which an outbreak occurred need to be compared with a stable baseline mean before the outbreak. In our case series, we suspected that there may be unusual variations and clustering between points 18–26. Therefore, the baseline mean was calculated from the data before this point, that is, from point 1–17. The mean was 588.4. The lower control limit set at 2 SD, which was at 108.2 of cases. The criteria aforementioned were applied to test point 18–26. The first point that fell below the control limit was 24. Two out of three consecutive points (points 18 and 19) fell between the first and second SD, suggesting an unusual variation in the system. There was also seven consecutive points (points 18–24) below the mean baseline, which was another indicator of an unusual variation.
However, the decision to suspend operating and start an investigation was not instigated until point 26. This was noted after five cases of endophthalmitis that occurred within 3 months in the beginning of 2003. Operating was suspended and a multidisciplinary team investigated the outbreak. No specific cause was found despite extensive investigations, but additional steps were taken to ensure sterility. Povidine iodine in conjunctival sac was also introduced. The interval between point 24 (the first point below the control limit) and point 26 was 20 days.
After the additional steps have been applied, a new mean was calculated from point 27–34. This was 1154.6 of non-infected cases between endophthalmitis cases (see Appendix). Compared with the mean of 195.2 from point 18–26, there is a clinically and statistically significant improvement (P=0.0006).
Discussion
The beginning of an outbreak of postcataract surgery endophthalmitis is not always easily identifiable. It may be tempting to classify a few closely presented cases of endophthalmitis as an epidemic, which may result in expensive investigation and suspension of operating theatre. Conversely, these cases could be passed off as a normal variation, delaying investigations that could have prevented subsequent cases.
The purpose of this study was to use the control chart to improve the monitoring of an endophthalmitis outbreak. As postcataract surgery endophthalmitis is usually a rare event, this method of plotting a control chart of non-infective cases between endophthalmitis cases is preferred to plotting the number of infective cases or endophthalmitis incidence against time as mentioned by Anuradha and Feyi-Waboso.12 The conventional control chart recommendation is to set the lower control limit at 3 SD. This may be acceptable when dealing with fairly non-serious outcome, for example, the number of days it takes to mail the patient an outpatient appointment; however, when patients' sight are involved, one could argue not to wait until a data point exceeds 3 SD before taking action. Therefore, a case can be made for using 2 SD as the lower control limit in this situation as in certain chemical industries that could not afford risking much variation in their production line. However, this will lead to an increase in false-positive rate and should be taken into consideration.13 We are aware that the control limits of the 2 SD may be an arbitrary value for endophthalmitis-outbreak monitoring. The right balance needs to be struck between sensitivity and specificity, and this process could only be obtained in a rigorous statistical analysis of large pools of data.
For the purpose of endophthalmitis-outbreak monitoring, the first two guidelines for control chart are more useful in detecting an early outbreak, as fewer points are required. However, the guideline for a ‘run’ could be altered to an appropriate level on the basis of previous data, for example, five rather seven consecutive points fall on the same side of the centerline. Similarly, detection of a ‘trend’ could be tightened accordingly. The latter two criteria are, however, useful to delineate the ‘drift’ or tendency in the system.
A control chart should have about 16–20 data points to apply the four guidelines described above.11, 14 With fewer data entry points, there is an increased danger of missing special causes (type-II error). Conversely, with more than 35 data entry points, there is increasing likelihood of finding special causes due to chance (type-I error). The most common special causes in these instances would be from a ‘trend’, and the points outside the control limits that tend to regress towards a mean. One way to avoid this is by sampling less frequently. For example, if there is a new case of endophthalmitis, the control chart could be used over the most recent 16–20 cases to detect a special cause. However, this entails good book-keeping of not just the number of endophthalmitis cases but the number of non-infected cases between infected ones.
Other methods of identification of an outbreak that have been described are the Monte-Carlo4 and binomial theorem probability.5 The main advantages of the control chart over these methods are that it allows the data to be analysed over a recent period of time and the control limits set according to recent performance. The 2 SD control limits are by no means rigid and we recommend individualization of the thresholds and control chart guidelines accordingly to the departmental record. However, this could only be achieved with frequent usage of the control chart through time and experience. The control chart also enables data in one period of time to be compared with another. The graphical presentation allows large amount of data to be easily understood without understanding complex statistics. Improvement trends are also notable on the control chart.
The main drawback of the control chart method in accurately detecting an unusual variation in rare events like endophthalmitis is that the system would need to have at least a ‘background’ mean to start off with. Ideally, to achieve this, it needs to meet the minimal data points (16 cases of endophthalmitis in the system). Other statistical methods may be more suitable for a newly established department.
The control chart is potentially a sensitive method to detect an outbreak of endophthalmitis. Its control limits are dynamic and factor in recent performance to detect an early outbreak of postcataract surgery endophthalmitis. We advocate that appropriate measures or investigations should be performed when certain signals emerged from a control chart. Further studies are required to identify a more accurate level of control limit appropriate for the identification of an outbreak or unusual variation.
References
Kamalarajah S, Silvetri G, Sharma N, Khan A, Foot B, Ling R et al. Surveillance of endophthalmitis following cataract surgery in the UK. Eye 2004; 18: 580–587.
Taban M, Behrens A, Newcomb RL, Nobe MY, Saedi G, Sweet PM et al. Acute endophthalmitis following cataract surgery: a systematic review of the literature. Arch Ophthalmol 2005; 123: 613–620.
Endophthalmitis Vitrectomy Study Group. Results of the Endophthalmitis Vitrectomy Study: a randomized trial of immediate vitrectomy and of intravenous antibiotics for the treatment of postoperative bacterial endophthalmitis. Arch Ophthalmol 1995; 113: 1479–1496.
Sellick JA . The use of statistical process control charts in hospital epidemiology. Infect Control Hosp Epidemiol 1993; 14: 649–656.
Finison LJ, Finison KS, Bliersback CM . The use of control charts to improve healthcare quality. Quality Progres 1995; 28: 120–124.
VanderVeen LM . Statistical process control: a practical application for hospitals. J Healthcare Quality 1992; 14: 20–29.
Adab P, Rouse AM, Mohammed MA . Performance league table: the NHS deserves better. Br Med J 2002; 324: 95–98.
Tekkis PP, McCulloch P, Steger AC, Benjamin IS, Poloniecki JD . Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data. BMJ 2003; 326: 786–788.
Sparrow JM . Monte–Carlo simulation of random clustering of endophthalmitis following cataract surgery. Eye 2007; 21: 209–213.
Allardice GM, Wright EM, Peterson M, Miller JM . A statistical approach to an outbreak of endophthalmitis following cataract surgery at a hospital in the West of Scotland. J Hosp Infect 2001; 49: 23–29.
Kelley DL . Chapter 7: Beginning to use control charts. In: How to Use Control Charts for Healthcare. ASQ Quality Press: Milwaukee, 1999.
Anuradha S, Feyi-Waboso A . Letter: The use of control charts in monitoring endophthalmitis. Acta Ophthal Scand 84 (3): 440–441.
Westgard JO, Barry PL, Hunt MR . A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem 1981; 27: 493–501.
Carey EG . Chapter 1: Basic SPC concepts and the run chart. In: Improving Healthcare with Control Charts: Basic and advanced SPC method and case studies. ASQ Quality Press: Milwaukee, 2003.
Kelley DL Chapter 6: Variable Control Charts. How to use control charts for healthcare. ASQ Quality Press: Milwaukee, 1999, p89.
Acknowledgements
Neither author has a financial or proprietary interest in the material or method mentioned in the article.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
Calculation for the lower control limit from point 1–17 (taken at 2 standard deviations)
The Moving Range (MR) mean needs to be calculated to obtain the Standard Deviation (SD). The MR is defined as the difference between the number of non-infected cases between two subsequent cases of endophthalmitis, for example, the MR between cases 1 and 2 (see Figure 1) is 807−618=189.
The mean MR for first 17 consecutive cases of endophthalmitis, y=(189+208+118+77+634+556+39+158+37+20+80+701+751+41+78+647)/16=270.875
Mean of non-infected cases between endophthalmitis cases from cases 1–17, z=(807+618+410+528+451+1085+529+490+332+369+389+469+1170+419+378+456+1103)/17=588.412
For this method of control chart calculation at 3 SD control limit, a constant A=2.659 is used (taken from the table of constant provided by Kelley 15).
SD=constant A × y/3=2.659 × 270.88/3=240.090
Therefore, the control limit at 2 SD
=z−2SD
=588.412−2(240.090)
=108.2
Calculation for the lower control limit from points 27–34 (taken at 2 SD)
The mean MR from points 27–34, a=(130+399+733+413+364+422+452)/7=416.413
Mean of non-infected cases between endophthalmitis cases from cases 27–34, b=(527+657+1056+1689+1276+912+1334+1786)/8=1154.625
SD=constant A × a/3
=2.659 × 416.413/3
=369.081
Therefore, the control limit from points 27–34
=b−2SD
=1154.625−2 (369.081)
=416.5
Rights and permissions
About this article
Cite this article
Chiam, P., Feyi-Waboso, A. The use of control charts in monitoring postcataract surgery endophthalmitis. Eye 23, 1028–1031 (2009). https://doi.org/10.1038/eye.2008.257
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/eye.2008.257
Keywords
This article is cited by
-
Intraclass reliability for assessing how well Taiwan constrained hospital-provided medical services using statistical process control chart techniques
BMC Medical Research Methodology (2012)
-
Statistical process control charts for ophthalmology
Graefe's Archive for Clinical and Experimental Ophthalmology (2011)