1. ¹Division of Newborn Medicine, Department of Pediatrics, Weill Cornell Medical College, New York Presbyterian Hospital/Weill Cornell Medical Center, New York, NY, USA
2. ²Division of Outcomes and Effectiveness, Department of Public Health, Weill Cornell Medical College, New York Presbyterian Hospital/Weill Cornell Medical Center, New York, NY, USA
3. ³MRC Biostatistics Unit, Institute of Public Health, University Forvie Site, Cambridge, UK
4. ⁴Department of Medicine, Children's Hospital Boston, Boston, MA, USA

Correspondence: Dr J Schulman, Division of Newborn Medicine, Department of Pediatrics, Weill Cornell Medical College, New York Presbyterian Hospital/Weill Cornell Medical Center, 525 East 68th Street, Box 106, New York, NY 10065, USA. E-mail: jos2039@med.cornell.edu

Received 3 December 2007; Revised 13 April 2008; Accepted 14 April 2008; Published online 17 July 2008.

Abstract

Comparative performance reports continue to proliferate, so it is increasingly important that healthcare workers can interpret the graphically displayed results correctly. This article acquaints readers with key concepts for thinking clearly and critically about such displays: (1) articulating the question a display answers, along with reflecting on questions the display might appear to, but does not, answer; (2) establishing that provider comparisons are made fairly, ever mindful of methodological assumptions and limitations; (3) accounting for systematic differences among performers that are unexplained by specified predictors, that is, random effect methods that yield 'shrunken' estimates; (4) understanding funnel plots used to summarize complex analyses and how one may vary the interrogative focus so that 'outlier' values most likely signal extraordinary performance. Finally, these concepts are given broader context in a view of the ultimate aim of the evaluative enterprise.