Abstract
Objective:
To systematically rate measures of care quality for very low birth weight infants for inclusion into Baby-MONITOR, a composite indicator of quality.
Study Design:
Modified Delphi expert panelist process including electronic surveys and telephone conferences. Panelists considered 28 standard neonatal intensive care unit (NICU) quality measures and rated each on a 9-point scale taking into account pre-defined measure characteristics. In addition, panelists grouped measures into six domains of quality. We selected measures by testing for rater agreement using an accepted method.
Result:
Of 28 measures considered, 13 had median ratings in the high range (7 to 9). Of these, 9 met the criteria for inclusion in the composite: antenatal steroids (median (interquartile range)) 9(0), timely retinopathy of prematurity exam 9(0), late onset sepsis 9(1), hypothermia on admission 8(1), pneumothorax 8(2), growth velocity 8(2), oxygen at 36 weeks postmenstrual age 7(2), any human milk feeding at discharge 7(2) and in-hospital mortality 7(2). Among the measures selected for the composite, the domains of quality most frequently represented included effectiveness (40%) and safety (30%).
Conclusion:
A panel of experts selected 9 of 28 routinely reported quality measures for inclusion in a composite indicator. Panelists also set an agenda for future research to close knowledge gaps for quality measures not selected for the Baby-MONITOR.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Kohn LT, Corrigan JM, Donaldson MS . Institute of Medicine. To Err Is Human. National Academy Press: Washington, DC, 1999.
Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academy Press: Washington, DC, 2001.
Horbar JD, Badger GJ, Lewit EM, Rogowski J, Shiono PH . Hospital and patient characteristics associated with variation in 28-day mortality rates for very low birth weight infants. Vermont Oxford Network. Pediatrics 1997; 99: 149–156.
Sankaran K, Chien L-Y, Walker R, Seshia M, Ohlsson A, Lee SK . Variations in mortality rates among Canadian neonatal intensive care units. CMAJ 2002; 166: 173–178.
Rogowski JA, Staiger DO, Horbar JD . Variations in the quality of care for very-low-birthweight infants: implications for policy. Health Aff 2004; 23: 88–97.
Morales LS, Staiger DO, Horbar JD, Carpenter J, Kenny M, Geppert J et al. Mortality among very low-birthweight infants in hospitals serving minority populations. Am J Public Health 2005; 95: 2206–2212.
Profit J, Zupancic JA, McCormick MC, Richardson DK, Escobar GJ, Tucker J et al. Moderately premature infants at Kaiser Permanente Medical Care Program in California are discharged home earlier than their peers in Massachusetts and the United Kingdom. Arch Dis Child Fetal Neonatal Ed 2006; 91: 245–250.
Phibbs CS, Baker LC, Caughey AB, Danielsen B, Schmitt SK, Phibbs RH . Level and volume of neonatal intensive care and mortality in very-low-birth-weight infants. N Engl J Med 2007; 356: 2165–2175.
Institute of Medicine. Rewarding Provider Performance: Aligning Incentives in Medicare. National Academy Press: Washington, DC, 2006.
Mattke S, Epstein AM, Leatherman S . The OECD Health Care Quality Indicators Project: history and background. Int J Qual Health Care 2006; 18: 1S–14.
Hibbard JH, Stockard J, Tusler M . Hospital performance reports: impact on quality, market share, and reputation. Health Aff 2005; 24: 1150–1160.
Epstein AM . Paying for performance in the United States and abroad. N Engl J Med 2006; 355: 406–408.
Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007; 365: 486–496.
Petersen LA, Woodard LD, Urech T, Daw C, Sookanan S . Does pay-for-performance improve the quality of health care? Ann Intern Med 2006; 145: 265–272.
Schoen C, Davis K, How SKH, Schoenbaum SC . U.S. health system performance: a national scorecard. Health Aff 2006; 25: w457–w475.
Epstein AJ . Do cardiac surgery report cards reduce mortality? Assessing the evidence. Med Care Res Rev 2006; 63: 403–426.
Grossbart SR . What's the return? Assessing the effect of ‘pay-for-performance’ initiatives on the quality of care delivery. Med Care Res Rev 2006; 63: 29S–248.
Petra E, Varughese P, Epifania L, Buneo L, Scarfone K . Use of quality index tracking to drive improvement in clinical outcomes. Nephrol News Issues 2006; 20: 67–83.
Peterson ED, Delong ER, Masoudi FA, O’Brien SM, Peterson PN, Rumsfeld JS et al. ACCF/AHA 2010 Position Statement on Composite Measures for Healthcare Performance Assessment: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures (Writing Committee to develop a position statement on composite measures). Circulation 2010; 121: 1780–1791.
Zaslavsky AM, Shaul JA, Zaborski LB, Cioffi MJ, Cleary PD . Combining health plan performance indicators into simpler composite measures. Health Care Financ Rev 2002; 23: 101–115.
Kaplan SH, Normand SL . Conceptual and Analytical Issues in Creating Composite Measures of Ambulatory Care Performance. Robert Wood Johnson Foundation: Washington, DC, 2006.
Profit J, Typpo KV, Hysong SJ, Woodard LD, Kallen MA, Petersen LA . Improving benchmarking by using an explicit framework for the development of composite indicators: an example using pediatric quality of care. Implement Sci 2010; 5: 13.
Nardo M, Saisana M, Saltelli A, Tarantolo S, Hoffman A, Giovanini E . Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD Publishing: Paris, France, 2005, pp 1–08.
Sexton BJ, Grillo S, Fullwood C, Pronovost PJ . Assessing and improving safety culture. In: Frankel A, Leonard M, Simmonds T, Haraden C, Vega KB (eds). The Essential Guide for Patient Safety Officers. Joint Commission Resources with the Institute for Healthcare Improvement: Chicago, IL, 2009 pp 11–20.
Thomas EJ, Sherwood GD, Mulhollem JL, Sexton JB, Helmreich RL . Working together in the neonatal intensive care unit: provider perspectives. J Perinatol 2004; 24: 552–559.
Jacobs R, Goddard M, Smith PC . How robust are hospital ranks based on composite performance measures? Med Care 2005; 43: 1177–1184.
Halasyamani LK, Davis MM . Conflicting measures of hospital quality: ratings from ‘Hospital Compare’ versus ‘Best Hospitals’. J Hosp Med 2007; 2: 128–134.
Brook RH . The RAND/UCLA Appropriateness Method. Agency for Health Care Policy and Research; Rockville, MD, 1994.
Fitch K, Bernstein SJ, Aguilar MS, Burnand B, LaCalle JR, Lazaro P et al. The RAND/UCLA Appropriateness Method User's Manual. RAND Corporation: Santa Monica, CA, 2001.
Leape LL, Hilborne LH, Kahan JP, Stason WB, Park RE, Kamberg CJ et al. Coronary Artery Bypass Graft: A Literature Review and Ratings of Appropriateness and Necessity. RAND Corporation; Santa Monica, CA, 1991.
Fitch K, Lazaro P, Aguilar MD, Kahan JP, van het LM, Bernstein SJ . European criteria for the appropriateness and necessity of coronary revascularization procedures. Eur J Cardiothorac Surg 2000; 18: 380–387.
Kocher R, Sahni NR . Physicians versus hospitals as leaders of accountable care organizations. N Engl J Med 2010; 363: 2579–2582.
Profit J, Zupancic JA, Gould JB, Pietz K, Petersen LA . NICU performance ratings are not sensitive to variations in definitions of mortality. E-PAS 2009; 5510: 5175.
Saisana M, Saltelli A, Tarantola S . Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. J R Stat Soc Ser A 2005; 168: 307–323.
Peerzada JM, Richardson DK, Burns JP . Delivery room decision-making at the threshold of viability. J Pediatr 2004; 145: 492–498.
Escobar GJ, McCormick MC, Zupancic JAF, Coleman-Phox K, Armstrong MA, Greene JD et al. Unstudied infants: outcomes of moderately premature infants in the neonatal intensive care unit. Arch Dis Child Fetal Neonatal Ed 2006; 91: F238–F244.
Profit J, Petersen LA, McCormick MC, Escobar GJ, Coleman-Phox K, Zheng Z et al. Patient-to-nurse ratios and outcomes of moderately preterm infants. Pediatrics 2010; 125: 320–326.
Acknowledgements
JP, JBG, JAFZ and LAP led the design, data analysis, interpretation of results and writing of the manuscript. JAFZ, JBG and ARS participated as panelists and helped with panelist recruitment. ARS also contributed to writing of the manuscript. KMW and MAK coordinated the study, prepared study materials, assisted with data acquisition and contributed to writing of the manuscript. MM and KP undertook data analysis and interpretation. EJT contributed to the study design, interpretation of results, writing of the manuscript and moderated panel discussions. JP is guarantor of the paper. We gratefully acknowledge the contributions and effort by all expert panelists listed here alphabetically: Judy L Aschner, MD (Vanderbilt University), Reese H Clark, MD (Pediatrix Medical Group), Edward F Donovan, MD (University of Cincinnati), William H Edwards, MD (Dartmouth University), Gabriel E Escobar (Kaiser Permanente Medical Care Program), Donald A Goldmann, MD (Harvard University and Institute for Healthcare Improvement), Jeffrey H Horbar, MD (Vermont Oxford Network and University of Vermont), Martin J McCaffrey, MD (University of North Carolina, Chapel Hill), Lu-Ann Papile, MD (Baylor College of Medicine), Roger F Soll, MD (Vermont Oxford Network and University of Vermont), Jon E Tyson, MD, MPH (University of Texas, Houston), Michele C Walsh, MD (Case Western University). The following experts co-authored this manuscript: Jeffrey B Gould, MD, MPH, Ann R Stark, MD, John A Zupancic, MD, ScD. We also acknowledge the contribution and support from Dr Carl Bose in facilitating the validation survey component of this research. Jochen Profit's contribution is supported in part by the Eunice Kennedy Shriver National Institute of Child Health and Human Development K23 HD056298-01 (PI Jochen Profit, MD, MPH). Dr Petersen was a recipient of the American Heart Association Established Investigator Award (Grant number 0540043N) at the time this work was conducted. Drs Petersen, Pietz, and Mr Mei also receive support from a Veterans Administration Center Grant (VA HSR&D CoE HFP90-20). Dr Thomas is supported by Eunice Kennedy Shriver National Institute of Child Health and Human Development Grant K24 HD053771.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
Drs Profit, Zupancic and Gould will serve as Expert Consultants with the Vermont Oxford Network's NICQ 7 Quality Improvement Collaborative. Dr Gould is the principal investigator for the California Perinatal Quality Care Collaborative.
Additional information
The results of this study were presented as a poster at the Pediatric Academic Societies’ Annual Meeting in Baltimore on 5 May 2009.
Supplementary Information accompanies the paper on the Journal of Perinatology website
Supplementary information
Rights and permissions
About this article
Cite this article
Profit, J., Gould, J., Zupancic, J. et al. Formal selection of measures for a composite index of NICU quality of care: Baby-MONITOR. J Perinatol 31, 702–710 (2011). https://doi.org/10.1038/jp.2011.12
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/jp.2011.12
Keywords
This article is cited by
-
Measuring quality of care in moderate and late preterm infants
Journal of Perinatology (2022)
-
Prioritizing suicide prevention guideline recommendations in specialist mental healthcare: a Delphi study
BMC Psychiatry (2020)
-
Burnout in the neonatal intensive care unit and its relation to healthcare-associated infections
Journal of Perinatology (2017)
-
Quality use of medicines in neonatal care: a review of measures of quality used to evaluate the appropriateness and rational use of medication within the NICU
Drugs & Therapy Perspectives (2016)
-
The swiss neonatal quality cycle, a monitor for clinical performance and tool for quality improvement
BMC Pediatrics (2013)