Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Original Article
  • Published:

Formal selection of measures for a composite index of NICU quality of care: Baby-MONITOR

Subjects

Abstract

Objective:

To systematically rate measures of care quality for very low birth weight infants for inclusion into Baby-MONITOR, a composite indicator of quality.

Study Design:

Modified Delphi expert panelist process including electronic surveys and telephone conferences. Panelists considered 28 standard neonatal intensive care unit (NICU) quality measures and rated each on a 9-point scale taking into account pre-defined measure characteristics. In addition, panelists grouped measures into six domains of quality. We selected measures by testing for rater agreement using an accepted method.

Result:

Of 28 measures considered, 13 had median ratings in the high range (7 to 9). Of these, 9 met the criteria for inclusion in the composite: antenatal steroids (median (interquartile range)) 9(0), timely retinopathy of prematurity exam 9(0), late onset sepsis 9(1), hypothermia on admission 8(1), pneumothorax 8(2), growth velocity 8(2), oxygen at 36 weeks postmenstrual age 7(2), any human milk feeding at discharge 7(2) and in-hospital mortality 7(2). Among the measures selected for the composite, the domains of quality most frequently represented included effectiveness (40%) and safety (30%).

Conclusion:

A panel of experts selected 9 of 28 routinely reported quality measures for inclusion in a composite indicator. Panelists also set an agenda for future research to close knowledge gaps for quality measures not selected for the Baby-MONITOR.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1
Figure 2

Similar content being viewed by others

References

  1. Kohn LT, Corrigan JM, Donaldson MS . Institute of Medicine. To Err Is Human. National Academy Press: Washington, DC, 1999.

    Google Scholar 

  2. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academy Press: Washington, DC, 2001.

  3. Horbar JD, Badger GJ, Lewit EM, Rogowski J, Shiono PH . Hospital and patient characteristics associated with variation in 28-day mortality rates for very low birth weight infants. Vermont Oxford Network. Pediatrics 1997; 99: 149–156.

    Article  CAS  Google Scholar 

  4. Sankaran K, Chien L-Y, Walker R, Seshia M, Ohlsson A, Lee SK . Variations in mortality rates among Canadian neonatal intensive care units. CMAJ 2002; 166: 173–178.

    PubMed  PubMed Central  Google Scholar 

  5. Rogowski JA, Staiger DO, Horbar JD . Variations in the quality of care for very-low-birthweight infants: implications for policy. Health Aff 2004; 23: 88–97.

    Article  Google Scholar 

  6. Morales LS, Staiger DO, Horbar JD, Carpenter J, Kenny M, Geppert J et al. Mortality among very low-birthweight infants in hospitals serving minority populations. Am J Public Health 2005; 95: 2206–2212.

    Article  Google Scholar 

  7. Profit J, Zupancic JA, McCormick MC, Richardson DK, Escobar GJ, Tucker J et al. Moderately premature infants at Kaiser Permanente Medical Care Program in California are discharged home earlier than their peers in Massachusetts and the United Kingdom. Arch Dis Child Fetal Neonatal Ed 2006; 91: 245–250.

    Article  Google Scholar 

  8. Phibbs CS, Baker LC, Caughey AB, Danielsen B, Schmitt SK, Phibbs RH . Level and volume of neonatal intensive care and mortality in very-low-birth-weight infants. N Engl J Med 2007; 356: 2165–2175.

    Article  CAS  Google Scholar 

  9. Institute of Medicine. Rewarding Provider Performance: Aligning Incentives in Medicare. National Academy Press: Washington, DC, 2006.

  10. Mattke S, Epstein AM, Leatherman S . The OECD Health Care Quality Indicators Project: history and background. Int J Qual Health Care 2006; 18: 1S–14.

    Article  Google Scholar 

  11. Hibbard JH, Stockard J, Tusler M . Hospital performance reports: impact on quality, market share, and reputation. Health Aff 2005; 24: 1150–1160.

    Article  Google Scholar 

  12. Epstein AM . Paying for performance in the United States and abroad. N Engl J Med 2006; 355: 406–408.

    Article  CAS  Google Scholar 

  13. Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007; 365: 486–496.

    Article  Google Scholar 

  14. Petersen LA, Woodard LD, Urech T, Daw C, Sookanan S . Does pay-for-performance improve the quality of health care? Ann Intern Med 2006; 145: 265–272.

    Article  Google Scholar 

  15. Schoen C, Davis K, How SKH, Schoenbaum SC . U.S. health system performance: a national scorecard. Health Aff 2006; 25: w457–w475.

    Article  Google Scholar 

  16. Epstein AJ . Do cardiac surgery report cards reduce mortality? Assessing the evidence. Med Care Res Rev 2006; 63: 403–426.

    Article  Google Scholar 

  17. Grossbart SR . What's the return? Assessing the effect of ‘pay-for-performance’ initiatives on the quality of care delivery. Med Care Res Rev 2006; 63: 29S–248.

    Article  Google Scholar 

  18. Petra E, Varughese P, Epifania L, Buneo L, Scarfone K . Use of quality index tracking to drive improvement in clinical outcomes. Nephrol News Issues 2006; 20: 67–83.

    PubMed  Google Scholar 

  19. Peterson ED, Delong ER, Masoudi FA, O’Brien SM, Peterson PN, Rumsfeld JS et al. ACCF/AHA 2010 Position Statement on Composite Measures for Healthcare Performance Assessment: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures (Writing Committee to develop a position statement on composite measures). Circulation 2010; 121: 1780–1791.

    Article  Google Scholar 

  20. Zaslavsky AM, Shaul JA, Zaborski LB, Cioffi MJ, Cleary PD . Combining health plan performance indicators into simpler composite measures. Health Care Financ Rev 2002; 23: 101–115.

    PubMed  PubMed Central  Google Scholar 

  21. Kaplan SH, Normand SL . Conceptual and Analytical Issues in Creating Composite Measures of Ambulatory Care Performance. Robert Wood Johnson Foundation: Washington, DC, 2006.

    Google Scholar 

  22. Profit J, Typpo KV, Hysong SJ, Woodard LD, Kallen MA, Petersen LA . Improving benchmarking by using an explicit framework for the development of composite indicators: an example using pediatric quality of care. Implement Sci 2010; 5: 13.

    Article  Google Scholar 

  23. Nardo M, Saisana M, Saltelli A, Tarantolo S, Hoffman A, Giovanini E . Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD Publishing: Paris, France, 2005, pp 1–08.

    Google Scholar 

  24. Sexton BJ, Grillo S, Fullwood C, Pronovost PJ . Assessing and improving safety culture. In: Frankel A, Leonard M, Simmonds T, Haraden C, Vega KB (eds). The Essential Guide for Patient Safety Officers. Joint Commission Resources with the Institute for Healthcare Improvement: Chicago, IL, 2009 pp 11–20.

    Google Scholar 

  25. Thomas EJ, Sherwood GD, Mulhollem JL, Sexton JB, Helmreich RL . Working together in the neonatal intensive care unit: provider perspectives. J Perinatol 2004; 24: 552–559.

    Article  Google Scholar 

  26. Jacobs R, Goddard M, Smith PC . How robust are hospital ranks based on composite performance measures? Med Care 2005; 43: 1177–1184.

    Article  Google Scholar 

  27. Halasyamani LK, Davis MM . Conflicting measures of hospital quality: ratings from ‘Hospital Compare’ versus ‘Best Hospitals’. J Hosp Med 2007; 2: 128–134.

    Article  Google Scholar 

  28. Brook RH . The RAND/UCLA Appropriateness Method. Agency for Health Care Policy and Research; Rockville, MD, 1994.

    Google Scholar 

  29. Fitch K, Bernstein SJ, Aguilar MS, Burnand B, LaCalle JR, Lazaro P et al. The RAND/UCLA Appropriateness Method User's Manual. RAND Corporation: Santa Monica, CA, 2001.

    Google Scholar 

  30. Leape LL, Hilborne LH, Kahan JP, Stason WB, Park RE, Kamberg CJ et al. Coronary Artery Bypass Graft: A Literature Review and Ratings of Appropriateness and Necessity. RAND Corporation; Santa Monica, CA, 1991.

    Google Scholar 

  31. Fitch K, Lazaro P, Aguilar MD, Kahan JP, van het LM, Bernstein SJ . European criteria for the appropriateness and necessity of coronary revascularization procedures. Eur J Cardiothorac Surg 2000; 18: 380–387.

    Article  CAS  Google Scholar 

  32. Kocher R, Sahni NR . Physicians versus hospitals as leaders of accountable care organizations. N Engl J Med 2010; 363: 2579–2582.

    Article  CAS  Google Scholar 

  33. Profit J, Zupancic JA, Gould JB, Pietz K, Petersen LA . NICU performance ratings are not sensitive to variations in definitions of mortality. E-PAS 2009; 5510: 5175.

    Google Scholar 

  34. Saisana M, Saltelli A, Tarantola S . Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. J R Stat Soc Ser A 2005; 168: 307–323.

    Article  Google Scholar 

  35. Peerzada JM, Richardson DK, Burns JP . Delivery room decision-making at the threshold of viability. J Pediatr 2004; 145: 492–498.

    Article  Google Scholar 

  36. Escobar GJ, McCormick MC, Zupancic JAF, Coleman-Phox K, Armstrong MA, Greene JD et al. Unstudied infants: outcomes of moderately premature infants in the neonatal intensive care unit. Arch Dis Child Fetal Neonatal Ed 2006; 91: F238–F244.

    Article  CAS  Google Scholar 

  37. Profit J, Petersen LA, McCormick MC, Escobar GJ, Coleman-Phox K, Zheng Z et al. Patient-to-nurse ratios and outcomes of moderately preterm infants. Pediatrics 2010; 125: 320–326.

    Article  Google Scholar 

Download references

Acknowledgements

JP, JBG, JAFZ and LAP led the design, data analysis, interpretation of results and writing of the manuscript. JAFZ, JBG and ARS participated as panelists and helped with panelist recruitment. ARS also contributed to writing of the manuscript. KMW and MAK coordinated the study, prepared study materials, assisted with data acquisition and contributed to writing of the manuscript. MM and KP undertook data analysis and interpretation. EJT contributed to the study design, interpretation of results, writing of the manuscript and moderated panel discussions. JP is guarantor of the paper. We gratefully acknowledge the contributions and effort by all expert panelists listed here alphabetically: Judy L Aschner, MD (Vanderbilt University), Reese H Clark, MD (Pediatrix Medical Group), Edward F Donovan, MD (University of Cincinnati), William H Edwards, MD (Dartmouth University), Gabriel E Escobar (Kaiser Permanente Medical Care Program), Donald A Goldmann, MD (Harvard University and Institute for Healthcare Improvement), Jeffrey H Horbar, MD (Vermont Oxford Network and University of Vermont), Martin J McCaffrey, MD (University of North Carolina, Chapel Hill), Lu-Ann Papile, MD (Baylor College of Medicine), Roger F Soll, MD (Vermont Oxford Network and University of Vermont), Jon E Tyson, MD, MPH (University of Texas, Houston), Michele C Walsh, MD (Case Western University). The following experts co-authored this manuscript: Jeffrey B Gould, MD, MPH, Ann R Stark, MD, John A Zupancic, MD, ScD. We also acknowledge the contribution and support from Dr Carl Bose in facilitating the validation survey component of this research. Jochen Profit's contribution is supported in part by the Eunice Kennedy Shriver National Institute of Child Health and Human Development K23 HD056298-01 (PI Jochen Profit, MD, MPH). Dr Petersen was a recipient of the American Heart Association Established Investigator Award (Grant number 0540043N) at the time this work was conducted. Drs Petersen, Pietz, and Mr Mei also receive support from a Veterans Administration Center Grant (VA HSR&D CoE HFP90-20). Dr Thomas is supported by Eunice Kennedy Shriver National Institute of Child Health and Human Development Grant K24 HD053771.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to J Profit.

Ethics declarations

Competing interests

Drs Profit, Zupancic and Gould will serve as Expert Consultants with the Vermont Oxford Network's NICQ 7 Quality Improvement Collaborative. Dr Gould is the principal investigator for the California Perinatal Quality Care Collaborative.

Additional information

The results of this study were presented as a poster at the Pediatric Academic Societies’ Annual Meeting in Baltimore on 5 May 2009.

Supplementary Information accompanies the paper on the Journal of Perinatology website

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Cite this article

Profit, J., Gould, J., Zupancic, J. et al. Formal selection of measures for a composite index of NICU quality of care: Baby-MONITOR. J Perinatol 31, 702–710 (2011). https://doi.org/10.1038/jp.2011.12

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/jp.2011.12

Keywords

This article is cited by

Search

Quick links