Metrics for NICU antibiotic use: which rate is right?

For more than a decade, investigators have interrogated the Medicare database to learn about relationships between health care resource use and outcomes.¹ As a rule, studies find wide variation in resource use and little relationship with patient outcomes;¹ thereby challenging the belief that directing incrementally more resources at certain healthcare problems necessarily produces incrementally better results. More care is not always better.¹ Not surprisingly, these findings have fueled the debate concerning recent federal health care reform legislation and how to optimally deliver and measure quality care.² The research strategies typically examine resource use and outcomes together rather than individually, along with adjusting appropriately for severity of illness and other pertinent sources of potential bias (for example, length of follow-up) and confounding (for example, socioeconomic status; different healthcare providers).

Similar research strategies have the potential to assist neonatal care providers to optimize resource use and provide improved outcomes for infants hospitalized in the Neonatal intensive care unit (NICU). To this end, in this issue Wirtschafter et al.³ invite reflection on how to measure how much, how often, and why antibiotics are used in the NICU. These investigators also highlight the relatively simplistic database design and analytical methods currently applied to such neonatal health services research. We know that wide variation in outcomes exists among NICUs.^{4, 5, 6, 7} Wirtschafter et al.³ direct our attention to the potentially wide variation in resource use among NICUs. The charge now is to build on current knowledge by connecting adjusted NICU resource use with NICU patient outcomes. Only then can we thoughtfully grapple with the question ‘Which rate is right?’

Do NICUs vary widely in their use of antibiotics? It is difficult to generalize from the findings of Wirtschafter et al.³ However, it is known that antibiotic use varies widely across children's hospitals, even after adjusting for hospital- and patient-level demographic and clinical characteristics typically associated with appropriate antibiotic use.⁸ If antibiotic use does vary widely among NICUs, how much is too much—or too little? Of course, if appropriate antibiotic treatment correlated perfectly with documented infection then the investigative focus would be directed toward variation in infection rates—already a subject of much scrutiny, including by neonatal collaboratives across North America.^{5, 7, 9} Furthermore, as Wirtschafter et al.³ demonstrate, treatment of definitive infections represents the minority of antibiotic use in the NICU population.

By describing overall rates of NICU antibiotic use, Wirtschafter et al.³ have opened a discussion about an aspect of NICU care for which evaluation and benchmarking are quite problematic. When infection is difficult to prove, such as in the case of culture negative sepsis, what constitutes overuse or underuse of antibiotics? We make the following analogy. There is an acceptable rate for surgical removal of un-inflamed appendices from patients with an acute abdomen. However, if a negative pathology report had poor specificity (true negative rate), how would surgeons be able to develop this acceptable rate and know if they have operated appropriately? Measuring NICU antibiotic prescribing practices suffers from such a constraint: negative cultures may not indicate there is not an infection.

How do we know that the rate at which we start neonates on empiric antibiotic treatment for late onset sepsis, while we await further clinical and laboratory feedback, is appropriate? We have some idea about how widely the incidence of bacteremia varies,^{4, 5, 10, 11, 12, 13} but we lack similar reliable data for the variation in rate for rule-out sepsis (treatment for ⩽4 days) or presumed sepsis (treatment for ⩾5 days). We are similarly ignorant about the sensitivity (true positive rate) of these diagnostic categories—whether stratified by individual provider or by NICU. Moreover, as Wirtschafter et al.³ described, these diagnostic categories accounted for most of the antibiotic use, but the rate of use for these indications varied among the study NICUs.

If antibiotic use does vary widely across NICUs, does it matter? The importance of treating true infections is obvious, but it is also important to scrutinize variations in antibiotic prescribing that could represent inappropriate use. We have yet to understand the full range of harms that could be the result of exposure to antibiotics, but some of the risks are well known and presumably related to alterations in the microbiome as described further below. Antimicrobial use is associated with the selection of multidrug-resistant pathogens, which are themselves associated with increased morbidity, mortality, cost and length of stay.¹⁴ Use of broad spectrum antibiotics (third generation cephalosporin and carbapenem agents) in very low birth weight infants is associated with an increased risk of candidemia, which is associated with increased mortality and neurodevelomental delay.¹⁴ Treatment of culture negative early onset sepsis is associated with the development of necrotizing enterocolitis.¹⁵ Thus, appropriate antibiotic use, which rate is right, could be measured both quantitatively and qualitatively.

As alluded to above, it may be difficult to fully elucidate (at present) the effect of antibiotics on the microbiome, but consider this. There are 10 times more microbial cells in the human gastrointestinal tract than eukaryotic cells in an individual human¹⁶ and 99% of the DNA in the gastrointestinal tract is microbial!¹⁷ In the coming years, it is vital to better understand both the short- and long-term effects of the complex risks and benefits of suppression/alteration of normal flora with antibiotic treatment.

In recognition of the need to both preserve antimicrobial agents in an era of increasing antimicrobial resistance as well as the need to use antibiotics appropriately, antimicrobial stewardship guidelines, that is, the appropriate selection, dosing, route and duration of antimicrobial therapy, have been endorsed by the American Academy of Pediatrics, the Centers for Disease Control and Prevention, the Society of Healthcare Epidemiologists of America, and the Infectious Disease Society of America.¹⁸ Moreover, these organizations consider the appropriate use of antimicrobials to be an essential part of patient safety.¹⁸ Nonetheless, gaps appear to exist between current NICU practice and available best evidence about appropriate use of antimicrobials.^{19, 20} We have much to learn about how to operationally define ‘appropriate.’

Future studies that seek to answer which rate of antibiotic use is right will require adjusting the relationship between antibiotic use and outcomes for bias and confounding. We illustrate how complex and challenging this task will be by providing factors to consider. Do patient characteristics differ between high and low user rate NICUs? Have all the important analytically pertinent characteristics, and only the analytically pertinent ones, been accurately measured and accounted for? A more highly specified model is not always a better model; ill-chosen model variables can obscure or distort true relationships. Do provider characteristics differ between high and low user rate NICUs? What is the relationship between the rate of antibiotic use and the rate of documented infection and/or necrotizing enterocolitis rate? Are there geographic, temporal or seasonal trends in antibiotic use? What is the impact of antibiotic exposure on patients’ microbial flora individually, within, and among NICUs? Is there a relationship between the rate of antibiotic use and NICU bed capacity/availability, staffing levels, and/or third party payment arrangements? How much of the observed variation in antibiotic use across NICUs does the model explain? After adjustment, what proportion of the variation may be due to chance alone?

In conclusion, Wirtschafter et al.³ have asked provocative questions about how to measure antibiotic use and encouraged further discussions of antibiotic stewardship in the NICU. ‘When different physicians are recommending different things for essentially the same patients, it is impossible to claim that they are all doing the right thing.’²¹ We have a fundamental responsibility to our patients to learn which rate of antimicrobial resource use is right.