Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Clinical Research Article
  • Published:

An all-inclusive model for predicting invasive bacterial infection in febrile infants age 7–60 days

Pediatric Research (2024)Cite this article

Abstract

Background

Invasive bacterial infections (IBIs) in febrile infants are rare but potentially devastating. We aimed to derive and validate a predictive model for IBI among febrile infants age 7–60 days.

Methods

Data were abstracted retrospectively from electronic records of 37 emergency departments (EDs) for infants with a measured temperature >=100.4 F who underwent an ED evaluation with blood and urine cultures. Models to predict IBI were developed and validated respectively using a random 80/20 dataset split, including 10-fold cross-validation. We used precision recall curves as the classification metric.

Results

Of 4411 eligible infants with a mean age of 37 days, 29% had characteristics that would likely have excluded them from existing risk stratification protocols. There were 196 patients with IBI (4.4%), including 43 (1.0%) with bacterial meningitis. Analytic approaches varied in performance characteristics (precision recall range 0.04–0.29, area under the curve range 0.5–0.84), with the XGBoost model demonstrating the best performance (0.29, 0.84). The five most important variables were serum white blood count, maximum temperature, absolute neutrophil count, absolute band count, and age in days.

Conclusion

A machine learning model (XGBoost) demonstrated the best performance in predicting a rare outcome among febrile infants, including those excluded from existing algorithms.

Impact

  • Several models for the risk stratification of febrile infants have been developed. There is a need for a preferred comprehensive model free from limitations and algorithm exclusions that accurately predicts IBIs.

  • This is the first study to derive an all-inclusive predictive model for febrile infants aged 7–60 days in a community ED sample with IBI as a primary outcome.

  • This machine learning model demonstrates potential for clinical utility in predicting IBI.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Study Cohort Assembly.
Fig. 2: Precision recall and receiver operating curves.
Fig. 3: Relative contributions to Predictive Model.

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. McCaig, L. F. & Nawar, E. W. National Hospital Ambulatory Medical Care Survey: 2004 Emergency Department Summary. Adv. Data, 372, 1–29 (2006).

  2. Aronson, P. L. et al. Variation in care of the febrile young infant <90 days in US Pediatric Emergency Departments. Pediatrics 134, 667–677 (2014).

    Article  PubMed  Google Scholar 

  3. Greenhow, T. L., Hung, Y. Y. & Herz, A. M. Changing Epidemiology of Bacteremia in infants aged 1 week to 3 months. Pediatrics 129, e590–e596 (2012).

    Article  PubMed  Google Scholar 

  4. Kuppermann, N. et al. A clinical prediction rule to identify febrile infants 60 days and younger at low risk for serious bacterial infections. JAMA Pediatr. 173, 342–351 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Nigrovic, L. E. et al. The Yale observation scale score and the risk of serious bacterial infections in febrile infants. Pediatrics 140, e20170695 (2017).

    Article  PubMed  Google Scholar 

  6. Greenhow, T. L., Hung, Y. Y., Herz, A. M., Losada, E. & Pantell, R. H. The changing epidemiology of serious bacterial infections in young infants. Pediatr. Infect. Dis. J. 33, 595–599 (2014).

    Article  PubMed  Google Scholar 

  7. Woll, C. et al. Epidemiology and etiology of invasive bacterial infection in infants </=60 days old treated in emergency departments. J. Pediatr. 200, 210–217 e211 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Pantell, R. H. et al. Evaluation and management of well-appearing febrile infants 8 to 60 days old. Pediatrics 148, e2021052228 (2021).

    Article  PubMed  Google Scholar 

  9. Fisher, K. A., Landyn, V., Lindenauer, P. K. & Walkey, A. J. Procalcitonin test availability: a survey of acute care hospitals in Massachusetts. Ann. Am. Thorac. Soc. 14, 1489–1491 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  10. Burstein, B., Alathari, N. & Papenburg, J. Guideline-based risk stratification for febrile young infants without procalcitonin measurement. Pediatrics 149, e2021056028 (2022).

    Article  PubMed  Google Scholar 

  11. Aronson, P. L. et al. Association of Clinical Practice Guidelines with Emergency Department Management of Febrile Infants </=56 Days of Age. J. Hosp. Med. 10, 358–365 (2015).

    Article  PubMed  Google Scholar 

  12. Nguyen, T. H. P., Young, B. R., Poggel, L. E., Alabaster, A. & Greenhow, T. L. Roseville Protocol for the Management of Febrile Infants 7-60 Days. Hosp Pediatr 11, 52–60 (2021).

  13. Powell, E. C. et al. Epidemiology of bacteremia in febrile infants aged 60 days and younger. Ann. Emerg. Med 71, 211–216 (2018).

    Article  PubMed  Google Scholar 

  14. Tzimenatos, L. et al. Accuracy of the urinalysis for urinary tract infections in febrile infants 60 days and younger. Pediatrics 141, e20173068 (2018).

    Article  PubMed  Google Scholar 

  15. Goldstein, B. A., Navar, A. M. & Carter, R. E. Moving beyond regression techniques in cardiovascular risk prediction: applying machine learning to address analytic challenges. Eur. Heart J. 38, 1805–1814 (2017).

    PubMed  Google Scholar 

  16. Writing Committee Members. Writing Committee 2021 AHA/ACC/ASE/Chest/SAEM/SCCT/SCMR Guideline for the Evaluation and Diagnosis of Chest Pain: Executive Summary: A Report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. J. Am. Coll. Cardiol. 78, 2218–2261 (2021).

    Article  Google Scholar 

  17. Frizzell, J. D. et al. Prediction of 30-Day all-cause readmissions in patients hospitalized for heart failure: comparison of machine learning and other statistical approaches. JAMA Cardiol. 2, 204–209 (2017).

    Article  PubMed  Google Scholar 

  18. Grant L, J. P., Nemnom, M. J. & Thiruganasambandamoorthy, V. Machine learning versus traditional methods for the development of risk stratification scores: a case study using original canadian syncope risk score data. Intern. Emerg. Med. 17, 1145–1153 (2022).

    Article  PubMed  Google Scholar 

  19. Dorney, K. & Bachur, R. G. Febrile Infant Update. Curr. Opin. Pediatr. 29, 280–285 (2017).

    Article  CAS  PubMed  Google Scholar 

  20. Ramgopal, S., Horvat, C. M., Yanamala, N. & Alpern, E. R. Machine learning to predict serious bacterial infections in young febrile infants. Pediatrics 146, e20194096 (2020).

  21. Yaeger, J. P. et al. Using clinical history factors to identify bacterial infections in young febrile infants. J. Pediatrics 232, 192–199. e192 (2021).

    Article  Google Scholar 

  22. Gordon, N. & Lin, T. The Kaiser Permanente Northern California Adult Member Health Survey. Perm. J. 20, 15–225 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  23. Davis, A. C., Voelkel, J. L., Remmers, C. L., Adams, J. L. & McGlynn, E. A. Comparing Kaiser Permanente members to the general population: implications for generalizability of Research. Perm. J. 27, 87 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  24. Association, W. M. World Medical Association Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects. Jama 310, 2191–2194 (2013).

    Article  Google Scholar 

  25. Feudtner, C., Feinstein, J. A., Zhong, W., Hall, M. & Dai, D. Pediatric complex chronic conditions classification system version 2: updated for icd-10 and complex medical technology dependence and transplantation. BMC Pediatr. 14, 199 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  26. Escobar, G. J. et al. Prediction of obstetrical and fetal complications using automated electronic health record data. Am. J. Obstet. Gynecol. 224, 137–147 e137 (2021).

    Article  PubMed  Google Scholar 

  27. Steyerberg, E. W. & Vergouwe, Y. Towards better clinical prediction models: seven steps for development and an Abcd for validation. Eur. Heart J. 35, 1925–1931 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  28. Steyerberg, E. W. et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21, 128–138 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Vickers, A. J., Van Calster, B. & Steyerberg, E. W. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ 352, i6 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  30. Bertsimas, D., Dunn, J., Steele, D. W., Trikalinos, T. A. & Wang, Y. Comparison of machine learning optimal classification trees with the pediatric emergency care applied research network head trauma decision rules. JAMA Pediatr. 173, 648–656 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Nguyen, T. H. et al. Using AAP guidelines for managing febrile infants without C-reactive protein and procalcitonin. Pediatrics 151, e2022058495 (2022).

  32. Neal, J. T. et al. The effect of bedside ultrasonographic skin marking on infant lumbar puncture success: a randomized controlled trial. Ann. Emerg. Med 69, 610–619 e611 (2017).

    Article  PubMed  Google Scholar 

  33. Foster, L. Z. et al. Implementation of Febrile Infant Management Guidelines reduces hospitalization. Pediatr. Qual. Saf. 5, e252 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  34. Price, W. N. 2nd, Gerke, S. & Cohen, I. G. Potential liability for physicians using artificial intelligence. JAMA 322, 1765–1766 (2019).

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Research reported in this publication was supported by the Kaiser Permanente Garfield Memorial Fund. We would also like to thank the KPNC Division of Research and the KPSC Care Improvement Research Team for their infrastructure support.

Prior presentation/publication

May 2022 at the Society for Academic Emergency Medicine Annual Meeting in New Orleans, LA.

Author information

Authors and Affiliations

  1. The Permanente Medical Group, Oakland, CA, USA

    Dustin W. Ballard, Dustin G. Mark, David R. Vinson, Mamata V. Kene & Tara L. Greenhow

  2. Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA

    Dustin W. Ballard, Jie Huang, Dustin G. Mark, David R. Vinson, Adina S. Rauchwerger, Jennifer Y. Zhang & Mary E. Reed

  3. Kaiser Permanente Bernard J. Tyson School of Medicine, Health Systems Science Department, Pasadena, CA, USA

    Adam L. Sharp

  4. Department of Hospital Pediatrics, Kaiser Permanente Northern California, Roseville, CA, USA

    Tran H. P. Nguyen & Beverly R. Young

  5. Department of Pediatrics, Kaiser Permanente Southern California, Anaheim, CA, USA

    Patrick Van Winkle

  6. Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA, USA

    Stacy J. Park

  7. Division of Infectious Diseases, Department of Pediatrics, Kaiser Permanente Northern California, San Francisco, CA, USA

    Tara L. Greenhow

Authors
  1. Dustin W. Ballard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adam L. Sharp
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dustin G. Mark
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tran H. P. Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Beverly R. Young
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. David R. Vinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Patrick Van Winkle
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Mamata V. Kene
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Adina S. Rauchwerger
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jennifer Y. Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Stacy J. Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Mary E. Reed
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Tara L. Greenhow
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

DWB conceptualized and designed the study, collected data, drafted the initial manuscript, and reviewed and revised the manuscript. MER, JH, TLG, ALS, AND DGM conceptualized and designed the study, collected data, and critically reviewed and revised the manuscript. DRV, THPN, BRY, PVW, AND MVK, conceptualized and designed the study, and critically reviewed and revised the manuscript. ASR, JYZ, AND SJP assisted in conceptualizing and designing the study, designed the data collection instruments, coordinated data collection, and reviewed and revised the manuscript. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

Corresponding author

Correspondence to Dustin W. Ballard.

Ethics declarations

Competing interests

The authors declare no competing interests.

Consent statement

A waiver of informed consent was granted by the Kaiser Permanente Northern California Institutional Review Board.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ballard, D.W., Huang, J., Sharp, A.L. et al. An all-inclusive model for predicting invasive bacterial infection in febrile infants age 7–60 days. Pediatr Res (2024). https://doi.org/10.1038/s41390-024-03141-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41390-024-03141-3

Search

Advanced search

Quick links