Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Workload In Neonatology (WORKLINE): Validation and feasibility of a system for measuring clinician workload integrated into the electronic health record

Journal of Perinatology volume 43, pages 936–942 (2023)Cite this article

Subjects

Abstract

Objective

The purpose of the study was to validate WORKLINE, a NICU specific clinician workload model and to evaluate the feasibility of integrating WORKLINE into our EHR.

Study design

This was a prospective, observational study of the workload of 42 APPs and physicians in a large academic medical center NICU over a 6-month period. We used regression models with robust clustered standard errors to test associations of WORKLINE values with NASA Task Load Index (NASA-TLX) scores.

Results

We found significant correlations between WORKLINE and NASA-TLX scores. APP caseload was not significantly associated with WORKLINE scores. We successfully integrated the WORKLINE model into our EHR to automatically generate workload scores.

Conclusion

WORKLINE provides an objective method to quantify the workload of clinicians in the NICU, and for APPs, performed better than caseload numbers to reflect workload. Integrating the WORKLINE model into the EHR was feasible and enabled automated workload scores.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Holden RJ, Carayon P, Gurses AP, Hoonakker P, Hundt AS, Ozok AA, et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics. 2013;56:1669–86.

    Article  PubMed  Google Scholar 

  2. Pillay T, Nightingale P, Owen S, Kirby D, Spencer A. Neonatal nurse staffing and delivery of clinical care in the SSBC Newborn Network. Arch Dis Child Fetal Neonatal Ed. 2012;97:F174–8.

    Article  PubMed  Google Scholar 

  3. Sherenian M, Profit J, Schmidt B, Suh S, Xiao R, Zupancic JA, et al. Nurse-to-patient ratios and neonatal outcomes: a brief systematic review. Neonatology. 2013;104:179–83.

    Article  PubMed  Google Scholar 

  4. Watson SI, Arulampalam W, Petrou S, Marlow N, Morgan AS, Draper ES, et al. The effects of a one-to-one nurse-to-patient ratio on the mortality rate in neonatal intensive care: a retrospective, longitudinal, population-based study. Arch Dis Child Fetal Neonatal Ed. 2016;101:F195–200.

    Article  CAS  PubMed  Google Scholar 

  5. Rogowski JA, Staiger D, Patrick T, Horbar J, Kenny M, Lake ET. Nurse staffing and NICU infection rates. JAMA Pediatr. 2013;167:444–50.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Tubbs-Cooley HL, Mara CA, Carle AC, Mark BA, Pickler RH. Association of nurse workload with missed nursing care in the neonatal intensive care unit. JAMA Pediatr. 2019;173:44–51.

    Article  PubMed  Google Scholar 

  7. Neuraz A, Guerin C, Payet C, Polazzi S, Aubrun F, Dailler F, et al. Patient mortality is associated with staff resources and workload in the ICU: A Multicenter observational study. Crit Care Med. 2015;43:1587–94.

    Article  PubMed  Google Scholar 

  8. Steyrer J, Schiffinger M, Huber C, Valentin A, Strunk G. Attitude is everything? The impact of workload, safety climate, and safety tools on medical errors: a study of intensive care units. Health Care Manag Rev. 2013;38:306–16.

    Article  Google Scholar 

  9. Chana N, Kennedy P, Chessell ZJ. Nursing staffs’ emotional well-being and caring behaviours. J Clin Nurs. 2015;24:2835–48.

    Article  PubMed  Google Scholar 

  10. Purcell SR, Kutash M, Cobb S. The relationship between nurses’ stress and nurse staffing factors in a hospital setting. J Nurs Manag. 2011;19:714–20.

    Article  PubMed  Google Scholar 

  11. Harry E, Sinsky C, Dyrbye LN, Makowski MS, Trockel M, Tutty M, et al. Physician task load and the risk of burnout among US physicians in a national survey. Jt Comm J Qual Patient Saf. 2021;47:76–85.

    PubMed  Google Scholar 

  12. Krajnak KM. Potential contribution of work-related psychosocial stress to the development of cardiovascular disease and type II diabetes: A brief review. Environ Health Insights. 2014;8:41–5.

    PubMed  PubMed Central  Google Scholar 

  13. Slopen N, Glynn RJ, Buring JE, Lewis TT, Williams DR, Albert MA. Job strain, job insecurity, and incident cardiovascular disease in the Women’s Health Study: results from a 10-year prospective study. PloS One. 2012;7:e40512.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Dye E, Wells N. Subjective and objective measurement of neonatal nurse practitioner workload. Adv Neonatal Care. 2017;7:E3–E12.

    Article  Google Scholar 

  15. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inf. 2009;42:377–81.

    Article  Google Scholar 

  16. Human Performance Research Group. NASA Task Load Index (TLX): Paper and pencil package: NASA Ames Research Center. 1986. http://humansystems.arc.nasa.gov/groups/TLX/downloads/TLX_pappen_manual.pdf.

  17. Hoonakker P, Carayon P, Gurses A, Brown R, McGuire K, Khunlertkit A, et al. Measuring workload of ICU nurses with a questionnaire survey: The NASA task load Index (TLX). IIE Trans Health Syst Eng. 2011;1:131–43.

    Article  Google Scholar 

  18. Young G, Zavelina L, Hooper V. Assessment of workload using NASA Task Load Index in perianesthesia nursing. J Perianesth Nurs. 2008;23:102–10.

    Article  PubMed  Google Scholar 

  19. France DJ, Levin S, Hemphill R, Chen K, Rickard D, Makowski R, et al. Emergency physicians’ behaviors and workload in the presence of an electronic whiteboard. Int J Med Inf. 2005;74:827–37.

    Article  Google Scholar 

  20. Tubbs-Cooley HL, Mara CA, Carle AC, Gurses AP. The NASA Task Load Index as a measure of overall workload among neonatal, paediatric and adult intensive care nurses. Intensive Crit Care Nurs. 2018;46:64–69.

    Article  PubMed  Google Scholar 

  21. Kilpatrick SJ, Papile LMD, Faap, Macones GA Guidelines for Perinatal Care. American Academy of Pediatrics: Elk Grove Village, IL, 2017.

  22. Kapu AN, McComiskey CA, Buckler L, Derkazarian J, Goda T, Lofgren MA, et al. Advanced practice providers’ perceptions of patient workload: results of a multi-institutional survey. J Nurs Adm. 2016;46:521–9.

    Article  PubMed  Google Scholar 

  23. Shanafelt TD, Dyrbye LN, Sinsky C, Hasan O, Satele D, Sloan J, et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc. 2016;91:836–48.

    Article  PubMed  Google Scholar 

  24. Frintner MP, Kaelber DC, Kirkendall ES, Lourie EM, Somberg CA, Lehmann CU. The effect of electronic health record burden on pediatricians’ work-life balance and career satisfaction. Appl Clin Inf. 2021;12:697–707.

    Article  Google Scholar 

  25. Apathy NC, Rotenstein L, Bates DW, Holmgren AJ. Documentation dynamics: Note composition, burden, and physician efficiency. Health Serv Res. 2022;1–12. https://doi.org/10.1111/1475-6773.14097.

  26. Sawatzky-Dickson D, Bodnaryk K. Validation of a tool to measure neonatal nursing workload. J Nurs Manag. 2009;17:84–91.

    Article  PubMed  Google Scholar 

  27. Lemieux-Bourque C, Piedboeuf B, Gignac S, Taylor-Ducharme S, Julien AS, Beltempo M. Comparison of three nursing workload assessment tools in the neonatal intensive care unit and their association with outcomes of very preterm infants. Am J Perinatol. 2022;39:640–5.

    Article  PubMed  Google Scholar 

  28. Daraiseh NM, Vidonish WP, Kiessling P, Lin L. Developing a patient classification system for a neonatal ICU. J Nurs Adm. 2016;46:636–41.

    Article  PubMed  Google Scholar 

  29. Fleiss N, Coggins SA, Lewis AN, Zeigler A, Cooksey KE, Walker LA, et al. Evaluation of the neonatal sequential organ failure assessment and mortality risk in preterm infants with late-onset infection. JAMA Netw Open. 2021;4:e2036518.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Snapp B, Moore TA, Wallman C, Staebler S. 2020 Neonatal nurse practitioner workforce survey: An executive summary. Adv Neonatal Care. 2021;21:242–6.

    Article  PubMed  Google Scholar 

  31. Hart SG. NASA Task Load Index (NASA-TLX); 20 years later. Proc Hum Factors Ergon Soc Annu Meet. 2006;50:904–8.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Vanderbilt University Medical Center Office of Evidence-Based Practice and Nursing Research for their support of the project and the Departments of Pediatrics and Bioinformatics for Dr. Alrifai’s grant funding to support the project. This work was supported by the Evelyn Selby Stead Fund for Innovation, Vanderbilt University Medical Center, and the Department of Pediatrics Turner-Hazinski Research Award, Vanderbilt University Medical Center (MWA). Use of the Research Electronic Data Capture program (REDCap) was supported by UL1 TR000445 from NCATS/NIH.

Funding

Dr. Hatch serves on the Scientific Advisory Board of Novonate Inc. and receives stock options as part of that appointment.

Author information

Authors and Affiliations

  1. Monroe Carell Jr. Children’s Hospital at Vanderbilt, Nashville, TN, USA

    M. Eva Dye, Patti Runyan, L. Dupree Hatch & Mhd Wael Alrifai

  2. Department of Pediatrics, Division of Neonatology, Vanderbilt University Medical Center, Nashville, TN, USA

    M. Eva Dye, Theresa A. Scott, L. Dupree Hatch & Mhd Wael Alrifai

  3. Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA

    Mary S. Dietrich

  4. School of Nursing, Vanderbilt University, Nashville, TN, USA

    Mary S. Dietrich

  5. Center for Child Health Policy, Vanderbilt University Medical Center, Nashville, TN, USA

    L. Dupree Hatch

  6. Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, TN, USA

    Daniel France

  7. Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, TN, USA

    Daniel France

  8. Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA

    Mhd Wael Alrifai

Authors
  1. M. Eva Dye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patti Runyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Theresa A. Scott
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mary S. Dietrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. L. Dupree Hatch
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Daniel France
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mhd Wael Alrifai
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MED was responsible for creating the workload model (WORKLINE), writing the research protocol, recruiting study participants, interpreting results, and writing the manuscript. PR, LDH, and DF provided guidance on protocol development, analysis of results, and manuscript edits. TAS was responsible for development of the electronic surveys and set-up of the text messaging application used during the study and blinded results of the participants to insure confidentiality during the study. MSD conducted the statistical analysis for the study and assisted with manuscript edits. MWA was responsible for integrating the WORKLINE model into the EHR, revising the research protocol, recruiting study participants, interpreting results, obtaining grant funding, and revising the manuscript.

Corresponding author

Correspondence to M. Eva Dye.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dye, M.E., Runyan, P., Scott, T.A. et al. Workload In Neonatology (WORKLINE): Validation and feasibility of a system for measuring clinician workload integrated into the electronic health record. J Perinatol 43, 936–942 (2023). https://doi.org/10.1038/s41372-023-01678-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41372-023-01678-5

Search

Advanced search

Quick links