Introduction

Prolonged hospital length of stay (LOS) is an important quality of care metric that may indicate severe illness,1 suboptimal care coordination,2 administration of certain medications,3,4,5 or complex and evolving treatment plans. The most important concern during hospitalization of a patient is proper and high quality of care leading towards recovery from illness. However, prolonged hospitalizations disrupt the social routines of the patient and their families as well as the economic status of both families and hospitals. On the one hand, parents may lose productivity at work while hospital bills increase. On the other hand, hospitals and healthcare systems face ever-increasing pressures to provide optimal care with payments and resource allocations that are challenged to meet demand. These concerns cut across all medical specialties,6,7,8,9,10 including trauma and emergency general surgery wherein there is need to study and reduce unnecessarily long hospital LOS.11 Trauma centers and health systems under operational stress face the pervasive challenge of determining patient discharge prioritization while maintaining high quality of care and optimal resource utilization.12 The ability to predict prolonged LOS of pediatric trauma patients can be valuable to healthcare providers in managing resources more efficiently.13 This, however, brings into question whether the LOS of trauma patients can be reduced. To this end, previous studies and experience in practice have indicated that there are opportunities to improve the quality of care of patients that will result in reduced LOS.14,15,16 This may include the implementation of models such as the Lewin’s Change Model,14 improvement of communication between physicians and physical therapists,16 and anticipation of inefficiencies in the management of complex patients.

Interventions for improved quality of care that will result in reduced LOS may be carried out indiscriminately on all trauma patients only in the presence of unlimited clinical intervention resources and time. This utopian state of hospital operation is not achievable, thereby necessitating the need for targeted interventions. This need for targeted interventions can be partly met by the development of statistical or machine learning models to determine patients most at risk of prolonged LOS. Real-time demand capacity management17 may be used to predict capacity, demand, and develop a plan of action for patients’ next steps; however, the plan is decided through clinician meetings to select which patients and how many patients should be discharged that same day.18 Previous studies have indicated that the use of tools such as the National Surgical Quality Improvement Program (NSQIP) Risk Calculator or clinicians to predict the LOS of trauma patients result in dissatisfactory performance/accuracy. But early and accurate mobilization across all care setting of pediatric trauma may result in improved quality of care19 (including reduction in infections and medical complications) and reduce financial cost of care for both patients and hospitals.19 Therefore, an implementable model predicting pediatric trauma patients’ hospital LOS can be an effective tool for providers to mobilize early intervention efforts. These early interventions will lead to more efficient utilization of facilities and staff members within hospitals. A model with the capability of predicting a patient’s prospective hospital LOS can consequently lead to the optimization of treatment plans for patients, reduced cost, improved resource allocation, and diminished hospital-acquired infections.20 In addition, the accurate prediction of hospital LOS can result in families’ emotional satisfaction.21

In this study, two needs in pediatric trauma literature on hospital LOS were addressed. First, a nested mixed effects model was built to determine associations between prolonged LOS and clinical presentation up to the first 24 h of admission of the patient. The type of trauma, previous healthcare utilization variables, and number of comorbid diagnoses among other variables were studied. Second, a machine learning model that has greater flexibility in capturing nonlinear relationships as well as complex interactions between variables was built for prediction of prolonged LOS. In this study, prolonged LOS was defined as >1 week,22,23,24 a priori.

Methods

Setting and data sources

This study was approved by the Institutional Review Board of CHOC Children’s (#180857). The data source for the study is the Cerner Health Facts Database. The database consists of data captured by Cerner Corporation from over 100 US healthcare systems and over 650 facilities (as at 2018). The data were aggregated and organized into consumable datasets to facilitate research and reporting. In addition, the data were deidentified and secured with encryption to maintain patient confidentiality and ensure compliance with HIPAA privacy regulations. It is in the form of a structured SQL database with tables on patient demographics, encounters, medications, laboratory tests, clinical events, and diagnoses among others. At the time of this study, the Health Facts DB consisted of 6.9 million patients and 503.8 million encounters across all care settings. The database is available to researchers at healthcare systems that contribute data to it.

Encounters of all pediatric trauma hospital admissions for which the primary diagnosis or reason for visit was trauma were initially retrieved. This definition was captured using the International Classification of Diseases, Tenth Revision (ICD-10-CM) codes of S00-T79. Only patients <18 years old and with hospital LOS >24 h were included in the study. Hospital inclusion criteria consisted of hospitals that have 500 encounters with primary diagnosis for trauma. This threshold of 500 was set, a priori, to guard against noise in the database and to ensure that each hospital-level parameter can be estimated with enough power in a statistical or machine learning model. Another threshold was set for inclusion of categorical variables with rare outcomes. Each categorical variable had to meet the condition that the rarest level of the variable has at least 100 response to guard against statistical separation.25 Small sample analyses, such as exact statistical tests,26 are required for very rare outcomes which is outside the scope and objective of this study. Lastly, a test for multicollinearity27,28 was carried out to ensure that highly correlated variables were removed prior to model development. Statistical (or class) separability and multicollinearity may result in inflated type I or II errors, unstable models, and inestimable parameters.

Variables of interest

The outcome variable, prolonged LOS, was defined as any encounter for which the hospital LOS >1 week.22,23,24 We defined injuries and trauma to all body regions as captured by the International Classification of Diseases, Tenth Revision (ICD-10-CM). Independent variables of interest consisted of trauma diagnoses, patient demographics, patient history of healthcare utilizations, count of comorbid conditions, number of medications within the first 24 h of admission, indicator for whether the hospital is a free-standing pediatric hospital, and type of surgeries carried out during the encounter. A free-standing pediatric hospital was defined as one where the average age of all patients was <18 years. A list of variables and summary statistics is shown in Table 1.

Table 1 Summary statistics.
Full size table

Statistical and machine learning considerations

Statistical inference was provided using a nested mixed effects model29,30 with the hospitals as random intercepts and patients nested within hospitals. We developed a full mixed effect consisting of all the variables in Table 2. Full models are statistical models that include all variable of interest without variable selection procedures such as stepwise regression. Extreme gradient boosting was chosen as the variant of stochastic gradient boosting due to its highly competitive model performance in literature, in competitions, and in practice.31,32,33 Model selection was achieved through cross-validated hyperparameter tuning over the values of hyperparameters shown in Table 2.

Table 2 Hyperparameters for extreme gradient boosting.
Full size table

The predictive performance of the extreme gradient boosting model was evaluated based on measures including the area under the receiver operator characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). The values of sensitivity, positive predictive value (PPV), negative predictive value (NPV), relative risk score, and F1 score were calculated at a specificity of 90%. These performance values are based on the measures of the confusion matrix for binary prediction: true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN).

Sensitivity = TP / (TP + FN)

Specificity = TN / (TN + FP)

PPV = TP / (TP + FP)

NPV = TN / (TN + FN)

Analyses for this study were carried out using high-performance parallel distributed cloud computing resources on Amazon Web Services and the R statistical computing programming language.31,34,35,36

Results

Baseline characteristics

A total of 27 hospitals met the inclusion criteria for hospitals of which 7 (25.9%) were free-standing pediatric hospitals, 71,528 were patients and 81,929 were encounters that met the inclusion criteria. The seven free-standing pediatric hospitals contributed 54.7% of the patients in the study. The average age of the patients was 8 years (standard deviation of 5 years). There were 42.2% female patients, 58.0% Caucasians, 20.2% Blacks or African American, 2.6% Hispanic, and 19.2% patients of other or unknown race/ethnicity. In addition, 30.8%, 43.7%, and 25.5% had commercial, governmental, and other healthcare insurance types, respectively. The rate of prolonged LOS was 18.7%. The variables that met the inclusion criteria for the study are shown in Table 2. Results from the nested mixed effects model (referred to as the statistical model from here on) indicate significant predictors from each category of variables considered.

Statistical model—trauma/injuries

Trauma-specific risk factors of prolonged LOS were obtained from the statistical model (see Table 3). The patients most likely to have prolonged LOS include patients with burns and corrosion (152% increase in odds), patients confirmed to be suffering from child abuse (106% increase), and patients suspected to be suffering from child abuse/neglect/maltreatment (78% increase). There were increases in odds for patients with injuries to the ankle and foot (49%); injuries to the thorax (47%); and injuries to the abdomen, lower back, lumber spine, pelvis, and external genitals (44%). Patients with certain early complications of trauma (such as air embolism, traumatic shock, anuria, and compartment syndrome) have a 24% increase in odds of prolonged LOS. Lastly, injuries resulting from foreign objects entering the natural orifice of the body is associated with 19% increase in odds.

Table 3 Nested mixed effects logistic regression model.
Full size table

Certain other injuries/trauma were associated with shorter LOS. These include injuries/trauma to the neck, shoulder and upper arm, elbow and forearm, hip and thigh, knee and lower leg. The decrease in odds associated with these conditions ranged from 17 to 48%. There were no significant differences associated with injuries/trauma to the wrist and hand, head, multiple body regions (ICD-10-CM: T07), and unspecified body region (ICD-10-CM: T14).

We however suspected that there may be significant statistical interactions between head injuries and the number of comorbid diagnoses. In other words, we expect a difference in risk for prolonged LOS between a patient with a mild injury to the head and a patient with traumatic brain injury. In the absence of a measure of injury severity, we used the number of comorbid diagnoses as a proxy of severity of illness/injury. In Fig. 1, we provided the interaction plot between head injuries and the number of comorbid conditions. The interaction plot indicates that there is significant association between head trauma, but the effect is modified by comorbid conditions. Simple head injuries not resulting in complications (such as neurological complications) do not result in prolonged LOS but as the number of comorbid condition increases, head injuries become risk factors for prolonged LOS.

Fig. 1: Statistical interaction between head trauma and comorbid conditions.
figure 1

0: No (Did not have head trauma). 1: Yes (Had head trauma).

Full size image

Statistical model—demographics, payer, resource utilization, and severity of illness

The results of the statistical model indicated that older patients tend to have shorter LOS (14% decrease in odds per 1-year difference). There was no sex difference in prolonged LOS, and lower income patients (on governmental healthcare insurance) have a higher tendency for prolonged LOS. There was a significant difference between Caucasian and Black/African American patients who have 11% increase in odds of having a prolonged LOS compared to their counterpart (Caucasians) but no other significant difference was found beyond it.

In terms of severity of illness and resource utilization, hypothermia (which may be the side effect of underlying injuries/trauma) was associated with 105% increase in odds of prolonged LOS. There is a 32 and 25% increase in odds for every non-trauma comorbid condition and every additional medication that is administered during the first 24 h of visit. Patients admitted through the ED and patients who had a readmission within the prior 6 months have 87 and 45% increase in odds of prolonged LOS. The maximum LOS of previous visits was also predictive of the likelihood of prolonged LOS. Every difference in 1 day of LOS is associated with 1.6% increase in odds.

Machine-learning prediction model

Cross-validated hyperparameter tuning resulted in the selection of 128 boosting operations, maximum tree depth of 8, learning rate of 0.2, gamma of 2, and minimum child (tree node) weight of 1 as hyperparameters that optimizes model performance. The range of values searched, and significance is shown in Table 2. Overall, these selected hyperparameters for the extreme gradient boosting model indicate that hospital LOS may be driven by a set of important complex mix of interacting factors. The performance of the resulting model as measured by the AUROC was 0.912 (0.907, 0.917). In Fig. 2, we show the relative variable importance and frequencies of use by the gradient boosting model. The top five most important variables according to the machine learning model are number of medications administered during the first 24 h, age of the patient, number of comorbid conditions/diagnoses, the previous maximum hospital LOS in the prior 6 months, and injuries due to burns and corrosions.

Fig. 2
figure 2

Relative variable importance and frequency of use in the extreme gradient boosting model.

Full size image

The sensitivities of the model and predicted probability thresholds for flagging patients as high-risk for prolonged LOS are shown in Table 4 for pre-specified specificities. The choice of model specificity or predicted probability threshold should be informed by avoidance of false alert fatigue, the size of clinical intervention teams, and an acceptable level of sensitivity. Recommendations from this study is to set specificity at 90%. This implies setting the predicted probability threshold at 0.252 and result in capturing 74% of patients with prolonged LOS during the first day of admission. Patients flagged at high-risk would be ten times as likely to have prolonged LOS as those not flagged by the model. The number needed to evaluate (NNE) of 1.6 indicates that of every 16 patients flagged at high-risk by the model, 10 will indeed have prolonged LOS. Finally, the area under the precision-recall curve for this model is 0.780, indicating that it is a very strong model and the AUROC is not inflated/misleading.

Table 4 Model performance at pre-specified levels of specificities.
Full size table

Discussion

Efficient patient flow and hospital resource utilization are integral to meeting the rising demand for pediatric trauma care. The development and deployment of a machine learning prediction model to be used by a clinical intervention team provides concrete information on how to maximize intervention efforts by focusing on patients most at risk of prolonged LOS. The machine learning model provided in this study provides a high-performance model for classifying patients into risk strata. The model can be integrated into the electronic medical records of the trauma center to provide a list of patients with their level of risk, or as an alert system. The choice and method of implementation will be driven largely by the individual and peculiar needs of providers at corresponding hospitals/trauma centers.

Several approaches may be used for risk stratification using the machine learning model. First, a two-strata system may be adopted as recommended in the “Results” section of this study. In this system patients that the model determined to have a predicted probability >0.252 (see Table 4) will be classified as high-risk otherwise low risk. Second, a three-strata system may also be developed to classify patients at “high”, “moderate”, and “low” risk of prolonged LOS. Our suggestion for this three-strata system is to set the predicted probability threshold at 0.7471 (corresponding to model specificity of 90%) for classifying patients as “high risk”. Patients with predicted probabilities between 0.7471 and 0.252 may then be classified as “moderate risk”, and patients with probabilities <0.252 as “low risk”. The overall model performance for both approaches will be the same except that we have different risk strata. Lastly, experimentation may be required to find the optimal number of risk strata and corresponding probability thresholds peculiar to the needs and resources of each hospital.

Predicting patients who are at high risk of prolonged LOS is of limited value if nothing can be done to either shorten corresponding LOS or to mitigate the effects of such a prolonged stay. Patients with anticipated increased LOS would benefit from earlier comprehensive nutritional, PT/OT, DVT, and pressure ulcer risk assessments. For appropriate aged children, psychological and academic support resources could be targeted toward those at greatest anticipated need. These patients may require early referral for post-discharge rehabilitation services. For parents, awareness of need for prolonged LOS may allow time to mitigate disruption to family and work routines. Finally, the ability to better predict LOS will allow hospitals to more efficiently and appropriately deploy limited resources.

The mixed effects model (see Table 3) provides opportunity for personalized interventions beyond the general issues of efficient care coordination. The model identified patients suffering from burns and corrosion, confirmed or suspected child abuse/neglect, and injuries to the ankle and foot, thorax, abdomen, lower back, lumber spine, pelvis, and external genitals as most at risk of prolonged LOS. While this study does not provide specific reasons why these trauma/injuries are associated with prolonged LOS, clinical intervention teams can use it as a tool for personalized interventions. These interventions may be developed through careful chart reviews for issues relating to potential quality of care issues, unnecessary complications, and smooth transition of care. In addition, the impact of potential social worker intervention due to confirmed or suspected child abuse can be factored into the care of the patient to reduce impact of child abuse/neglect investigations. The goal of including child abuse (or non-accidental trauma) is to ensure the model accounts for all pertinent risk factors or variables that may impact the hospital LOS of a patient. Furthermore, a more detailed understanding of the medical versus social factors impacting LOS may allow a more appropriate and timely allocation of medical and social work resources that effectively maximizes patient safety. Interventions may be extended to findings on severity of illness, number of comorbid conditions, prior healthcare resource utilization, and social determinants. The association between previous readmission and prolonged LOS of the current/index visit is novel and was included during the reverse association between current LOS and future risk of readmission.37,38,39,40 Unlike the machine learning model, this mixed effects model did not incorporate important interactions between predictors. Rather, it was used to provide simplified inference that does not capture the full interplay of factors associated with prolonged LOS. However, a two-way statistical interaction between head trauma and number of comorbid conditions indicate the existence of more complex relations between the variables. Therefore, we included a machine learning model that can capture all these complex interactions in a computationally feasible way. The machine learning model captured the importance of the trauma to the head ranking it among the top ten most important variables.

There are several limitations of this study. The most important of these is the use of diagnosis codes for determination of trauma and the absence of the mechanism of injury. The use of diagnosis codes is unavoidable in the case of retrospective studies using large multicenter electronic medical record databases. The most consistent documentation of patient condition across multiple hospitals are standard diagnostic codes. However, across a given diagnosis code a wide range of injury severity may occur. The relatively frequent closed head injury with small intracranial hemorrhage and short hospital stay may be coded similarly as the much less frequent massive head injury with extensive and permanent neurologic damage and prolonged LOS. A diagnosis-based model that fails to account for severity or need for intervention may fail to properly predict LOS in a subset of severely injured patients. The closest to a proxy for severity of illness or injury in this study are the number of medications administered during the first 24 h of admission, and the number of comorbid diagnoses.

The mechanism of injury may provide additional information in understanding and predicting prolonged LOS. Future studies should be conducted to see if the mechanism of injury modifies the risk of prolonged LOS beyond the type of trauma. Lastly, the mixed effects model identified associations with prolonged LOS which may not be causal factors.

Conclusion

The use of statistical and machine learning models to predict LOS in a large pediatric trauma database offers the potential to allow early targeted interventions, improved assessments, and optimal utilization of resources both for individual patients and across trauma systems. This study aimed to identify both discrete risk factors as well as a machine learning tool that could alert front-line caregivers upon admission which patients may have a longer LOS. This has implications for system demand, bed planning, staffing decisions, as well as individual patient concerns such as risk assessment, hospital-acquired condition risk, early mobilization goals, family caregiver arrangements, and post-discharge rehabilitation needs. Continued refinement of this model may inform prevention, assessment, and mitigation efforts for both pediatric and adult trauma patients.