Artificial neural network-boosted Cardiac Arrest Survival Post-Resuscitation In-hospital (CASPRI) score accurately predicts outcome in cardiac arrest patients treated with targeted temperature management

Existing prognostic models to predict the neurological recovery in patients with cardiac arrest receiving targeted temperature management (TTM) either exhibit moderate accuracy or are too complicated for clinical application. This necessitates the development of a simple and generalizable prediction model to inform clinical decision-making for patients receiving TTM. The present study explores the predictive validity of the Cardiac Arrest Survival Post-resuscitation In-hospital (CASPRI) score in cardiac arrest patients receiving TTM, regardless of cardiac event location, and uses artificial neural network (ANN) algorithms to boost the prediction performance. This retrospective observational study evaluated the prognostic relevance of the CASPRI score and applied ANN to develop outcome prediction models in a cohort of 570 patients with cardiac arrest and treated with TTM between 2014 and 2019 in a nationwide multicenter registry in Taiwan. In univariate logistic regression analysis, the CASPRI score was significantly associated with neurological outcome, with the area under the receiver operating characteristics curve (AUC) of 0.811. The generated ANN model, based on 10 items of the CASPRI score, achieved a training AUC of 0.976 and validation AUC of 0.921, with the accuracy, precision, sensitivity, and specificity of 89.2%, 91.6%, 87.6%, and 91.2%, respectively, for the validation set. CASPRI score has prognostic relevance in patients who received TTM after cardiac arrest. The generated ANN-boosted, CASPRI-based model exhibited good performance for predicting TTM neurological outcome, thus, we propose its clinical application to improve outcome prediction, facilitate decision-making, and formulate individualized therapeutic plans for patients receiving TTM.

www.nature.com/scientificreports/ Cardiac arrest is implicated in a notable proportion of premature deaths and disabilities worldwide [1][2][3] . A substantial proportion of patients treated for out-of-hospital cardiac arrest (OHCA) die before emergency department arrival, and a significant proportion of in-hospital cardiac arrest (IHCA) patients-associated deaths occur during the initial resuscitation. Of those who have restoration of spontaneous circulation (ROSC) after initial resuscitation, a large proportion die before discharge, and only a minority of patients with cardiac arrest achieve favorable functional outcome at discharge 1,4 . However, it is noteworthy that several factors, including patients' age, pre-arrest neurological condition, functional status, rhythm type (shockable vs non-shockable) play vital role in the outcome of OHCA 5,6 . In context of these outcome predictors, we note that of the minority of patients who are discharged alive with favorable functional status, the proportion sdischarged with favorable neurological status is relatively high 6 , and this may explain the attribution of majority of OHCA post-resuscitation deaths to brain injury 7 , howbeit without accounting for deaths due to early withdrawal of life-sustaining treatment which in itself frequently truncates the opportunity for brain recovery 8 .
To prevent or reduce the poor outcomes associated with cardiac arrest, targeted temperature management (TTM) has been introduced and touted to reduce mortality and improve the neurological recovery in patients with cardiac arrest [9][10][11][12] .
Several prognostic factors associated with the outcome in cardiac arrest patients treated with TTM have been identified, and a number of tools have been proposed for outcome prognostication [13][14][15][16][17] . So far, these prognostic and/or predictive models either exhibit moderately acceptable accuracy, are designed exclusively for OHCA patients, or are too complicated with multiple clinical variables for the clinical applications [13][14][15][16] . This necessitates the development of a simple and yet generalizable prediction model to inform clinical decision-making and formulation of therapeutic strategies for patients receiving or indicated for TTM with different clinical status.
The Cardiac Arrest Survival Post-resuscitation In-hospital (CASPRI) score consisting of eleven items, was designed to predict clinical outcome of patients who achieve ROSC after experiencing IHCA 18 , and has been validated in different cohorts with good discrimination power reported 18,19 . However, its predictive performance in a non-selective patient cohort who received TTM regardless of place of cardiac arrest (IHCA and OHCA) has not been validated. In the present study, we hypothesized that the CASPRI score is clinically applicable in predicting the outcome in cardiac arrest patients treated with TTM, regardless of place of event.
Advances in machine learning algorithms coupled with increased computational power continue to enable enhanced diagnostic and prognostic capabilities in various medical fields. Recently published reports suggest the capability of artificial neural networks (ANN), a supervised machine learning algorithm, to accurately predict neurological outcomes, including survival, for patients with OHCA or IHCA 15,33,34 .
The present study explored the clinical validity of the CASPRI score in patients with cardiac arrest, regardless of place of event, who received TTM, and proffered improvement of the predictive accuracy of the CASPRI score by applying ANN-based prediction models.

Materials and methods
Participants. This retrospective cohort study used clinical data from medical records obtained from the Taiwan Network of Targeted Temperature Management for Cardiac Arrest (TIMECARD) registry 14 . TIMECARD registry is a nationwide multicenter registry project conducted from January 2014 and September 2019 in 9 medical centers in Taiwan. An on-line case report form was built for every participating hospital to report their patient-level data. All electronic medical data was decoupled from patient identifying information.
The inclusion criteria for the TIMECARD registry were: (1) participants aged 18 years or older, (2) a cardiac event occurring inside or outside the hospital, (3) receipt of cardiopulmonary resuscitation (CPR) with ROSC, (4) Glasgow coma scale (GCS) less than 8 or inability to obey commands after ROSC, and (5) receipt of TTM less than 12 h after ROSC.
All eligible patients were treated using the TTM protocol consistent with the consensus of scientific statement from the Taiwan Society of Emergency & Critical Care Medicine 12 . The variables were retrieved from archived patients' registry data based on the updated Utstein Resuscitation Registry template, and included baseline characteristics, comorbidities, coupled with information on the cardiac arrest event, etiology, post-arrest care, and the outcomes 12,14,20 . Modeled after the CASPRI score development and validation studies, which to the best of our knowledge, employed retrospective determination of pre-and post-arrest CPC score for predicting neurological outcomes for patients with cardiac arrest, the CPC score in our study was retrospectively determined from patients' information garnered from family members and/or medical records by the research investigator, who is also consultant neurologist in each medical center 14,[19][20][21][22] .
A favorable neurological outcome was defined as CPC score of 1-2 (conscious and alert with good or moderate cerebral performance) at the time of discharge, while poor outcome was defined as CPC score of 3-5 (severe neurological disability, persistent vegetative state, or death) 12,14,20-22 . Ethical approval. The study was approved by the Joint Institutional Review Board of Taipei Medical University (TMU-JIRB Approval No. N201711046). Waiver of informed consent were approved by the TMU-JIRB for this retrospective study involving the secondary analysis of existing anonymized data. All methods were performed in accordance with the relevant guidelines and regulations. NC, USA). Variables were summarized using descriptive statistics. Continuous variables are presented as mean ± standard deviation, and categorical variables are expressed as counts and percentages. One-way ANOVA was used to determine the statistical significance of differences between means of ≥ 3 independent variables and Fisher's exact test to determine non-random associations between 2 categorical variables. A two-tailed p-value of < 0.05 was considered statistically significant.
Application of CASPRI score. The CASPRI score (Table S1) was calculated for each patient as earlier described by Chan et al. in the original development and internal validation study 18 . In TIMECARD registry, all clinical information of interventions prior to the time of cardiac arrest were excluded 14 , thus, mechanical ventilation, indicated as one variable of the CASPRI score 18 , was not included in the analysis. To calculate the CASPRI score for OHCA patients, arrest location was scored 3 points, being the score for patients from non-monitored unit 18 . Overall, 10 out of the 11 items of the CASPRI score were incorporated into our logistic regression and ANN models (Table 1, Fig. 1). Univariate logistic regression analysis, with the CASPRI score considered a continuous variable, was performed to determine probable association between the total score and the outcome. We assessed our regression model using area under the receiver operating characteristics curve (AUC), with the accuracy, precision, sensitivity, and specificity of the univariate logistic regression model indicated.
Development and validation of ANN models. The ANN model was developed using STATISTICA ver.
13.3 (TIBCO Software Inc., Tulsa, Oklahoma, USA). The applied ANN architecture was a multilayer perceptron, Table 1. Baseline demographic characteristics of patients according to neurological outcomes at hospital discharge. CASPRI Cardiac Arrest Survival Postresuscitation In-hospital, CI confidence interval, COPD chronic obstructive pulmonary disease, CPC cerebral performance category, IHCA in-hospital cardiac arrest, MAP mean arterial pressure, OHCA out-of-hospital cardiac arrest, OR odds ratio, ROSC return of spontaneous circulation, VF ventricular fibrillation, VT ventricular tachycardia. a Variables used to calculate CASPRI score. b Odds ratio of per unit changes.

Variables
Whole cohort (n = 570) www.nature.com/scientificreports/ containing an input layer, one hidden layer, and an output layer (Fig. 1). Continuous variables included age, prearrest CPC score, duration of resuscitation, and mean arterial pressure (MAP) at ROSC. Categorical variables included initial arrest rhythm, arrest location, renal or hepatic insufficiency, sepsis, and malignancy. The arrest locations were indicated as four independent input neurons in the ANN model, namely, OHCA, telemetry unit, intensive care unit, and non-monitored unit. The numbers of neurons in the hidden layer were set empirically, ranging from 1 to 50.
Oversampling of the minority classes. To reduce the disproportionate ratio of patients with favorable neurological outcomes to those with unfavorable outcomes in the dataset, we applied the Synthetic Minority Over-sampling Technique (SMOTE) to the minority class, namely the subset of favorable neurological outcomes 23 . By analyzing samples in minority class and synthesizing new samples based on them, SMOTE can improve classification performance and help circumvent limitations associated with overly skewed or imbalanced data, thus enhancing the accuracy and generalizability of the prediction model 23 . By the SMOTE, 336 samples of favorable neurological outcomes were synthetically oversampled to re-balance the class distribution (Table S2). After oversampling, 453 samples each of favorable and unfavorable neurological outcomes were randomly partitioned into 80% training and 20% validation sets in the ANN models while maintaining an identical proportion of favorable and unfavorable outcomes.
Model evaluation. The generalizability of the analysis was assessed using five-fold cross-validation. The model performance was evaluated using five independent validation sets. The mean AUC of the five training and validation sets and the mean accuracy, precision, sensitivity, and specificity of the five validation sets are reported.

Results
Cohort demographics and baseline characteristics. A total of 580 patients were registered in the TIMECARD database. Ten patients without documented CPC score at discharge were excluded from the analysis.   The association between CASPRI score and the outcomes. The mean CASPRI score was 17.8 ± 5.6 points for the whole cohort, and the score was significantly higher in patients with unfavorable neurological status at hospital discharge (18.9 ± 5.5 vs 13.2 ± 3.8) ( Table 1). Unadjusted binary regression analysis showed that every point increase in the CASPRI score was associated with 1.28-fold (95% CI 1.21-1.36; p < 0.0001) or 1.14fold (95% CI 1.10-1.18; p < 0.0001) increase in the likelihood of an unfavorable outcome or mortality outcome, respectively, for cardiac arrest patients who received TTM. For the patients with CASPRI score < 10, there was a 56.7% probability of favorable neurological outcome and 80.0% probability of survival at discharge, while patients with CASPRI score ≥ 25 had no chance of a favorable neurological outcome ( Fig. 2A). As shown in Fig. 2B, the AUC of CASPRI score to predict favorable outcome was 0.811; 95% CI 0.779-0.843, with the accuracy, precision, sensitivity, and specificity of 79.8%, 52.5%, 17.9%, and 95.8%, respectively. The results indicated that the original CASPRI score exhibits good specificity but relatively low sensitivity in the prediction of neurological outcomes for the patients with cardiac arrest who received TTM.
Boosting the predictive performance of CASPRI score by using ANN. As alluded earlier, ten baseline characteristics from the items of CASPRI score were used as the input attributes to develop the ANN model for predicting neurological outcomes. After adequate training, the ANN-boosted CASPRI models containing 8, 27, 45, 46, and 47 hidden neurons achieved the best prediction performance for the fivefold cross-validation sets, with a mean training accuracy of 93.5 ± 3.8% and validation accuracy of 89.2 ± 2.5%. The precision of the validation set was 91.6 ± 1.3%, sensitivity was 87.6 ± 4.2%, and specificity was 91.2 ± 1.1%. The AUC was 0.976 ± 0.024 for the training set (Fig. 3A) and 0.921 ± 0.033 for the validation set (Fig. 3B).
A comparative analysis of the predictive performance of the original CASPRI score and ANN-boosted models were performed. As shown in Table 2, the results indicate that the ANN models achieved relatively higher accuracy, precision, sensitivity, and AUC values in predicting favorable neurological outcomes, with improved accuracy when predicting the clinical outcomes of patients with cardiac arrest who received TTM.
Relative significance of predictors. A sensitivity analysis was performed to assess the predictive value of each parameter in the ANN model, and evaluated the relative contribution of that parameter alone and in combination with other factors in the model. The relative significance of each factor was indicated by its mean importance value through five repetitions during five-fold cross-validation. Among all parameters of the CAS-PRI score, sepsis, malignancy, hepatic insufficiency, initial cardiac arrest rhythm, and arrest location were the strongest predictors of neurological outcomes (Fig. 4). These results provide some insight into the significance of the parameters that contribute to neurological prognosis of cardiac arrest patients who were treated with TTM.

Discussion
The present study validates the CASPRI score and the application of ANN-based models to predict or boost prediction of clinical outcomes in patients who received TTM. The CASPRI score exhibited prognostic relevance with an AUC of 0.811 to predict favorable neurological outcomes in patients who received TTM after cardiac arrest. Interestingly, the established ANN-boosted CASPRI score model achieved better predictive performance with an AUC of 0.921 for predicting the neurological outcomes in the validation set. This is of relevance for precision medicine, because the AUC measures the degree of discriminability between groups, thus, the relatively   www.nature.com/scientificreports/ higher AUC values indicate that the ANN-boosted models bode well for patient stratification, and can distinguish the groups of interest, namely favorable versus unfavorable neurological outcomes in cardiac arrest patients who received TTM. The CASPRI score was initially developed using successfully resuscitated IHCA patients-a population in which prognostication is particularly helpful in making decisions regarding the intensity of life support and associated management strategy 18 . Chan et al. in their study, reported CASPRI score with an AUC of 0.802 for predicting favorable neurological outcomes 18 . This score has been validated using patients of East Asian descent, wherein the AUC for CASPRI score was 0.77-0.79 19,24 and was recommended as a good tool for categorizing patients with varying chances of hospital survival 25 . Consistent with the findings of these studies, in the present study, we report CASPRI score with an AUC of 0.811 for predicting favorable neurological outcomes in cardiac arrest patients who are treated with TTM.
Though the original CASPRI score was developed exclusively for IHCA patients, individual components of the CASPRI score have been associated with the outcomes of TTM, including age [14][15][16]26 , initial arrest rhythm 14,16,26 , pre-arrest CPC score 14 , arrest location 15,16 , duration of resuscitation 26 , and comorbidities. 14,15 , however, to the best of our knowledge, this is the first study that evaluated the predictive validity of the CASPRI score for patients receiving TTM with concomitant application of ANN algorithms to boost its predictive performance. Our results demonstrate that CASPRI score is also of clinical relevance for patients who received TTM after cardiac arrest. We are cognizant of several other documented models for predicting neurological outcomes for patients receiving or who have received TTM, however, most of them exhibit inferior discrimination power or predictive potential. One such model, the Acute Physiology and Chronic Health Evaluation (APACHE) II score for predicting favorable neurological outcome for patients with OHCA who received TTM exhibited an acceptable discrimination power with an AUC of 0.697 27 . Another model called the Mild Therapeutic Hypothermia score for predicting in-hospital mortality among OHCA patients treated with TTM, reported an AUC of 0.74 28 , while the risk score proposed by Martinell et al. for patients with OHCA receiving TTM yielded AUCs of 0.818-0.842 16 . These scoring systems only demonstrated moderate accuracy, thus, limiting their clinical applications for precise outcome prediction or patient stratification.
More so, understanding the multifactorial nature and complex interplay between baseline conditions, characteristics at the time of cardiac arrest, and the outcomes after TTM, coupled with the challenges associated with obtaining accurate predictions using conventional scoring systems, we exploited the benefits of ANN, a supervised learning algorithm which through emulation of the biological neural architecture, aids identification of relevant predictive markers in the diagnostic task, determines nonlinear data relationships, enhances data interpretation, and informs the design of more efficient diagnostic and predictive models [29][30][31][32] . Against this background, our generated ANN-boosted predictive model exhibited high AUC with good accuracy, precision, sensitivity, and specificity, highlighting the applicability of machine learning algorithms to improve the performance and accuracy of CASPRI-based predictive models. Our finding is particularly interesting and clinically relevant because literature review reveals that only few studies have explored the use of machine-learning algorithms to predict the prognoses of patients treated with TTM. Correspondingly, AUCs of 0.82-0.95 were reported by Andersson et al. who included several clinical variables, clinically accessible, and research-grade biomarkers, as predictors of clinical outcomes for patients with OHCA 33 . Johnsson et al. using a cohort of 932 OHCA patients from 36 medical centers, who were treated with TTM, reported an AUC of 0.891 based on 54 clinical variables, and an AUC of 0.852 when three variables, namely, age, time to ROSC, and first monitored rhythm, were used 15 . More so, a previous study by our team using five clinical predictors in the ANN model demonstrated a good predictive performance and notable discrimination power with an AUC of 0.906 for IHCA patients who received TTM 34 . Our current study demonstrates the ANN-boosted technique can accurately predict the neurological outcomes for cardiac arrest patients who received TTM with an AUC of 0.921.
Compared to earlier mentioned studies that focused exclusively on IHCA or OHCA patients, or used complex clinical and serum biomarkers 15,33,34 , our current findings, taking advantage of the simplicity of the widely known CASPRI score, and using readily accessible patient information, highlight the all-inclusive capability of our ANN-boosted model to stratify patients into prognostic groups (favorable outcome vs unfavorable outcome), regardless of cardiac arrest location (IHCA and OHCA). The high AUC value of current study connotes enhanced capability and feasibility of the ANN-boosted CASPRI model with generic predictors to predict the outcome in cardiac arrest patients treated with TTM. The accuracy of clinical predictions can be critical in assisting clinical decision-making for rapid implementation of post-resuscitation therapies. Based on the sensitivity analysis of our ANN model, we also ranked the predictive variables according to their prognostic relevance in patients with cardiac attack who were treated with TTM. Thus, we proffer an ANN-based predictive model with improved predictive performance, that is relatively superior to other conventional statistical approaches or preexisting predictive scoring systems. This ANN-based model is clinically feasible and might further provide the information on the selection of patients who would potentially benefit from TTM treatments.
As with studies of this nature, the present study has some limitations. First, this is a retrospective observational study comprising a relatively small sample of patients who received TTM after successful resuscitation from a cardiac arrest. The limited sample size may restrict the generalizability of current model to a broader population with variable characteristics and prevent complete exclusion of the possibility of model overfitting. Therefore, a large multicenter multi-ethnic cohort with a wide range of clinical and molecular characteristics is required to represent the disease population and validate our results. Second, there was a lack of randomization into TTM or non-TTM groups in current study. The restriction of enrollment to those who received TTM limits the application and generalizability of the current model. Third, the dataset used in the current study did not include the information on interventions in place at the time of cardiac arrest, such as mechanical ventilation, thus, the generated models consisting of 10 predictor variables does not completely represent the CASPRI score that comprised 11 variables. Fourth, previous studies have demonstrated that the time to cooling initiation, time to