Artificial neural network based prediction of postthrombolysis intracerebral hemorrhage and death

Despite the salient benefits of the intravenous tissue plasminogen activator (tPA), symptomatic intracerebral hemorrhage (sICH) remains a frequent complication and constitutes a major concern when treating acute ischemic stroke (AIS). This study explored the use of artificial neural network (ANN)-based models to predict sICH and 3-month mortality for patients with AIS receiving tPA. We developed ANN models based on evaluation of the predictive value of pre-treatment parameters associated with sICH and mortality in a cohort of 331 patients between 2009 and 2018. The ANN models were generated using eight clinical inputs and two outputs. The generalizability of the model was validated using fivefold cross-validation. The performance of each model was assessed according to the accuracy, precision, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). After adequate training, the ANN predictive model AUC for sICH was 0.941, with accuracy, sensitivity, and specificity of 91.0%, 85.7%, and 92.5%, respectively. The predictive model AUC for 3-month mortality was 0.976, with accuracy, sensitivity, and specificity of 95.2%, 94.4%, and 95.5%, respectively. The generated ANN-based models exhibited high predictive performance and reliability for predicting sICH and 3-month mortality after thrombolysis; thus, its clinical application to assist decision-making when administering tPA is envisaged.

Thrombolysis using intravenously administered recombinant tissue plasminogen activator (tPA) represents the standard of care and most effective treatment for acute ischemic stroke (AIS) 1,2 . However, the intended benefits of tPA is dampened by the risk of symptomatic intracerebral hemorrhage (sICH). Post-thrombolysis sICH herald poor clinical outcome and account for most early excess deaths 1 , with a high 3-month mortality rate 3 . This necessitates the development of therapy-decision support tools based on the thrombolysis risk-to-benefit ratio, especially as the accurate and early identification of patients with high risk of post-tPA sICH and death is increasingly shown to play a crucial role in informing clinicians' decision-making and therapeutic strategies.
In the last decade, several scoring tools have been introduced to predict sICH or death after tPA [4][5][6][7][8][9] , however, the related studies only demonstrate moderate prediction accuracy and this has not translated into reduced incidence of sICH or increased 3-month clinical outcome. This, in part, forms the basis of the present study which explored the use of artificial neural network (ANN)-based predictive models for identifying patients with high risk of post-thrombolysis sICH and death, as well as stratification based on likelihood of benefiting from tPA therapy.
Evolving medical adaptation of artificial intelligence (AI) proffers the means to investigate nonlinear data relationships, enhance data interpretation, and design more efficient diagnostic and predictive tool 10 . ANN, an AI method simulating the structure and functionalities of the human neural architecture, is able to predict existent complex relationships between input and output variables by repeated learning and validation process, and is increasingly applied in various aspects of medical diagnosis and prediction 11,12 . The present study generated Statistical analyses. All statistical analyses were performed using the JMP software, version 11.0.0 (SAS Institute Inc., Cary, NC, USA). Variables were summarized using descriptive statistics. Continuous variables with normal distribution are presented as mean ± standard deviation, and categorical variables are expressed as percentages with corresponding 95% confidence intervals (CIs). One-way ANOVA was used for continuous variables, and Fisher's exact test was used for categorical variables. A p-value of < 0.05 was considered statistically significant.
Application of ANN modeling. All ANN models were developed using STATISTICA 10.0 (StatSoft, Tulsa, Oklahoma, USA). The applied computational architecture was multilayer perceptron (MLP), a feed-forward ANN, combined with back-propagation algorithm for training the feed-forward ANNs. To train the ANN, supervised learning was performed by providing a series of input and output variables from the training dataset, such that by iteratively adjusting the connection weights, a desirable input-output mapping function was generated 12 . After appropriate training, optimization of the ANN architecture is initiated until the most satisfactory performance is achieved. The number of neurons in the hidden layer is set empirically. For our models, the input layer consisted of 8 neurons and the output layer consisted of 2 neurons (Figs. 1A,B).  www.nature.com/scientificreports/ Model development. The input attributes of the ANN models included clinical features extracted from the patients' medical history that were associated with the output attributes of interest, namely post-tPA sICH and 3-month mortality for model 1 and model 2, respectively. The generalizability of the analysis was assessed by fivefold cross-validation. Briefly, for cross-validation, the dataset was randomly shuffled and partitioned into five subsets (folds), followed by five rounds of training and validation of the ANN models. In each round of the analysis, four subsets served as the training subsets, and the remaining 1 subset was retained to validate the ANN model. Each of the five subsets was only used once as the validation set in the cross-validation process. The performance of the ANN models was evaluated using the five independent validation sets. The model performance was measured and visualized by the area under the receiver operating characteristic curve (AUC) of the training and validation sets, and the mean accuracy, precision, sensitivity, and specificity of the five validation sets were also reported.

Results
Cohort demographical and baseline clinicopathological characteristics. During the study period, 380 patients received tPA treatment for AIS. Among them, 331 patients (133 women and 198 men, mean age 69.2 ± 12.2 years) were eligible and enrolled into this study. Forty-nine patients who received endovascular interventions following intravenous tPA were excluded from baseline analyses. The average onset-to-treatment time was 122.1 ± 45.3 min. The mean baseline NIHSS score was 12.6 ± 6.3. The stroke subtypes of the cohort included large artery atherosclerosis (41.0%), cardioembolism (22.9%), small vessel occlusion (25.0%), and others (11.1%). There were 68 patients (20.5%) aged over 80 years. At baseline, older age was associated with lower GCS scores (p < 0.0001), higher initial NIHSS scores (p < 0.0001), lower body weight (p < 0.0001) and lower total tPA dose (p < 0.0001).

Clinical outcomes after thrombolytic therapy.
Within 72 h of administering tPA, 25 patients (7.6%) exhibited sICH (Table 1). Among these, 2 patients (8.0%) died during hospitalization. During the 3-month follow-up after tPA, 43 patients (~ 13.0%) from the total cohort were lost to follow-up, and the remaining 288 patients were enrolled into the ANN analysis for post-tPA 3-month mortality ( Table 2). Of these 288 patients, 31 deaths were recorded (17 in-hospital and 16 after discharge). Compared with the non-sICH group, patients who developed sICH after tPA had less favorable outcomes, as demonstrated by higher mRS scores ( Fig. 2A), and greater risk of 3-month mortality (Fig. 2B) during the first 3 months post-tPA.
Patients with sICH presented with an exacerbated clinical phenotype compared with the non-sICH group. The demographics and baseline clinicopathological characteristics of patients with or without post-tPA sICH are presented in Table 1. At baseline, patients with sICH exhibited higher diastolic BP, lower LDL level, and higher prevalence of atrial fibrillation or any other type of heart disease, lower prevalence of hyperlipidemia, when compared with patients who did not develop sICH. The sICH group also differed in their stroke subtypes to the non-sICH group. There were more patients with the cardio-embolic subtype and fewer small-vessel occlusions in the sICH group.
Attributes associated with mortality within 3 months of thrombolytic therapy. The demographics and mortality-related clinicopathological characteristics of our total cohort in the first 3 months after tPA are presented in Table 2. We observed that patients who were older, received lower dose of tPA, had lower GCS and higher NIHSS score at baseline, co-morbid with diabetes mellitus (DM) and ischemic heart disease, and higher fasting glucose level, were more likely to die within the first 3 months after tPA therapy.
Random oversampling. Understanding the detrimental effect of an imbalance or severely skewed dataset on the accuracy and generalizability of any prediction model, we sought to reduce the disproportionate ratio of our sICH to non-sICH and the 3-month mortality patients to survivors in the cohort, and performed random oversampling of the minority classes, namely the sICH and 3-month mortality subsets. Thus, one hundred samples were randomly selected from the sICH or 3-month mortality subset to increase the size of the training and validation sets and rebalance the class distribution for the ANN model 1 or model 2, respectively. This naïve method not only rebalances the class distribution, but also improve overall classification performance 15,16 . After random oversampling, 100 sICH and 306 non-sICH samples were randomly partitioned into 326 training and 80 validation sets in ANN model 1, and 100 mortality samples and 257 survivals were randomly partitioned into 285 training and 72 validation sets in ANN model 2, respectively (Table 3).
Predictive performance of our post-tPA sICH and 3-month mortality ANN models. ANN model 1. For model 1, the ANN was trained to predict post-tPA sICH. The input attributes, based on results of analyses in Tables 1, included the baseline diastolic BP, level of LDC, history of hyperlipidemia, Af, or any kind of heart disease (Table 3); however, while stroke subtype is associated with sICH, it was not included in the model because the accurate classification of stroke subtype would usually require a complete evaluation of stroke etiology and this was not obtainable before tPA treatment. The output attribute of this model was sICH. After adequate training, the ANN models that contained 8,11,15,16, and 20 neurons in the hidden layer achieved the best predictive performance for the five validation sets (validation 1-5), with a mean training accuracy of 89.2 ± 4.1% and validation accuracy of 91.0 ± 3.5%. The mean precision of the validation sets was 81.0 ± 11.1%, sensitivity was 85.7 ± 14.0%, and specificity was 92.5 ± 5.4%. The mean AUC was 0.951 ± 0.02 for the training sets and 0.941 ± 0.03 for the validation sets (Figs. 3A

Comparison of the predictive performance of ANN models with other prediction scores.
To better understand the predictive performance of ANN models and available outcome prediction scores, we calculated the Stroke Prognostication using Age and NIH Stroke Scale index (SPAN-100) 6 , Totaled Health Risks in Vascular Events (THRIVE) 7,17 , and Safe Implementation of Treatments in Stroke (SITS) 5 scores with the present clinical data, and used receiver operating characteristic (ROC) curve analysis to compare our ANN models with the scores (Table 4). ANN model 1 and model 2 showed remarkably higher AUC values in ROC analysis than www.nature.com/scientificreports/ the scoring systems in predicting sICH and 3-month mortality, indicating the greater discrimination ability of the ANN models for the measured outcome.

Discussion
The present study generated ANN-based predictive models to predict sICH within 72 h of intravenous tPA administration (model 1) and the post-tPA 3-month mortality (model 2) of patients with AIS. ANN model 1 and ANN model 2 achieved high validation performance, with AUC 0.941 and 0.976, respectively. This is predictive relevance, as the AUC measures and portrays the degree of separability; thus, the relatively high AUC values indicate that the models are capable of distinguishing the classes of interest, namely, sICH versus non-sICH and mortality versus survival of tPA administration. Our result demonstrated that high baseline diastolic BP, lower level of LDL, history of hyperlipidemia, Af, and heart disease were associated with sICH, while aging, lower dose of tPA, lower baseline GCS score, higher NIHSS score, history of DM and ischemic heart disease, were predictors of the post-tPA 3-month mortality. Thus, the rationale for the application of these variables as input attributes, and in part informs the demonstrated reliability and accuracy of our ANN-based predictive models. Stroke is a multi-factorial neurological disorder with broad systemic implications. Individual factors/variables have been shown to be modestly associated with therapeutic outcome, and accurate prediction of clinical www.nature.com/scientificreports/   18 . Thus, the complexity of AIS limits the conventional prediction models and scoring systems, as well as curtails their predictive reliability for the individualization of treatment. Against this background, in an effort to correctly predict which patients are at the greatest risk of post-thrombolysis sICH and death, the present study employed a composite and integrated prediction model consisting of multiple demographic and clinical factors, with the evidence-based predictive or prognostic capability in AIS. The complex synergy between these variables, we believe, is crucial for clinical application. Consistent with contemporary knowledge, we identified several factors related to post-tPA sICH and 3-month mortality, and most of these factors are of demonstrable prognostic relevance in AIS. Underscoring the rationality of our selected input attributes, for example, high NIHSS score, and Af have been suggested to increase the  Table 4. Comparison of the predictive performance of different models. The AUC value of ANN models showed the mean AUC of the five validation sets. The AUC value of SPAN-100 index was calculated using a univariable regression model with SPAN-100 score (age plus NIHSS). Compared to the SPAN, THRIVE, and SITS scores, the predictive performance was remarkably higher in the ANN models. ANN, artifice al neural network. AUC, the area under the receiver operating characteristic curve. NIHSS, National Institutes of Health Stroke Scale. sICH, symptomatic intracerebral hemorrhage. SITS, Safe Implementation of Treatments in Stroke score. SPAN-100, Stroke Prognostication using Age and NIH Stroke Scale index. THRIVE, Totaled Health Risks in Vascular Events score. tPA, tissue plasminogen activator. www.nature.com/scientificreports/ risk of post-thrombolysis hemorrhagic transformation and poor AIS outcome 19,20 ; in fact, Whiteley WN, et al. in a systematic review and meta-analysis of 55 studies showed that a higher stroke severity with an odds ratio of ~ 1.1 per NIHSS point is associated with post-tPA sICH, and that the odds double in the presence of Af-related cardioembolic stroke subtype 18 .
In addition, consistent with our finding that AIS patients at high risk for sICH were concurrently morbid with heart disease, there is indication that being comorbid with congestive heart failure or ischemic heart disease increases the risk of sICH after tPA in patients with AIS 18 . Evidence of brain-heart interactions continues to accrue, with cardiac dysfunction being associated with brain injury; concordant with the conclusions of a population-based 30-year cohort study, "Heart failure was associated with increased short-term and long-term risk of all stroke subtypes" 21 .
In our model 1, high diastolic BP was an indicator of sICH. This is corroborated by findings indicating that elevated BP within the first 24 h following tPA administration in patients with AIS is an independent predictor of sICH 22 , and that extremely high or low systolic and diastolic BP are significantly associated with mortality and disability 23,24 . Thus, guiding medical decision-making, to maintain optimal control of BP during the acute phase of AIS would play a therapeutically significant role in lowering the potential risks of sICH and improve clinical outcome 2, 23 .
It has been suggested that low LDL level damages the integrity of the smooth muscle cells, impairs the endothelial function of cerebral vessels, and consequently increase the risk of hemorrhage 25 . The actual effect of altered serum lipid level on sICH might be confounded by the presence of Af 26 , as lower blood lipid levels have been shown in patients with Af, and hypolipoproteinemia has been touted to increase susceptibility to developing Af 26 . This is corroborated by our finding that LDL and hyperlipidemia were significantly associated with sICH in our AIS cohort, and informed their inclusion into the ANN model for predicting sICH before tPA treatment.
Aging has a negative effect on the long-term outcome of AIS, especially as patients aged > 80 years present with more severe AIS than their younger counterparts 27 , and this is consistent with higher NIHSS score being strongly linked with AIS-related mortality 8,28 . This is further corroborated by findings of the SPAN-100 index study which demonstrated the prognostic relevance of combining patients' age (years) and baseline NIHSS score in patients with AIS 6 . In partial concordance, in our present study both patients' age and NIHSS score were shown to be critical indicators, and were applied in the ANN model for predicting the post-tPA 3-month mortality for patients with AIS.
Furthermore, it is clinically relevant that in our AIS cohort, the patients who died within 3 months of intravenous thrombolysis (i.e. 3-month mortality group) received lower dose of tPA than the survivors. Intuitively, older patients had lower body weights and received lower total dose of tPA. In addition, 51.5% of our AIS cohort who were patients aged > 80 years received low dose tPA (0.6 mg/kg). This 'aging' concept may explain in part the strong association between lower tPA dose and the post-tPA 3-month mortality; this is more likely so, considering cumulative evidence of the safety, efficacy and therapeutic non-inferiority of low dose tPA, compared to standard dose [29][30][31] . In our model, the predictive value for tPA dose suggests the applicability of our ANN model to optimize the tPA dose for individual patients. In our ANN model 2, DM was considered an essential input to predict mortality after tPA. This derives from statistical inference from our clinicopathological analysis, and consistent with published evidence that patients comorbid with DM have less favorable clinical outcomes, including higher death rate and more long-term morbidity after tPA for AIS 32 . Similarly, consistent with our observation that survivors had an lower initial GCS score in contrast to non-survivors, we considered the initial GCS score as a predictor of mortality after tPA for patients with AIS, and included it as one of the attributes of our ANN predictive model for post-tPA 3-month mortality. This is congruous with the findings of previous studies indicating that an impaired consciousness level is an early and independent indicator of unfavorable therapeutic outcome and mortality in patients with AIS 8,33 .
As already alluded, there are a number of studies focused on the prediction of sICH 4-7 and death 7-9 after tPA in patients with AIS. It is noteworthy that the SPAN-100 index which touted to be a simple and fast scoring system achieved a post-tPA sICH detection rate of only 42% in SPAN-100 pos patients, and data on the predictive power of this index are inconsistent 6,9 . Furthermore, the THRIVE score for predicting clinical outcome after thrombolysis which exhibited some superiority to other scoring systems, predicted the risk of hemorrhage and death with AUC of 0.64 and 0.72, respectively 7,34 . Similarly, the SITS score to predict the risk of sICH had an AUC of 0.70 5 . It is thus clinically-relevant that while these studies predict the sICH and mortality after tPA with moderate predictive power, our ANN predictive model AUC for sICH was 0.941, and the ANN predictive model AUC for 3-month mortality was 0.976, respectively. Consistent with contemporary knowledge, our study demonstrates that the training of ANN under supervised learning could emulate human expert diagnostic performance and identify relevant predictive markers for any diagnostic task 12 . We herein exploited some of the benefits of neural networks, such as the requirement of little (or no) formal statistical training, minor user input, the implicit capacity to detect complex nonlinear relationships between dependent and independent variables, and the ability to delineate all possible interactions between predictor variables 35 . The high predictive power of our models is consistent with known application of ANN to assist with the diagnosis of stroke and prediction of its outcome [35][36][37] . More specifically, the present study demonstrated that the application of ANN helped improve the accuracy of predicting sICH within 72 h of intravenous tPA and the risk of death after the intravenous administration of tPA. For patients with high risk of post-tPA sICH and poor outcome, ANN may facilitate the timely institution of indicated adjunctive therapies, such as intra-arterial thrombolysis and mechanical thrombectomy to improve the patients' quality of life and functional outcome. Thus, we demonstrate that ANN has the potential to assist clinicians in patient stratification, and to establish individualized treatment plans for optimized AIS management.
As with other studies of this sort, our study has some limitations. First, the sample size is relatively small and the study is based on single-center data. A larger cohort of more patients from a multi-center setting with Scientific Reports | (2020) 10:20501 | https://doi.org/10.1038/s41598-020-77546-5 www.nature.com/scientificreports/ variable characteristics may be needed for accurate representation of the disease population, and for derivation of robustly generalizable inferences. Secondly, the present study samples excluded patients who received intraarterial thrombectomy or endovascular intervention, thus might have inadvertently excluded a patient bloc with poorer response to treatment with intravenous tPA and thereby underestimate the severity and poor outcomes of AIS in this cohort. Thirdly, the retrospective design of the present study using past registry data makes it almost impractical to rule out investigator bias in control selection. Further studies with a prospective design are therefore warranted to develop AI-based diagnostic and/or prognostic tools that can be continuously updated with new information and evolve with the improvement of medical knowledge and healthcare management. This will help to establish more accurate prediction tools and clinical decision support systems.

Conclusion
Our study demonstrating that ANN techniques can predict sICH and 3-month mortality for patients with AIS who were treated with intravenous tPA, with high accuracy, does indicate that novel AI-based models can be used to derive new knowledge and improve current healthcare management. Results documented herein are potentially applicable in the emergent clinical setting of AIS and can aid decision-making for therapeutic plans.

Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request. www.nature.com/scientificreports/