Machine learning analysis to predict the need for ankle foot orthosis in patients with stroke

We investigated the potential of machine learning techniques, at an early stage after stroke, to predict the need for ankle–foot orthosis (AFO) in stroke patients. We retrospectively recruited 474 consecutive stroke patients. The need for AFO during ambulation (output variable) was classified according to the Medical Research Council (MRC) score for the ankle dorsiflexor of the affected limb. Patients with an MRC score of < 3 for the ankle dorsiflexor of the affected side were considered to require AFO, while those with scores ≥ 3 were considered not to require AFO. The following demographic and clinical data collected when patients were transferred to the rehabilitation unit (16.20 ± 6.02 days) and 6 months after stroke onset were used as input data: age, sex, type of stroke (ischemic/hemorrhagic), motor evoked potential data on the tibialis anterior muscle of the affected side, modified Brunnstrom classification, functional ambulation category, MRC score for muscle strength for shoulder abduction, elbow flexion, finger flexion, finger extension, hip flexion, knee extension, and ankle dorsiflexion of the affected side. For the deep neural network model, the area under the curve (AUC) was 0.887. For the random forest and logistic regression models, the AUC was 0.855 and 0.845, respectively. Our findings demonstrate that machine learning algorithms, particularly the deep neural network, are useful for predicting the need for AFO in stroke patients during the recovery phase.

www.nature.com/scientificreports/ advantages regarding the detection of possible interactions between many attributes/variables and hence may be useful in clinical prediction [9][10][11] . In previous studies, machine learning techniques have been used to predict motor and functional recovery in the acute and subacute stages of stroke [12][13][14][15][16] . However, to date, no machine learning study has investigated the prediction of the need for AFO in stroke patients. Therefore, considering its expected impact on stroke management, this study aimed to apply machine learning to predict the need for AFO in stroke patients.

Methods
This study was approved by the Institutional Review Board of Yeungnam University hospital, and informed consent was waived because of the retrospective nature of the study and because the analysis involved anonymous clinical data. All methods were carried out in accordance with relevant guidelines and regulations. This study included patients who were admitted to the rehabilitation department of a single university hospital because of stroke and who were diagnosed using magnetic resonance imaging from January 2009 to April 2020. The steps of the modeling process applied in this study are shown in Fig. 1.
Data collection. The inclusion criteria were as follows: (1) first-ever stroke; (2) age over 20 years; (3) hemiplegia or hemiparesis following stroke; (4) clinical data collected within 7-30 days (early stage, day of transfer, or day of admission to the rehabilitation department) after onset; (5) absence of serious medical complications, such as pneumonia or cardiac problems from onset to final evaluation; and (6) presence of a functional ambulation category (FAC) score of ≥ 1 at 6 months after stroke onset. The exclusion criteria were as follows: (1) ankle dorsiflexion strength of ≥ 3 at initial enrollment; (2) other preexisting brain or spinal cord lesions; and (3) presence of other peripheral neuropathies that could affect ankle dorsiflexion strength, such as peripheral polyneuropathy.
The following demographic and clinical data were collected when patients were transferred to the rehabilitation unit (16.2 ± 6.0 days after stroke onset): age, sex, type of stroke (ischemic/hemorrhagic), the presence of motor evoked potential (MEP) data for the tibialis anterior muscle of the affected side, modified Brunnstrom classification (MBC), FAC, and MRC score for muscle strength with respect to shoulder abduction, elbow flexion, finger flexion, finger extension, hip flexion, knee extension, and ankle dorsiflexion of the affected side. We have selected these input variables because they represent clinical data that is commonly collected when stroke patients are admitted or visit the hospital for rehabilitation. Regarding MEP evaluation, transcranial magnetic stimulation was performed using a Magstim Novametrix 200 magnetic stimulator (Novametrix Inc., Wallingford, CT, USA) with a circular coil (7-cm mean diameter). While the patients were in a relaxed state, MEPs were recorded from tibialis anterior. Details of the other stimulation methods have been outlined in a previous study 17 . Moreover, we determined the MRC score of ankle dorsiflexion for the affected side at 6 months after stroke onset. www.nature.com/scientificreports/ We used 3 machine learning algorithms: deep neural network, random forest, and logistic regression 14 . The deep neural network consists of layers of interconnected artificial neurons. An artificial neuron is designed based on the biological neuron and receives multiple inputs multiplied by weights, and outputs the sum of the inputs 18 . The random forest algorithm comprises several decision trees that consist of multiple true or false conditions using input variables 19 . The sum of the decisions made by the decision trees is used for the final classification 19 . The machine learning models were trained with all variables as inputs to classify patients that were likely to require AFO for the lower extremity of the affected side. For the deep neural network model, 4 layers with 256-512-1024-512 neurons, RMSProp optimizer, and relu activation were used. For the random forest model, 500 decision trees were used. We categorized the output variables as the necessity and non-necessity of AFO during ambulation. Patients with an MRC score of < 3 for the ankle dorsiflexor of the affected side were considered to require AFO, while patients with scores of ≥ 3 were considered not to require AFO.
To prevent overfitting, we reduced the network size (only 4 layers), applied dropout regulation and early stopping, and held back validation and test datasets to check potential overfitting. To avoid under-fitting, we used neural networks with the capability of capturing the variability of the training dataset.
Of the study population, 75% (n = 335), 18.75% (n = 89), and 6.25% (n = 30) were included in the training, validation, and test sets, respectively, to prevent overfitting of the models. TensorFlow version 1.1.0 (Google, Mountain View, CA) and scikit-learn toolkit version 0.18.1 (Google) were used to train the machine learning models.
Statistical analysis. Statistical analyses were performed using python 3.7.9 and scikit-learn version 0.23.2.
Receiver operating characteristic curve analysis was employed, and the area under the curve (AUC) was calculated. The confidence interval (CI) for the AUC was calculated using the approach used by DeLong et al 20 .

Discussion
To the best of our knowledge, this study is the first to use machine learning to predict the need for AFO in stroke patients. AFO is one of the most frequently prescribed braces for the rehabilitation of stroke patients with gait disturbance 21 . The tibialis anterior is one of the muscles that contributes most to ankle flexion, and it is one of the muscles that commonly experiences motor impairment in patients with gait disturbance 22 . In a normal gait, the tibialis anterior is activated during the loading and swing phases 23 . During the swing phase, the activity of the tibialis anterior lifts the foot and toe to obtain foot clearance 23 . In general, AFO can improve foot clearance during the swing and stance phases 24 .
The most noticeable improvements occur in the first few weeks after the onset of stroke, then the rate of improvement slows and reaches a relatively stable state after 3 months [25][26][27] . Within 3 months after stroke onset, www.nature.com/scientificreports/  www.nature.com/scientificreports/ 70% of recovery in motor function is known to occur 28 . After 6 months, recovery usually reaches its limit and enters a chronic phase 29 . Therefore, in this study, we used the MRC score for ankle dorsiflexion at 6 months after stroke onset as an indicator of the need for AFO in stroke patients. Machine learning models have been used to predict motor or cognition recovery in stroke patients [12][13][14][15][16] . For example, Lin et al. have investigated whether machine learning models can predict the recovery of activities of daily living in acute stroke patients 14 . They recruited 313 subjects and predicted the Barthel Index score at discharge using machine learning methods such as logistic regression, support vector machine, and random forest. The average of the AUC for the classification models (logistic regression, support vector machine, and random forest) were 0.755, 0.777 and 0.769 respectively. Other studies evaluated whether machine learning models could predict motor or cognition improvement in the acute and subacute stages of stroke 13,15,16 . Heo et al. have predicted the modified Rankin Scale score using deep neural network, logistic regression, and random forest with 2604 acute ischemic stroke subjects, and report AUCs of 0.888, 0.849, and 0.857, respectively 13 . Sale et al. have studied the predictability of improving motor and cognitive function after rehabilitation treatment from the early stages of stroke. They used data of 55 patients collected at the time of admission to the Department of Rehabilitation Medicine and at discharge, and predicted the Barthel Index and functional independence measure score with a linear support vector machine regression model. All output results and the actual measured results show a good correlation of 0.75-0.81 15 . Wang et al. have constructed a prognostic model of functional outcome using data from 333 patients with primary intracerebral hemorrhage. They utilized Auto-WEKA 2.0 that uses a sequential model-based algorithm configuration to determine the class with the best performance on the given data. Functional scores at 1 and 6 months after onset evaluated with the modified Rankin Scale were used as the outcome data. They show that the AUC predicting a 1-month outcome is 0.899, and the AUC predicting a 6-month outcome is 0.917 16 . The results of these studies are promising, with moderate to high accuracy. Similar to these previous studies, current study has demonstrated that machine learning models could accurately predict the need for AFO in acute stroke patients. Bearing in mind that AUCs of 0.7-0.8, 0.8-0.9, and > 0.9 are generally considered acceptable, excellent, and outstanding, respectively 30 , the ability of the machine learning models used in this study to predict the need for AFO is excellent, with the deep neural network model performing better than the other models (random forest and logistic regression models).
The deep neural network model may be more appropriate for predicting clinical outcomes 31 . Multiple layers of complex networks may be efficient for representing the complex characteristics of the clinical outcomes in a stroke patient 13 . However, the theoretical background underlying the improved performance reported for the deep neural network is unknown 32 . However, given that machine learning models can learn independently with additional data, the previously mentioned results could be improved 33 . Limitations. There are some limitations to this study. First, this was a single-center study, and should be verified with data from other sources. Second, variables used as inputs in machine learning algorithms are usually variables that can be acquired or evaluated in most cases. However, the prediction may be slightly affected by variables and may be adjusted to account for availability when considering data from different centers.

Conclusion
This study demonstrated that machine learning algorithms, particularly the deep neural network, can improve the prediction of the need for AFO in acute stroke patients.