Implementing machine learning in bipolar diagnosis in China

Bipolar disorder (BPD) is often confused with major depression, and current diagnostic questionnaires are subjective and time intensive. The aim of this study was to develop a new Bipolar Diagnosis Checklist in Chinese (BDCC) by using machine learning to shorten the Affective Disorder Evaluation scale (ADE) based on an analysis of registered Chinese multisite cohort data. In order to evaluate the importance of each item of the ADE, a case-control study of 360 bipolar disorder (BPD) patients, 255 major depressive disorder (MDD) patients and 228 healthy (no psychiatric diagnosis) controls (HCs) was conducted, spanning 9 Chinese health facilities participating in the Comprehensive Assessment and Follow-up Descriptive Study on Bipolar Disorder (CAFÉ-BD). The BDCC was formed by selected items from the ADE according to their importance as calculated by a random forest machine learning algorithm. Five classical machine learning algorithms, namely, a random forest algorithm, support vector regression (SVR), the least absolute shrinkage and selection operator (LASSO), linear discriminant analysis (LDA) and logistic regression, were used to retrospectively analyze the aforementioned cohort data to shorten the ADE. Regarding the area under the receiver operating characteristic (ROC) curve (AUC), the BDCC had high AUCs of 0.948, 0.921, and 0.923 for the diagnosis of MDD, BPD, and HC, respectively, despite containing only 15% (17/113) of the items from the ADE. Traditional scales can be shortened using machine learning analysis. By shortening the ADE using a random forest algorithm, we generated the BDCC, which can be more easily applied in clinical practice to effectively enhance both BPD and MDD diagnosis.


Introduction
Bipolar disorder (BPD) is characterized by recurrent depression and mania/hypomania 1 . Difficulties and delays in the diagnosis of BPD impede the effective treatment of patients. BPD is prone to misdiagnosis as major depressive disorder (MDD). Despite being one of the 10 most debilitating noncommunicable diseases 2,3 , BPD is misdiagnosed as recurrent MDD in 60% of patients seeking treatment for depression 4 . In particular, the recent 3rd national Chinese Mental Health Survey (CMHS) reported that the 12-month prevalence rates of both BPD and MDD had increased to as high as 4.5%, while the recognition rate of BPD versus current major depressive episodes (MDEs) was as high as 39.9% according to the BRIDGE-China study 5,6 .
Therefore, there is an urgent need to improve the early diagnosis of BPD, especially in terms of distinguishing patients with BPD from those with MDD. In light of the current large number of domestic patient diagnoses, easier and targeted diagnostic evaluation tools are needed.
The affective disorder evaluation (ADE) 7 was designed in 2003 as a confidence estimation tool to guide psychiatrists in diagnosing patients and developing treatment plans. The ADE is neither a screening instrument nor a self-report measure but provides a systematic process that helps psychiatrists apply their full assessment to judge the confidence of bipolar diagnosis. Psychometric results from the use of the ADE in routine clinical practice have been published in Russia 8 , the United States 9 , and China 10 , and the output measure is known as the bipolarity index (BPx). However, the ADE contains 145 questions and usually requires 45-90 min to derive the BPx score and form a diagnostic impression. Thus, the ADE is too time consuming for use in clinical practice because psychiatrists cannot allocate that amount of time to each visit, considering the heavy load of patients seeing psychiatrists in China.
Machine learning algorithms can effectively leverage cohort data to generate classifiers and measure the sensitivity and specificity of parameters with respect to diagnostic validity and similarity to the original and revised diagnostic evaluation tools. Machine learning algorithms have already been implemented to shorten many scales, such as the Autism Diagnostic Observation Schedule-Generic (ADOS) for autism diagnosis 11 and the Social Responsiveness Scale (SRS) for behavioral distinction between autism and attention-deficit/hyperactivity disorder (ADHD) 12 .
In this study, to develop a shortened version of the ADE feasible for use in clinical practice, which we named the Bipolar Diagnosis Checklist in Chinese (BDCC), we used a machine learning algorithm to shorten the original ADE based on a retrospective analysis of the Comprehensive Assessment and Follow-up Descriptive Study on Bipolar Disorder (CAFE-BD) data.

Data sample
The included MDD (N = 255) and BPD (N = 360) subjects were outpatients or inpatients at a health facility affiliated with the CAFE-BD. The healthy control (HC) (N = 228) subjects were recruited among people who responded to flyers distributed near the participating health centers.
The CAFE-BD is a collaborative study of nine health centers in China with the goal of implementing a set of standardized intake procedures among six psychiatric hospitals and the mental health departments of three general hospitals. The details of the CAFE-BD can be found at http://ClinicalTrials.gov under the identifier NCT02015143. Ethical approval was obtained from all participating centers.

Sample and assessment procedure
All subjects signed written informed consent and were then initially evaluated by CAFE-BD researchers using the

Machine learning
First, according to expert suggestions on our previous ADE study, we reduced the initial 145 items of the ADE to 113 items by deleting items that had no diagnostic relevance, such as type of medical insurance. Then, five machine learning algorithms were implemented to analyze the aforementioned data using the 113 questions in the ADE as features and MDD, BPD, and HC as prediction classes. Our machine learning pipeline was initialized by randomly splitting the entire data set into 10 stratified subsets, where each subset consisted of 10% of the MDD data (N = 255), BPD data (N = 360), and HC data (N = 228). Then, 10-fold cross-validation was implemented using these subsets, where each cross-validation trial iteratively utilized one subset for testing and the remaining nine subsets for training. For each trial, feature ranking was calculated using the 9-fold training set. All features were ranked based on the minimal-redundancymaximal-relevance (mRMR) mutual information criterion 16 . Then, forward feature selection was performed using the previously obtained ranks to train and test the five machine learning algorithms, with parameter tuning for each choice of features. This process was iteratively implemented for every 10 cross-validation trials, resulting in an average area under the receiver operating characteristic (ROC) curve (AUC) for each model calculated over 100 trials. All machine learning analyses were performed in R. The support vector regression (SVR) algorithm was implemented using the kernlab package with Weston and Watkins' native multiclass formulation 17 and a radial basis function (RBF) kernel 18 . LASSO was implemented using the glmnet package 19 , logistic regression was implemented using the net package 20 , and linear discriminant analysis (LDA) was implemented using the MASS package 21 .

Results
For each trial in the 10-fold cross-validation, the mutual information feature ranking was calculated over the 9-fold training set. In each of these 10 trials, the rankings were the same; the top 17 features are shown in Table 1.
Forward feature selection results using the five aforementioned machine learning algorithms are presented in Fig. 1. The optimal number of features was determined at the point at which there was no further gain in AUC when more features were added. The random forest algorithm performed better than the other algorithms using the same number of features. LASSO and LDA behaved quite similar for the MDD and BPD models, but LDA outperformed LASSO for the HC model. Owing to its instinct mechanism, excessive features led to overfitting of the logistic regression model, and this model exhibited a gradual decline in performance as more features were added to the model. Therefore, the random forest algorithm was selected to generate the BDCC.
As shown in Table 2, the best performance of each machine learning algorithm was compared. The random forest algorithm performed the best but had the most features. Applying 74, 91 or 111 of the ADE questions is still time consuming, and thus, the number of questions was further reduced to make the BDCC feasible in clinical practice. Additional feature reduction was performed using the following criteria: (1) Unique questions were included to allow MDD, BPD, and HC diagnoses using the same model; (2) The AUCs for MDD, BPD, and HC were required to be >95%; (3) The number of questions was minimized.

Discussion
Utilizing five machine learning algorithms, we abbreviated the ADE 22 , a time-efficient record-keeping instrument typically used in research studies, using Chinese multicenter cohort data. The increased AUC reported herein compared with our previous work 6 indicates the acceptability of this result and the validity of the Chinese version of the BDCC. In addition, the BDCC halves the time needed to collect clinical information. It takes more than 30 min to finish the ADE, whereas it takes only 10-15 min to complete the BDCC. Our current results reveal that the BDCC is as robust as the original version but more feasible to implement.
The 17 selected items of the BDCC fall into three categories: current clinical status (11 questions), lifetime clinical trials (5 questions including sudden onset and anger attacks of past depression, dysthymia, age at first use of antidepressant medication, and lifetime euphoria), and past psychiatry history (1 questionnaire on suicide attempts). These above categories correspond to diagnostic criteria, such as those of the DSM-IV.
In the domain of current clinical status, our results suggest that racing thoughts/flight of ideas (FOI), psychomotor agitation and irritability have highly significant correlations with BPD, which is generally in accordance with previous findings 23 . For lifetime traits, we referred to other studies describing more sudden onset 24 and anger attacks 25 among bipolar depressed patients than unipolar depressed patients and reporting that individuals with BPD feel more euphoria than those suffering from MDD 26 . Moreover, receiving antidepressant treatment at a relatively young age 27 and frequently attempting suicide seem to be common among BPD patients. On the other hand, dysthymia corresponded to MDD rather than BPD 28 . Thus, the lifetime features of the BDCC may enhance its stability and feasibility for the diagnosis of BPD and MDD.
In addition, symptoms addressed by the BDCC may explicitly relate to the switch from MDD to BPD. For example, the severity of current mood elevation, addressed by the BDCC, is highly suggestive of BPD, and a previous study identified this symptom as a promising predictor of switching from MDD to BPD 29 . Higher severity of other manic symptoms, including flight of ideas (FOI), psychomotor agitation (PMA), anxiety, and irritability, was associated with a higher risk of switching 30 . Sudden onset of past depression is also a risk factor for switching 31 from MDD to BPD. Thus, the BDCC may help in the early recognition of BPD.

Limitations
This study has several limitations, including its crosssectional nature and the available content of the data sets. Therefore, based on our existing work 10 , a prospective cohort study with a larger sample size will be conducted in the future. A classifier to distinguish bipolar II disorder (BD II) from bipolar I disorder (BD I) will be built by both retrospective and prospective analysis using a new cohort.

Conclusion
In summary, the BDCC scale is a reasonable alternative diagnostic instrument for identifying BPD and MDD and it is a balance between time consuming and amount of questionnaire items optimized by machine learning. Our future study will focus on prospective validation of the BDCC scale.