There is an ongoing revolution in computational behavioral analysis in business and government. Automated analysis of text is used to screen job applicants and score essays, and is applied to social media to influence individuals’ purchasing and voting choices. Automated face and emotion recognition is used for both surveillance and to supplement polygraph testing. Wearables are used to collect physiological data from athletes and astronauts, and increasingly for medical purposes.
Only recently have these computational approaches been applied in psychiatry to study disturbances in thought, emotion, and behavior, which traditionally have been assessed using only expert human appraisal, codified in standardized interviews and ratings, but labor-intensive and error-prone. Herein, we review a sample of ongoing lines of research, in respect to language, emotional expression and physiological parameters.
Automated speech analysis can characterize intoxication by different drugs of abuse with increased verbosity induced by methamphetamine, and increased semantic proximity to words such as friendship/rapport/support/intimacy characterizing intoxication from MDMA or ‘ecstasy’ (Bedi et al, 2014); acoustic features of speech are similarly discriminative (Agurto et al, 2017). This behavioral readout in speech of drug effects has implications for diagnosis, care and clinical trials, as well as for investigations of neural mechanisms of intoxication.
We have also used automated speech analysis to characterize the subtle language disturbance that precedes psychosis onset in schizophrenia (Bedi et al, 2015), identifying a highly accurate predictor of psychosis that comprises both decrease in sentence-level semantic coherence (indexing tangentiality), and decrease in syntactic complexity (indexing concreteness). Further, we have used automated metaphor identification to show a significantly higher rate of metaphor usage across stages of schizophrenia, including putative prodromal stages (Gutierrez et al, 2017). This technology is portable, inexpensive, and easy to implement, and can improve prognosis and understanding of mechanisms of thought disorder in schizophrenia.
Beyond words themselves, voice and face expression also provide important data. At the ACNP Annual Meeting in 2016, Dr Satrajit Ghosh presented data showing that voice acoustic features can be used to predict severity of depression and Parkinson’s disease (Ghosh, 2016). He introduced VoiceUp, his open source mobile platform for collection and analysis of voice data, a sensor into mental health feasible to track over time. In the same ACNP panel, Dr Justin Baker presented data on ‘face action units,’ which comprise specific movements of the mouth, cheeks, and eyes, as collected by video and analyzed using the OpenFace mobile platform (Baker, 2016). Specific face action unit abnormalities were associated with symptom severity, as was the extent of mutual gaze and vowel space in speech.
The smartphones in our pockets provide complex longitudinal in vivo data, much of it passively obtained, including spatial trajectories (GPS), physical movement, and sleep (accelerometer), and social networks and dynamics (phone communication logs); these ‘big data’ at the level of the individual can be used to promote precision psychiatry (Onnela and Rauch, 2016). Smartphones can record sleep patterns, respiration, and heart rate variability. Physiological data, along with language and facial data, can provide accurate and nuanced real-time readout in telepsychiatry, and lead to deep phenotyping that can be integrated with genetic and neuroimaging data.
Funding and disclosure
This research is funded by grants from the NIMH: R01MH107558 and R03MH108933. The authors declare no conflict of interest.
References
Agurto C, Norel R, Ostrand R, Bedi G, Cecchi GA (2017). Phonological markers of oxytocin and MDMA ingestion. Interspeech.
Baker JT (2016). Panel on computational approaches to behavior analysis in psychiatry. 55th Annual Meeting of the American College of Neuropsychopharmacology; 8 December.
Bedi G, Carrillo F, Cecchi GA, Slezak DF, Sigman M, Mota NB et al (2015). Automated analysis of free speech predicts psychosis onset in high-risk youths. NPJ Schizophr, article number 15030 10.
Bedi G, Cecchi GA, Slezak DF, Carrillo F, Sigman M, de Wit H (2014). A window into the intoxicated mind? Speech as an index of psychoactive drug effects. Neuropsychopharmacology 39: 2340–2348.
Ghosh S (2016). Panel on computational approaches to behavior analysis in psychiatry. 55th Annual Meeting of the American College of Neuropsychopharmacology; 8 December.
Gutierrez ED, Corlett PR, Corcoran CM, Cecchi GA (2017). Using automated metaphor identification to aid in detection and prediction of first-episode schizophrenia. Empirical methods in natural language processing (EMNLP).
Onnela JP, Rauch SL (2016). Harnessing smartphone-based digital phenotyping to enhance behavioral and mental health. Neuropsychopharmacology 41: 1691–1696.
Acknowledgements
We would like to acknowledge the work of Drs Gillinder Bedi, Satrajit Ghosh, and Justin T Baker.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Corcoran, C., Cecchi, G. Computational Approaches to Behavior Analysis in Psychiatry. Neuropsychopharmacol. 43, 225–226 (2018). https://doi.org/10.1038/npp.2017.188
Published:
Issue Date:
DOI: https://doi.org/10.1038/npp.2017.188