Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

AI-based analysis of social media language predicts addiction treatment dropout at 90 days

Neuropsychopharmacology volume 48, pages 1579–1585 (2023)Cite this article

Subjects

Abstract

The reoccurrence of use (relapse) and treatment dropout is frequently observed in substance use disorder (SUD) treatment. In the current paper, we evaluated the predictive capability of an AI-based digital phenotype using the social media language of patients receiving treatment for substance use disorders (N = 269). We found that language phenotypes outperformed a standard intake psychometric assessment scale when predicting patients’ 90-day treatment outcomes. We also use a modern deep learning-based AI model, Bidirectional Encoder Representations from Transformers (BERT) to generate risk scores using pre-treatment digital phenotype and intake clinic data to predict dropout probabilities. Nearly all individuals labeled as low-risk remained in treatment while those identified as high-risk dropped out (risk score for dropout AUC = 0.81; p < 0.001). The current study suggests the possibility of utilizing social media digital phenotypes as a new tool for intake risk assessment to identify individuals most at risk of treatment dropout and relapse.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Study design and descriptive information.
Fig. 2: Overall accuracy predicting treatment outcome at 90 days.
Fig. 3: Prediction of future abstinence, relapse, and dropout at 90 days.
Fig. 4: Performance of our full risk assessment model which predicts the probability of dropout by 90 days.

Similar content being viewed by others

References

  1. McLellan AT, Lewis DC, O’Brien CP, Kleber HD. Drug dependence, a chronic medical illness: implications for treatment, insurance, and outcomes evaluation. JAMA J Am Med Assoc. 2000;284:1689–95.

    Article  CAS  Google Scholar 

  2. Paliwal P, Hyman SM, Sinha R. Craving predicts time to cocaine relapse: further validation of the Now and Brief versions of the cocaine craving questionnaire. Drug Alcohol Depend. 2007;93:252–59.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Brecht M-L, Herbeck D. Time to relapse following treatment for methamphetamine use: a long-term perspective on patterns and predictors. Drug Alcohol Depend. 2014;139:18–25.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Volkow ND. Personalizing the treatment of substance use disorders. Am J Psychiatry. 2020;177:113–16.

    Article  PubMed  Google Scholar 

  5. Insel TR. Digital phenotyping: technology for a new science of behavior. JAMA. 2017;318:1215–16.

    Article  PubMed  Google Scholar 

  6. Kwako LE, Bickel WK, Goldman D. Addiction biomarkers: dimensional approaches to understanding addiction. Trends Mol Med. 2018;24:121–28.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Ashford RD, Lynch K, Curtis B. Technology and social media use among patients enrolled in outpatient addiction treatment programs: cross-sectional survey study. J Med Internet Res. 2018;20:e84.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Curtis BL, Ashford RD, Magnuson KI, Ryan-Pettes SR. Comparison of smartphone ownership, social media use, and willingness to use digital interventions between generation z and millennials in the treatment of substance use: Cross-sectional questionnaire study. J Med Internet Res. 2019;21:e13050.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Bergman BG, Greene MC, Hoeppner BB, Kelly JF. Expanding the reach of alcohol and other drug services: Prevalence and correlates of US adult engagement with online technology to address substance problems. Addict Behav. 2018;87:74–81.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Kern ML, Park G, Eichstaedt JC, Schwartz HA, Sap M, Smith LK, et al. Gaining insights from social media language: methodologies and challenges. Psychological Methods. 2016;21:507–25.

    Article  PubMed  Google Scholar 

  11. Son Y, Clouston SAP, Kotov R, Eichstaedt JC, Bromet EJ, Luft BJ, et al. World Trade Center responders in their own words: predicting PTSD symptom trajectories with AI-based language analyses of interviews. Psychological Med. 2021;53:1–9.

    Google Scholar 

  12. Coppersmith G. Digital life data in the clinical whitespace. Curr Directions Psychological Sci. 2022;31:34–40.

    Article  Google Scholar 

  13. Eichstaedt JC, Smith RJ, Merchant RM, Ungar LH, Crutchley P, Preoţiuc-Pietro D, et al. Facebook language predicts depression in medical records. Proc Natl Acad Sci. 2018;115:11203–08.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Youyou W, Kosinski M, Stillwell D. Computer-based personality judgments are more accurate than those made by humans. Proc Natl Acad Sci. 2015;112:1036–40.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Park G, Schwartz HA, Eichstaedt JC, Kern ML, Kosinski M, Stillwell DJ, et al. Automatic personality assessment through social media language. J Personal Soc Psychol. 2015;108:934.

    Article  Google Scholar 

  16. Curtis B, Giorgi S, Buffone AE, Ungar LH, Ashford RD, Hemmons J, et al. Can Twitter be used to predict county excessive alcohol consumption rates? PloS One. 2018;13:e0194290.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Ganesan AV, Matero M, Ravula AR, Vu H, Schwartz HA. Empirical evaluation of pre-trained transformers for human-level NLP: the role of sample size and dimensionality. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics; 2021. p. 4515–32. https://doi.org/10.18653/v1/2021.naacl-main.357.

  18. Devlin J, Chang M-W, Lee K, Toutanova K. Bert: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; 2019, p. 4171–86.

  19. Boyd RL, Schwartz HA. Natural language analysis and the psychology of verbal behavior: the past, present, and future states of the field. J Lang Soc Psychol. 2021;40:21–41.

    Article  PubMed  Google Scholar 

  20. Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, et al. On the opportunities and risks of foundation models. arXiv. 2021. https://arxiv.org/abs/2108.07258.

  21. Matero M, Idnani A, Son Y, Giorgi S, Vu H, Zamani M, et al. Suicide risk assessment with multi-level dual-context language and BERT. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology. Minneapolis, Minnesota: Association for Computational Linguistics; 2019, p. 39–44.

  22. Cacciola JS, Alterman AI, Habing B, McLellan AT. Recent status scores for version 6 of the Addiction Severity Index (ASI-6). Addiction 2011;106:1588–602.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Geurts P, Ernst D, Wehenkel L. Extremely randomized trees. Mach Learn. 2006;63:3–42.

    Article  Google Scholar 

  24. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning data mining, inference, and prediction, 2nd ed. Springer Series in Statistics. New York, NY: Springer New York; 2009. p. 219–59.

  25. Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett. 2006;27:861–74.

    Article  Google Scholar 

  26. Liu T, Giorgi S, Yadeta K, Schwartz HA, Ungar LH, Curtis B. Linguistic predictors from Facebook postings of substance use disorder treatment retention versus discontinuation. Am J Drug Alcohol Abuse. 2022:48(5):573–85.

  27. Schwartz HA, Giorgi S, Sap M, Crutchley P, Ungar L, Eichstaedt J. Dlatk: Differential language analysis toolkit. In Proceedings of the 2017 conference on empirical methods in natural language processing: System demonstrations; 2017, p. 55–60.

  28. Rogers Anna, Kovaleva Olga, Rumshisky Anna. “A primer in BERTology: what we know about how BERT works.” Transactions of the Association for. Computational Linguist. 2021;8:842–66.

    Google Scholar 

  29. Yinhan L, Myle O, Naman G, Jingfei D, Mandar J, Danqi C, et al. RoBERTa: a robustly optimized BERT pretraining approach. arXiv. 2019. https://arxiv.org/abs/1907.11692.

  30. Nguyen DQ, Vu T, Nguyen AT. BERTweet: a pre-trained language model for English Tweets. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics; 2020, p. 9–14.

  31. Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20:273–97.

    Article  Google Scholar 

  32. McCullagh P, Nelder JA. Generalized linear models. Monographs on Statistics and Applied Probability, 2 edn. Vol. 37. Chapman and Hall: CRC press; 1989.

  33. Rammos A, Gonzalez LAN, Weinberger DR, Mitchell KJ, Nicodemus KK. The role of polygenic risk score gene-set analysis in the context of the omnigenic model of schizophrenia. Neuropsychopharmacology. 2019;44:1562–69.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Ashford RD, Giorgi S, Mann B, Pesce C, Sherritt L, Ungar L, et al. Digital recovery networks: characterizing user participation, engagement, and outcomes of a novel recovery social network smartphone application. J Subst Abus Treat. 2020;109:50–55.

    Article  Google Scholar 

  35. Perrin A, Anderson M. Share of US adults using social media, including Facebook, is mostly unchange since 2018. 2019. https://www.pewresearch.org/fact-tank/2019/04/10/share-of-u-s-adults-using-social-media-including-facebook-is-mostly-unchanged-since-2018/.

  36. Social media fact sheet. 2021. https://www.pewresearch.org/internet/fact-sheet/social-media/.

Download references

Acknowledgements

The corresponding author, Dr. Brenda Curtis, had full access to all the data in the study and took responsibility for the integrity of the data and the accuracy of the data analysis.

Funding

This research was supported by: Intramural Research Program of the NIH, NIDA (BC), Templeton Research Trust (LU, HAS), National Institute on Drug Abuse Grant # R01DA039457 (BC, LU), National Institute on Alcohol Abuse and Alcoholism # R01 AA028032/AA (HAS, LU). The authors declare that they have no competing interests. This study was funded by the Intramural Research Program of the National Institutes of Health (NIH), National Institute on Drug Abuse (NIDA).

Author information

Authors and Affiliations

  1. Intramural Research Program, National Institute on Drug Abuse, Baltimore, MD, USA

    Brenda Curtis, Salvatore Giorgi, Tingting Liu & Kenna Yadeta

  2. Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA, USA

    Salvatore Giorgi & Lyle Ungar

  3. Positive Psychology Center, University of Pennsylvania, Philadelphia, PA, USA

    Lyle Ungar, David Yaden & Tingting Liu

  4. Department of Computer Science, Stony Brook University, Stony Brook, NY, USA

    Huy Vu & H. Andrew Schwartz

  5. Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, USA

    David Yaden

Authors
  1. Brenda Curtis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Salvatore Giorgi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lyle Ungar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huy Vu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Yaden
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tingting Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kenna Yadeta
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. H. Andrew Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: BC, SG, LU, HAS. Methodology: BC, SG, LU, HAS. Investigation: BC, SG. Visualization: SG, HV, HAS. Funding acquisition: BC, LU, HAS. Project administration: BC, LU. Supervision: BC, LU. Writing – original draft: BC, SG, HAS. Writing – review & editing: BC, SG, LU, DY, TL, KY, HAS. Software: SG, HV, HAS. Validation: TL, HV. Formal Analysis: SG, TL, HAS. Resources: BC. Data Curation: LU, HAS.

Corresponding author

Correspondence to Brenda Curtis.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Curtis, B., Giorgi, S., Ungar, L. et al. AI-based analysis of social media language predicts addiction treatment dropout at 90 days. Neuropsychopharmacol. 48, 1579–1585 (2023). https://doi.org/10.1038/s41386-023-01585-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41386-023-01585-5

This article is cited by

Search

Advanced search

Quick links