Suicide prediction models: a critical review of recent research with recommendations for the way forward

Abstract

Suicide is a leading cause of death. A substantial proportion of the people who die by suicide come into contact with the health care system in the year before their death. This observation has resulted in the development of numerous suicide prediction tools to help target patients for preventive interventions. However, low sensitivity and low positive predictive value have led critics to argue that these tools have no clinical value. We review these tools and critiques here. We conclude that existing tools are suboptimal and that improvements, if they can be made, will require developers to work with more comprehensive predictor sets, staged screening designs, and advanced statistical analysis methods. We also conclude that although existing suicide prediction tools currently have little clinical value, and in some cases might do more harm than good, an even-handed assessment of the potential value of refined tools of this sort cannot currently be made because such an assessment would depend on evidence that currently does not exist about the effectiveness of preventive interventions. We argue that the only way to resolve this uncertainty is to link future efforts to develop or evaluate suicide prediction tools with concrete questions about specific clinical decisions aimed at reducing suicides and to evaluate the clinical value of these tools in terms of net benefit rather than sensitivity or positive predictive value. We also argue for a focus on the development of individualized treatment rules to help select the right suicide-focused treatments for the right patients at the right times. Challenges will exist in doing this because of the rarity of suicide even among patients considered high-risk, but we offer practical suggestions for how these challenges can be addressed.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1

References

  1. 1.

    World Health Organization (WHO). Mental health: suicide data. 2018. http://www.who.int/mental_health/prevention/suicide/suicideprevent/en/. Accessed 30 April 2019.

  2. 2.

    Ahmedani B, Simon G, Stewart C, Beck A, Waitzfelder B, Rossom R, et al. Health care contacts in the year before suicide death. J Gen Intern Med. 2014;29:870–7.

  3. 3.

    Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. Am J Psychiatry. 2002;159:909–16.

  4. 4.

    Pearson A, Saini P, Da Cruz D, Miles C, While D, Swinson N, et al. Primary care contact prior to suicide in individuals with mental illness. Br J Gen Pr. 2009;59:825.

  5. 5.

    Schaffer A, Sinyor M, Kurdyak P, Vigod S, Sareen J, Reis C, et al. Population‐based analysis of health care contacts among suicide decedents: identifying opportunities for more targeted suicide prevention strategies. World Psychiatry. 2016;15:135–45.

  6. 6.

    Rosen A. Detection of suicidal patients: an example of some limitations in the prediction of infrequent events. J Consult Psychol. 1954;18:397–403.

  7. 7.

    Murphy GE. Clinical identification of suicidal risk. Arch Gen Psychiatry. 1972;27:356–9.

  8. 8.

    Belsher BE, Smolenski DJ, Pruitt LD, Bush NE, Beech EH, Workman DE, et al. Prediction models for suicide attempts and deaths: a systematic review and simulation. JAMA Psychiatry. 2019;76:642–51.

  9. 9.

    Kessler RC. Clinical epidemiological research on suicide-related behaviors: where we are and where we need to go. JAMA Psychiatry 2019;76:777–8.

  10. 10.

    Garb HN, Wood JM. Methodological advances in statistical prediction. Psychol Assess. 2019. https://doi.org/10.1037/pas0000673.

  11. 11.

    Naghavi M. Global, regional, and national burden of suicide mortality 1990–2016: systematic analysis for the Global Burden of Disease Study 2016. BMJ. 2019;364:l94.

  12. 12.

    Katz C, Bolton J, Sareen J. The prevalence rates of suicide are likely underestimated worldwide: why it matters. Soc Psychiatry Psychiatr Epidemiol. 2016;51:125–7.

  13. 13.

    Stone D, Simon T, Fowler K, Kegler S, Yuan K, Holland K et al. Vital signs: trends in state suicide rates - United States, 1999–2016 and circumstances contributing to suicide - 27 States, 2015. MMWR Morb Mortal Wkly Rep. 2018; 67: 617–24.

  14. 14.

    Heron M. Deaths: leading causes for 2016. National Vital Statistics Reports. 67. Hyattsville, MD: National Center for Health Statistics. 2018. https://www.cdc.gov/nchs/data/nvsr/nvsr67/nvsr67_06.pdf.

  15. 15.

    Carroll R, Metcalfe C, Gunnell D. Hospital presenting self-harm and risk of fatal and non-fatal repetition: systematic review and meta-analysis. PLoS ONE. 2014;9:e89944.

  16. 16.

    Walsh G, Sara G, Ryan CJ, Large M. Meta‐analysis of suicide rates among psychiatric in‐patients. Acta Psychiatr Scand. 2015;131:174–84.

  17. 17.

    Chung DT, Ryan CJ, Hadzi-Pavlovic D, Singh SP, Stanton C, Large MM. Suicide rates after discharge from psychiatric facilities: a systematic review and meta-analysis. JAMA Psychiatry. 2017;74:694–702.

  18. 18.

    Chung D, Hadzi-Pavlovic D, Wang M, Swaraj S, Olfson M, Large M. Meta-analysis of suicide rates in the first week and the first month after psychiatric hospitalisation. BMJ Open. 2019;9:e023883.

  19. 19.

    Carter G, Milner A, McGill K, Pirkis J, Kapur N, Spittal MJ. Predicting suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive predictive values for risk scales. Br J Psychiatry. 2017;210:387–95.

  20. 20.

    Chan KCG, Yam SCP, Zhang Z. Globally efficient non‐parametric inference of average treatment effects by empirical balancing calibration weighting. J R Stat Soc Ser B Stat Methodol. 2016;78:673–700.

  21. 21.

    Katz C, Randall JR, Sareen J, Chateau D, Walld R, Leslie WD, et al. Predicting suicide with the SAD PERSONS scale. Depress Anxiety. 2017;34:809–16.

  22. 22.

    Large M, Myles N, Myles H, Corderoy A, Weiser M, Davidson M, et al. Suicide risk assessment among psychiatric inpatients: a systematic review and meta-analysis of high-risk categories. Psychol Med. 2018;48:1119–27.

  23. 23.

    Large M, Kaneson M, Myles N, Myles H, Gunaratne P, Ryan C. Meta-analysis of longitudinal cohort studies of suicide risk assessment among psychiatric patients: heterogeneity in results and lack of improvement over time. PLoS ONE. 2016;11:e0156322.

  24. 24.

    Larkin C, Di Blasi Z, Arensman E. Risk factors for repetition of self-harm: a systematic review of prospective hospital-based studies. PLoS ONE. 2014;9:e84282.

  25. 25.

    Quinlivan L, Cooper J, Davies L, Hawton K, Gunnell D, Kapur N. Which are the most useful scales for predicting repeat self-harm? A systematic review evaluating risk scales using measures of diagnostic accuracy. BMJ Open. 2016;6:e009297.

  26. 26.

    Runeson B, Odeberg J, Pettersson A, Edbom T, Jildevik Adamsson I, et al. Instruments for the assessment of suicide risk: a systematic review evaluating the certainty of the evidence. PLoS ONE. 2017;12:e0180292.

  27. 27.

    Bolton JM, Gunnell D, Turecki G. Suicide risk assessment and intervention in people with mental illness. Brit Med J. 2015;351:h4978.

  28. 28.

    Woodford R, Spittal MJ, Milner A, McGill K, Kapur N, Pirkis J, et al. Accuracy of clinician predictions of future self-harm: a systematic review and meta-analysis of predictive studies. Suicide Life Threat Behav. 2019;49:23–40.

  29. 29.

    Barak-Corren Y, Castro VM, Javitt S, Hoffnagle AG, Dai Y, Perlis RH, et al. Predicting suicidal behavior from longitudinal electronic health records. Am J Psychiatry. 2017;174:154–62.

  30. 30.

    Ben-Ari A, Hammond K. Text mining the EMR for modeling and predicting suicidal behavior among US veterans of the 1991 Persian gulf war. 2015 48th Hawaii International Conference on System Sciences. Kauai, HI; 2015;3168–75. https://doi.org/10.1109/HICSS.2015.

  31. 31.

    Choi SB, Lee W, Yoon J-H, Won J-U, Kim DW. Ten-year prediction of suicide death using Cox regression and machine learning in a nationwide retrospective cohort study in South Korea. J Affect Disord. 2018;231:8–14.

  32. 32.

    Kessler RC, Hwang I, Hoffmire CA, McCarthy JF, Petukhova MV, Rosellini AJ, et al. Developing a practical suicide risk prediction model for targeting high‐risk patients in the Veterans Health Administration. Int J Methods Psychiatr Res. 2017;26. https://doi.org/10.1002/mpr.1575

  33. 33.

    Walsh CG, Ribeiro JD, Franklin JC. Predicting risk of suicide attempts over time through machine learning. Clin Psychol Sci. 2017;5:457–69.

  34. 34.

    Simon R. Improving suicide risk assessment with evidence-based psychiatry. In: Pompili M, Taterelli R editors. Evidence-based practice in suicidology: a sourcebook. Cambridge MA: Hogrefe Publishing; 2011, p. 45–54.

  35. 35.

    Beck A, Steer R. BHS, Beck Hopelessness Scale: manual. San Antonio TX: Psychological Corporation; 1988.

  36. 36.

    Beck A, Steer R, Brown G. Manual for the Beck Depression Inventory-II. San Antonio TX: Psychological corporation; 1996.

  37. 37.

    Beck A, Schuyler D, Herman I. Development of suicidal intent scales. In: Beck A, Lettieri D, Resnik H editors. The prediction of suicide. Bowie, MD: Charles Press; 1974, p. 45–56.

  38. 38.

    Koldsland BO, Mehlum L, Mellesdal LS, Walby FA, Diep LM. The suicide assessment scale: psychometric properties of a Norwegian language version. BMC Res Notes. 2012;5:417 https://doi.org/10.1186/1756-0500-5-417

  39. 39.

    Kreitman N, Foster J. The construction and selection of predictive scales, with special reference to parasuicide. Br J Psychiatry. 1991;159:185–92.

  40. 40.

    Randall JR, Rowe BH, Dong KA, Nock MK, Colman I. Assessment of self-harm risk using implicit thoughts. Psychol Assess. 2013;25:714–21.

  41. 41.

    Robins JM, Rotnitzky A. Semiparametric efficiency in multivariate regression models with missing data. J Am Stat Assoc. 2015;90:122–9.

  42. 42.

    Bolton JM. Suicide risk assessment in the emergency department: out of the darkness. Depress Anxiety. 2015;32:73–75.

  43. 43.

    Hoge CW. Suicide reduction and research efforts in service members and veterans-sobering realities. JAMA Psychiatry. 2019. https://doi.org/10.1001/jamapsychiatry.2018.4564.

  44. 44.

    Mulder R, Newton-Howes G, Coid JW. The futility of risk prediction in psychiatry. Br J Psychiatry. 2016;209:271–2.

  45. 45.

    Owens D, Kelley R. Predictive properties of risk assessment instruments following self-harm. Br J Psychiatry. 2017;210:384–6.

  46. 46.

    Wortzel HS, Nazem S, Bahraini NH, Matarazzo BB. Why suicide risk assessment still matters. J Psychiatr Pr. 2017;23:436–40.

  47. 47.

    Hunter C, Chantler K, Kapur N, Cooper J. Service user perspectives on psychosocial assessment following self-harm and its impact on further help-seeking: a qualitative study. J Affect Disord. 2013;145:315–23.

  48. 48.

    Owens C, Hansford L, Sharkey S, Ford T. Needs and fears of young people presenting at accident and emergency department following an act of self-harm: secondary analysis of qualitative data. Br J Psychiatry. 2016;208:286–91.

  49. 49.

    Taylor TL, Hawton K, Fortune S, Kapur N. Attitudes towards clinical services among people who self-harm: systematic review. Br J Psychiatry. 2009;194:104–10.

  50. 50.

    Palmer L, Blackwell H, Strevens P. Service users’ experience of emergency services following self harm: a national survey of 509 patients. College Centre for Quality Improvement, Royal College of Psychiatrists. 2007. https://www.rcpsych.ac.uk/. Accessed 20 Feb 2018

  51. 51.

    Rosen DC, Nakash O, Alegria M. The impact of computer use on therapeutic alliance and continuance in care during the mental health intake. Psychother (Chic). 2016;53:117–23.

  52. 52.

    Self-harm in over 8s: long-term management. National Institute for Health and Care Excellence (NICE). 2011. Accessed 30 April 2019

  53. 53.

    O’Connor E, Gaynes BN, Burda BU, Soh C, Whitlock EP. Screening for and treatment of suicide risk relevant to primary care: a systematic review for the US Preventive Services Task Force. Ann Intern Med. 2013;158:741–54.

  54. 54.

    National Institute for Health and Care Excellence. Preventing suicide in community and custodial settings. 2018. https://www.nspa.org.uk/wp-content/uploads/2018/09/preventing-suicide-in-community-and-custodial-settings-pdf-66141539632069.pdf.

  55. 55.

    Bernert R, Hom M, Roberts L. A review of multidisciplinary clinical practice guidelines in suicide prevention: toward an emerging standard in suicide risk assessment and management, training and practice. Acad Psychiatry. 2014;38:585–92.

  56. 56.

    Silverman JJ, Galanter M, Jackson-Triche M, Jacobs DG, Lomax JW, Riba MB, et al. The American Psychiatric Association practice guidelines for the psychiatric evaluation of adults. Am J Psychiatry. 2015;172:798–802.

  57. 57.

    Quinlivan L, Cooper J, Steeg S, Davies L, Hawton K, Gunnell D, et al. Scales for predicting risk following self-harm: an observational study in 32 hospitals in England. BMJ Open. 2014;4:e004732.

  58. 58.

    Rudd M. Core competencies, warning signs, and a framework for suicide risk assessment in clinical practice. In: Nock M editor. The Oxford handbook of suicide and self-injury. 1st ed. New York: Oxford University Press; 2014. p. 323–36.

  59. 59.

    Cooper J, Steeg S, Bennewith O, Lowe M, Gunnell D, House A, et al. Are hospital services for self-harm getting better? An observational study examining management, service provision and temporal trends in England. BMJ Open. 2013;3:e003444.

  60. 60.

    Dawes RM, Faust D, Meehl PE. Clinical versus actuarial judgment. Science. 1989;243:1668–74.

  61. 61.

    Ægisdóttir S, White MJ, Spengler PM, Maugherman AS, Anderson LA, Cook RS, et al. The meta-analysis of clinical judgment project: fifty-six years of accumulated research on clinical versus statistical prediction. Couns Psychol. 2006;34:341–82.

  62. 62.

    Large M, Sharma S, Cannon E, Ryan C, Nielssen O. Risk factors for suicide within a year of discharge from psychiatric hospital: a systematic meta-analysis. Aust N Z J Psychiatry. 2011;45:619–28.

  63. 63.

    Jobes D, Au J, Siegelman A. Psychological approaches to suicide treatment and prevention. Curr Treat Options Psychiatry. 2015;2:363–70.

  64. 64.

    Smith KA, Cipriani A. Lithium and suicide in mood disorders: updated meta-review of the scientific literature. Bipolar Disord. 2017;19:575–86.

  65. 65.

    Vermeulen JM, van Rooijen G, van de Kerkhof MPJ, Sutterland AL, Correll CU, de Haan L. Clozapine and long-term mortality risk in patients with schizophrenia: a systematic review and meta-analysis of Studies Lasting 1.1-12.5 Years. Schizophr Bull. 2019;45:315–29.

  66. 66.

    United States. Public Health Service. Office of the Surgeon General. 2012 National strategy for suicide prevention: goals and objectives for action. National Action Alliance for Suicide Prevention. Washington, DC: National Action Alliance for Suicide Prevention; 2012. https://www.surgeongeneral.gov/library/reports/national-strategy-suicide-prevention/full-report.pdf.

  67. 67.

    Brodsky BS, Spruch-Feiner A, Stanley B. The zero suicide model: applying evidence-based suicide prevention practices to clinical care. Front Psychiatry. 2018;9:33.

  68. 68.

    Jacobs DG. Suicide Assessment Five-step Evaluation and Triage for mental health professionals (SAFE-T). 2009. https://www.integration.samhsa.gov/images/res/SAFE_T.pdf.

  69. 69.

    Fernhoff PM. Newborn screening for genetic disorders. Pediatr Clin North Am. 2009;56:505–13.

  70. 70.

    Kessler R, Bernecker S, Bossarte R, Luedtke A, McCarthy JF, Nock MK, et al. The role of big data analytics in predicting suicide In: Passos I, Mwangi B, Kapczinski F, editors. Personalized Psychiatry—big data analytics in mental health. Springer Nature, 2019. p. 77–98.

  71. 71.

    Hammond KW, Laundry RJ, O’Leary TM, Jones WP. Use of text search to effectively identify lifetime prevalence of suicide attempts among Veterans. 2013 46th Hawaii International Conference on System Sciences; Wailea, Maui, HI; 2013. p. 2676–83.

  72. 72.

    Hammond KW, Laundry RJ. Application of a hybrid text mining approach to the study of suicidal behavior in a large population. 2014 47th Hawaii International Conference on System Science; Waikoloa, HI; 2014. p. 2555–61.

  73. 73.

    Fernandes AC, Dutta R, Velupillai S, Sanyal J, Stewart R, Chandran D. Identifying suicide ideation and suicidal attempts in a psychiatric clinical research database using natural language processing. Sci Rep. 2018;8:7426.

  74. 74.

    Carson NJ, Mullin B, Sanchez MJ, Lu F, Yang K, Menezes M, et al. Identification of suicidal behavior among psychiatrically hospitalized adolescents using natural language processing and machine learning of electronic health records. PLoS ONE. 2019;14:e0211116.

  75. 75.

    Hammond KW, Ben‐Ari AY, Laundry RJ, Boyko EJ, Samore MH. The feasibility of using large‐scale text mining to detect adverse childhood experiences in a VA‐treated population. J Trauma Stress. 2015;28:505–14.

  76. 76.

    McCoy TH, Pellegrini AM, Perlis RH. Research domain criteria scores estimated through natural language processing are associated with risk for suicide and accidental death. Depress Anxiety. 2019;36:392–9.

  77. 77.

    Simon GE, Johnson E, Lawrence JM, Rossom RC, Ahmedani B, Lynch FL, et al. Predicting Suicide attempts and suicide deaths following outpatient visits using electronic health records. Am J Psychiatry. 2018;175:951–60.

  78. 78.

    American Community Survery (ACS). United States Census Bureau. 2018. https://www.census.gov/programs-surveys/acs/about.html. Accessed 8 Aug 2019.

  79. 79.

    Lopez-Castroman J, Moulahi B, Aze J, Bringay S, Deninotti J, Guillaume S, et al. Mining social networks to improve suicide prevention: a scoping review. J Neurosci Res. 2019. https://doi.org/10.1002/jnr24404.

  80. 80.

    Pestian JP, Sorter M, Connolly B, Bretonnel Cohen K, McCullumsmith C, Gee JT, et al. A machine learning approach to identifying the thought markers of suicidal subjects: a prospective multicenter trial. Suicide Life Threat Behav. 2017;47:112–21.

  81. 81.

    Brown JL, Swartzendruber A, Diclemente RJ. Application of audio computer-assisted self-interviews to collect self-reported health data: an overview. Caries Res. 2013;47:40–5.

  82. 82.

    Gnambs T, Kaspar K. Disclosure of sensitive behaviors across self-administered survey modes: a meta-analysis. Behav Res Methods. 2015;47:1237–59.

  83. 83.

    Greist JH, Laughren TP, Gustafson DH, Stauss FF, Rowse GL, Chiles JA. A computer interview for suicide-risk prediction. Am J Psychiatry. 1973;130:1327–32.

  84. 84.

    Levine S, Ancill RJ, Roberts AP. Assessment of suicide risk by computer‐delivered self‐rating questionnaire: preliminary findings. Acta Psychiatr Scand. 1989;80:216–20.

  85. 85.

    Nock MK, Park JM, Finn CT, Deliberto TL, Dour HJ, Banaji MR. Measuring the suicidal mind: implicit cognition predicts suicidal behavior. Psychol Sci. 2010;21:511–7.

  86. 86.

    Bryan CJ, Rudd MD, Wertenberger E, Etienne N, Ray-Sannerud BN, Morrow CE, et al. Improving the detection and prediction of suicidal behavior among military personnel by measuring suicidal beliefs: an evaluation of the Suicide Cognitions Scale. J Affect Disord. 2014;159:15–22.

  87. 87.

    Dhingra K, Boduszek D, O’Connor RC. Differentiating suicide attempters from suicide ideators using the Integrated Motivational-Volitional model of suicidal behaviour. J Affect Disord. 2015;186:211–8.

  88. 88.

    Stefansson J, Nordstrom P, Runeson B, Asberg M, Jokinen J. Combining the Suicide Intent Scale and the Karolinska Interpersonal Violence Scale in suicide risk assessments. BMC Psychiatry. 2015;15:226.

  89. 89.

    Ursano RJ, Colpe LJ, Heeringa SG, Kessler RC, Schoenbaum M, Stein MB. The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Psychiatry. 2014;77:107–19.

  90. 90.

    Bernecker SL, Zuromski KL, Gutierrez PM, Joiner TE, King AJ, Liu H, et al. Predicting suicide attempts among soldiers who deny suicidal ideation in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Behav Res Ther. 2018. https://doi.org/10.1016/j.brat.2018.11.018

  91. 91.

    Boulesteix AL, Schmid M. Machine learning versus statistical modeling. Biom J. 2014;56:588–93.

  92. 92.

    Harrell JFE. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. 2nd ed. 2015. Cham, Switzerland: Springer International Publishing; 2015.

  93. 93.

    Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. 2019. https://doi.org/10.1016/j.clinepi.2019.02.004.

  94. 94.

    Archer K, Kimes R. Empirical characterization of random forest variable importance measures. Comput Stat Data Anal. 2008;52:2249–60.

  95. 95.

    Polley E, LeDell E, van der Laan MJ. Super learner: super learner prediction. R package version 2.0-21: The Comprehensive R Archive Network; 2016. [Computer software]. Available at: https://cran.rstudio.org/. Accessed 30 April 2019.

  96. 96.

    Feurer M, Klein A, Eggensperger K, Springenberg JT, Blum M, Hutter F. Efficient and robust automated machine learning. Proceedings of the 28th International Conference on Neural Information Processing Systems. Vol 2; Montreal, Canada; 2015.

  97. 97.

    Olson RS, Sipper M, La Cava W, Tartarone S, Vitale S, Fu W et al. A system for accessible artificial intelligence. arXiv:1705.00594v2. 2017. Available from: https://arxiv.org/abs/1705.00594.

  98. 98.

    Urbanowicz RJ, Meeker M, La Cava W, Olson RS, Moore JH. Relief based feature selection: introduction and review. arXiv:1711.08421. 2018. Available from: https://arxiv.org/abs/1711.08421.

  99. 99.

    Chawla N. Data mining for imbalanced datasets: an overview. In: Maimon O, Rokach L editors. Data mining and knowledge discovery handbook. 2nd ed. Berlin/Heidelberg, Germany: Springer; 2010. p. 875–86.

  100. 100.

    Kessler RC, Warner CH, Ivany C, Petukhova MV, Rose S, Bromet EJ, et al. Predicting suicides after psychiatric hospitalization in US Army Soldiers: the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). JAMA Psychiatry. 2015;72:49–57.

  101. 101.

    McCarthy JF, Bossarte RM, Katz IR, Thompson C, Kemp J, Hannemann CM, et al. Predictive modeling and concentration of the risk of suicide: implications for preventive interventions in the US Department of Veterans Affairs. Am J Public Health. 2015;105:1935–42.

  102. 102.

    Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. Brit Med J. 2016;352:i6.

  103. 103.

    Van Calster B, Wynants L, Verbeek JFM, Verbakel JY, Christodoulou E, Vickers AJ, et al. Reporting and interpreting decision curve analysis: a guide for investigators. Eur Urol. 2018;74:796–804.

  104. 104.

    McKernan LC, Clayton EW, Walsh CG. Protecting life while preserving liberty: ethical recommendations for suicide prevention with artificial intelligence. Front Psychiatry. 2018;9:650.

  105. 105.

    Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak. 2006;26:565–74.

  106. 106.

    Hoffmire C, Stephens B, Morley S, Thompson C, Kemp J, Bossarte RM. VA Suicide Prevention Applications Network: a national health care system–based suicide event tracking system. Public Health Rep. 2016;131:816–21.

  107. 107.

    Miller IW, Gaudiano BA, Weinstock LM. The coping long term with active suicide program: description and pilot. Suicide Life Threat Behav. 2018;46:752–61.

  108. 108.

    Stone JN, Robinson GJ, Lichtenstein HA, Bairey Merz NC, Blum BC, Eckel HR, et al. 2013 ACC/AHA Guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on practice guidelines. Circulation. 2014;129:S1–45.

  109. 109.

    Comtois KA, Kerbrat AH, DeCou CR, Atkins DC, Majeres JJ, Baker JC, et al. Effect of augmenting standard care for military personnel with brief caring text messages for suicide prevention: a randomized clinical trial. JAMA Psychiatry. 2019;76:474–83.

  110. 110.

    Mortality risk valuation. United States Environmental Protection Agency; [updated February 8, 2018]; Available from: https://www.epa.gov/environmental-economics/mortality-risk-valuation. Accessed Mar 2019.

  111. 111.

    Zalsman G, Hawton K, Wasserman D, van Heeringen K, Arensman E, Sarchiapone M, et al. Suicide prevention strategies revisited: 10-year systematic review. Lancet Psychiatry. 2016;3:646–59.

  112. 112.

    Jobes DA. The Collaborative Assessment and Management of Suicidality (CAMS): an evolving evidence-based clinical approach to suicidal risk. Suicide Life Threat Behav. 2012;42:640–53.

  113. 113.

    Frakt AB, Prentice JC, Pizer SD, Elwy AR, Garrido MM, Kilbourne AM, et al. Overcoming challenges to evidence-based policy development in a large, integrated delivery system. Health Serv Res. 2018;53:4789–807.

  114. 114.

    Cohen ZD, DeRubeis RJ. Treatment selection in depression. Annu Rev Clin Psychol. 2018;14:209–36.

  115. 115.

    VanderWeele T, Luedtke A, van der Laan MJ, Kessler RC. Selecting optimal subgroups for treatment using many covariates. arXiv:1802.09642. 2018. Available from: https://arxiv.org/abs/1802.09642.

  116. 116.

    Zubizarreta JR. Stable weights that balance covariates for estimation with incomplete outcome data. J Am Stat Assoc. 2015;110:910–22.

  117. 117.

    Luedtke AR, van der Laan MJ. Statistical inference for the mean outcome under a possibly non-unique optimal treatment strategy. Ann Stat. 2016;44:713–42.

  118. 118.

    Steeg S, Carr M, Emsley R, Hawton K, Waters K, Bickley H, et al. Suicide and all-cause mortality following routine hospital management of self-harm: propensity score analysis using multicentre cohort data. PLoS ONE. 2018;13:e0204670.

  119. 119.

    Ichimura A, Kato K, Taira T, Otsuka H, Seki T, Nakagawa Y et al. Psychiatric hospitalization after emergency treatment for deliberate self-harm is associated with repeated deliberate self-harm. Arch Suicide Res. 2018. https://doi.org/10.1080/13811118.2018.1438323.

  120. 120.

    Large MM, Kapur N. Psychiatric hospitalisation and the risk of suicide. Br J Psychiatry. 2018;212:269–73.

  121. 121.

    Luedtke AR, van Der, Laan MJ. Optimal individualized treatments in resource-limited settings. Int J Biostat. 2016;12:283–303.

Download references

Acknowledgements

This work was supported, in part, by the Department of Veterans Affairs Center of Excellence for Suicide Prevention and the Precision Treatment of Mental Disorders Initiative. The contents are solely the responsibility of the authors and do not necessarily represent the views of the Veteran’s Health administration. The authors appreciate the helpful comments of Matthew K. Nock, Vicki Shahly, Murray B. Stein, and Robert J. Ursano on an earlier version of the paper.

Author information

Correspondence to Ronald C. Kessler.

Ethics declarations

Conflict of interest

In the past 3 years, RCK was a consultant for Johnson & Johnson Wellness and Prevention, Sage, Shire, and Takeda and served on an advisory board for the Johnson & Johnson Services Inc. Lake Nona Life Project. Kessler is a co-owner of DataStat, Inc., a market research firm that carries out health care research. In the past 3 years, JRZ was a consultant for Johnson & Johnson. The remaining authors declare that they have no conflict of interest.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kessler, R.C., Bossarte, R.M., Luedtke, A. et al. Suicide prediction models: a critical review of recent research with recommendations for the way forward. Mol Psychiatry 25, 168–179 (2020). https://doi.org/10.1038/s41380-019-0531-0

Download citation