Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Consensus Statement
  • Published:

How to e-mental health: a guideline for researchers and practitioners using digital technology in the context of mental health

Nature Mental Health volume 1, pages 542–554 (2023)Cite this article

Subjects

Abstract

Despite an exponentially growing number of digital or e-mental health services, methodological guidelines for research and practical implementation are scarce. Here we aim to promote the methodological quality, evidence and long-term implementation of technical innovations in the healthcare system. This expert consensus is based on an iterative Delphi adapted process and provides an overview of the current state-of-the-art guidelines and practical recommendations on the most relevant topics in e-mental health assessment and intervention. Covering three objectives, that is, development, study specifics and intervention evaluation, 11 topics were addressed and co-reviewed by 25 international experts and a think tank in the field of e-mental health. This expert consensus provides a comprehensive essence of scientific knowledge and practical recommendations for e-mental health researchers and clinicians. This way, we aim to enhance the promise of e-mental health: low-threshold access to mental health treatment worldwide.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Flow diagram illustrating the paper creation process.
Fig. 2: E-mental health study conceptualization process.
Fig. 3: Overview of mobile sensors embedded in consumer electronics and variables they provide117.

Similar content being viewed by others

References

  1. World Health Organization. Mental Health and COVID-19: Early Evidence of the Pandemic’s Impact: Scientific Brief (2022).

  2. Adair, C. E. et al. Continuity of care and health outcomes among persons with severe mental illness. Psychiatr. Serv. 56, 1061–1069 (2005).

    Article  PubMed  Google Scholar 

  3. Schulte, J., Schulz, C., Wilhelm, S. & Buhlmann, U. Treatment utilization and treatment barriers in individuals with body dysmorphic disorder. BMC Psychiatry 20, 69 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  4. van Daele, T. et al. Online consultations in mental healthcare: modelling determinants of use and experience based on an international survey study at the onset of the pandemic. Internet Interv. 30, 100571 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Paganini, S., Teigelkötter, W., Buntrock, C. & Baumeister, H. Economic evaluations of internet- and mobile-based interventions for the treatment and prevention of depression: a systematic review. J. Affect. Disord. 225, 733–755 (2018).

    Article  PubMed  Google Scholar 

  6. Mendes-Santos, C., Nunes, F., Weiderpass, E., Santana, R. & Andersson, G. Understanding mental health professionals’ perspectives and practices regarding the implementation of digital mental health: qualitative study. JMIR Form. Res. 6, e32558 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  7. Atik, E., Schückes, M. & Apolinário-Hagen, J. Patient and therapist expectations for a blended cognitive behavioral therapy program for depression: qualitative exploratory study. JMIR Ment. Health 9, e36806 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  8. de Witte, N. A. J. et al. Online consultations in mental healthcare during the COVID-19 outbreak: an international survey study on professionals’ motivations and perceived barriers. Internet Interv. 25, 100405 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Smoktunowicz, E. et al. Consensus statement on the problem of terminology in psychological interventions using the internet or digital components. Internet Interv. 21, 100331 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  10. O’Loughlin, K., Neary, M., Adkins, E. C. & Schueller, S. M. Reviewing the data security and privacy policies of mobile apps for depression. Internet Interv. 15, 110–115 (2019).

    Article  PubMed  Google Scholar 

  11. Hennemann, S., Farnsteiner, S. & Sander, L. Internet- and mobile-based aftercare and relapse prevention in mental disorders: a systematic review and recommendations for future research. Internet Interv. 14, 1–17 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  12. Torous, J. et al. Creating a digital health smartphone app and digital phenotyping platform for mental health and diverse healthcare needs: an interdisciplinary and collaborative approach. J. Technol. Behav. Sci. 4, 73–85 (2019).

    Article  Google Scholar 

  13. Lal, S. & Adair, C. E. E-mental health: a rapid review of the literature. Psychiatr. Serv. 65, 24–32 (2014).

    Article  PubMed  Google Scholar 

  14. Pill, J. The Delphi method: substance, context, a critique and an annotated bibliography. Socio Econ. Plan. Sci. 5, 57–71 (1971).

    Article  Google Scholar 

  15. Bartlett Ellis, R. et al. Lessons learned: beta-testing the digital health checklist for researchers prompts a call to action by behavioral scientists. J. Med. Internet Res. 23, e25414 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  16. Shen, F. X. et al. An ethics checklist for digital health research in psychiatry: viewpoint. J. Med. Internet Res. 24, e31146 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  17. World Health Organization. Digital Implementation Investment Guide (DIIG): Integrating Digital Interventions into Health Programmes https://apps.who.int/iris/bitstream/handle/10665/334306/9789240010567-eng.pdf (2020)/

  18. Unsworth, H. et al. The NICE Evidence Standards Framework for digital health and care technologies—developing and maintaining an innovative evidence framework with global impact. Digit. Health 7, 20552076211018617 (2021).

    PubMed  PubMed Central  Google Scholar 

  19. Sundareswaran, V. & Sarkar, A. Chatbots RESET: A Framework for Governing Responsible Use of Conversational AI in Healthcare https://www3.weforum.org/docs/WEF_Governance_of_Chatbots_in_Healthcare_2020.pdf (World Economic Forum, 2020).

  20. Doraiswamy, P. M. et al. Empowering 8 billion minds: enabling better mental health for all via the ethical adoption of technologies. NAM Perspect. https://doi.org/10.31478/201910b (2019).

  21. Hekler, E. B. et al. Agile science: creating useful products for behavior change in the real world. Transl. Behav. Med. 6, 317–328 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  22. Fiedler, J., Seiferth, C., Eckert, T., Woll, A. & Wunsch, K. A just-in-time adaptive intervention to enhance physical activity in the SMARTFAMILY2.0 trial. Sport Exerc. Perform. Psychol. https://doi.org/10.1037/spy0000311 (2022).

  23. Chan, A. H. Y. & Honey, M. L. L. User perceptions of mobile digital apps for mental health: acceptability and usability—an integrative review. J. Psychiatr. Ment. Health Nurs. 29, 147–168 (2022).

    Article  PubMed  Google Scholar 

  24. Vial, S. & Boudhraâ, S. in Revolutions in Product Design for Healthcare (eds Subburaj, K. et al.) 21–34 (Springer, 2022).

  25. Narayan, S., Mok, H., Ho, K. & Kealy, D. I don't think they're as culturally sensitive: a mixed-method study exploring e-mental health use among culturally diverse populations. J. Ment. Health 32, 241–247 (2022).

    Article  PubMed  Google Scholar 

  26. Wright, M. T., Springett, J. & Kongats, K. in Participatory Health Research (eds Wright, M. T. & Kongats, K.) 3–15 (Springer, 2018).

  27. Wright, M. T. Partizipative Gesundheitsforschung: Ursprünge und heutiger Stand. Bundesgesundheitsblatt Gesundheitsforsch. Gesundheitsschutz 64, 140–145 (2021).

    Article  Google Scholar 

  28. Carman, K. L. et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Affairs 32, 223–231 (2013).

    Article  PubMed  Google Scholar 

  29. Mummah, S. A., Robinson, T. N., King, A. C., Gardner, C. D. & Sutton, S. IDEAS (Integrate, Design, Assess, and Share): a framework and toolkit of strategies for the development of more effective digital interventions to change health behavior. J. Med. Internet Res. 18, e317 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  30. McCurdie, T. et al. mHealth consumer apps: the case for user-centered design. Biomed. Instrum. Technol. https://doi.org/10.2345/0899-8205-46.s2.49 (2012).

  31. Knowles, S. E. et al. Qualitative meta-synthesis of user experience of computerised therapy for depression and anxiety. PLoS ONE 9, e84323 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  32. Werner-Seidler, A. et al. A smartphone app for adolescents with sleep disturbance: development of the Sleep Ninja. JMIR Ment. Health 4, e28 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  33. Braun, V. & Clarke, V. in APA Handbook of Research Methods in Psychology, Vol 2: Research Designs: Quantitative, Qualitative, Neuropsychological, and Biological (eds Cooper, H. et al.) 57–71 (American Psychological Association, 2012).

  34. Dopp, A. R., Parisi, K. E., Munson, S. A. & Lyon, A. R. A glossary of user-centered design strategies for implementation experts. Transl. Behav. Med. 9, 1057–1064 (2019).

    Article  PubMed  Google Scholar 

  35. Orlowski, S. K. et al. Participatory research as one piece of the puzzle: a systematic review of consumer involvement in design of technology-based youth mental health and well-being interventions. JMIR Hum. Factors 2, e12 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  36. Orji, R. & Moffatt, K. Persuasive technology for health and wellness: state-of-the-art and emerging trends. Health Inform. J. 24, 66–91 (2018).

    Article  Google Scholar 

  37. Esfandiari, N. et al. A specific internet-based cognitive behavior therapy for adolescents with social anxiety disorder: three-armed randomized control trial. Preprint at Research Square https://doi.org/10.21203/rs.3.rs-2123795/v1 (2022).

  38. Yardley, L., Bradbury, K. & Morrison, L. in Qualitative Research in Psychology: Expanding Perspectives in Methodology and Design 2nd edn (ed. Camic, P. M.) 263–282 (American Psychological Association, 2021).

  39. Bailey, E. et al. Ethical issues and practical barriers in internet-based suicide prevention research: a review and investigator survey. BMC Med. Ethics 21, 37 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  40. Sander, L. et al. Suicide risk management in research on internet-based interventions for depression: a synthesis of the current state and recommendations for future research. J. Affect. Disord. 263, 676–683 (2020).

    Article  PubMed  Google Scholar 

  41. Kaurin, A., Dombrovski, A. Y., Hallquist, M. N. & Wright, A. G. C. Integrating a functional view on suicide risk into idiographic statistical models. Behav. Res. Ther. 150, 104012 (2022).

    Article  PubMed  Google Scholar 

  42. Friedlander, A., Nazem, S., Fiske, A., Nadorff, M. R. & Smith, M. D. Self-concealment and suicidal behaviors. Suicide Life Threat Behav. 42, 332–340 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  43. Franklin, J. C. et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychol. Bull. 143, 187–232 (2017).

    Article  PubMed  Google Scholar 

  44. Steeg, S. et al. Accuracy of risk scales for predicting repeat self-harm and suicide: a multicentre, population-level cohort study using routine clinical data. BMC Psychiatry 18, 113 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  45. Sisti, D. A. & Joffe, S. Implications of zero suicide for suicide prevention research. JAMA 320, 1633–1634 (2018).

    Article  PubMed  Google Scholar 

  46. Büscher, R. et al. Digital cognitive-behavioural therapy to reduce suicidal ideation and behaviours: a systematic review and meta-analysis of individual participant data. Evid. Based Ment. Health 25, e8–e17 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Holmes, E. A. et al. The Lancet Psychiatry Commission on psychological treatments research in tomorrow’s science. Lancet Psychiatry 5, 237–286 (2018).

    Article  PubMed  Google Scholar 

  48. Torok, M. et al. Suicide prevention using self-guided digital interventions: a systematic review and meta-analysis of randomised controlled trials. Lancet Digit. Health 2, e25–e36 (2020).

    Article  PubMed  Google Scholar 

  49. Ferreira, T. E-health applications and data protection: a comparison of selected European Union members’ national legal systems. Bioethica 8, 74–84 (2022).

    Article  Google Scholar 

  50. Wilkowska, W. & Ziefle, M. Privacy and data security in E-health: requirements from the user’s perspective. Health Inform. J. 18, 191–201 (2012).

    Article  Google Scholar 

  51. Albrecht, U.-V. Chances and Risks of Mobile Health Apps (CHARISHMA) (Medizinische Hochschule Hannover, 2016).

  52. Lablans, M., Borg, A. & Ückert, F. A RESTful interface to pseudonymization services in modern web applications. BMC Med. Inf. Decis. Making 15, 2 (2015).

    Article  Google Scholar 

  53. Cummins, N., Schuller, B. W. & Baird, A. Speech analysis for health: current state-of-the-art and the increasing impact of deep learning. Methods 151, 41–54 (2018).

    Article  PubMed  Google Scholar 

  54. Ghosh, S., Löchner, J., Mitra, B. & De, P. in Quantifying Quality of Life (eds Wac, K. & Wulfovich, S.) 209–267 (Springer, 2022).

  55. Garriga, R. et al. Machine learning model to predict mental health crises from electronic health records. Nat. Med. 28, 1240–1248 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  56. Valstar, M. et al. AVEC 2013: the continuous audio/visual emotion and depression recognition challenge. In Proc. 3rd ACM International Workshop on Audio/Visual Emotion Challenge 3–10 (ACM, 2013).

  57. Wang, R. et al. Tracking depression dynamics in college students using mobile phone and wearable sensing. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 1–26 (2018).

    Google Scholar 

  58. Dhall, A., Goecke, R., Gedeon, T. & Sebe, N. Emotion recognition in the wild. J. Multimodal User Interf. 10, 95–97 (2016).

    Article  Google Scholar 

  59. Amin, M. M., Cambria, E. & Schuller, B. W. Will affective computing emerge from foundation models and general AI? A first evaluation on ChatGPT. IEEE Intelligent Systems 2, 15–23 (2023).

    Article  Google Scholar 

  60. Schuller, B. W. et al. Computational charisma—a brick by brick blueprint for building charismatic artificial intelligence. Preprint at https://doi.org/10.48550/arXiv.2301.00142 (2023).

  61. Véliz, C. Chatbots shouldn’t use emojis. Nature 615, 375 (2023).

    Article  PubMed  Google Scholar 

  62. Vazire, S. Who knows what about a person? The self-other knowledge asymmetry (SOKA) model. J. Person. Soc. Psychol. 98, 281–300 (2010).

    Article  Google Scholar 

  63. Garatva, P. et al. in Digital Phenotyping and Mobile Sensing (eds Montag, C. & Baumeister, H.) 395–411 (Springer, 2023).

  64. Torous, J., Kiang, M. V., Lorme, J. & Onnela, J.-P. New tools for new research in psychiatry: a scalable and customizable platform to empower data driven smartphone research. JMIR Ment. Health 3, e16 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  65. Cornet, V. P. & Holden, R. J. Systematic review of smartphone-based passive sensing for health and wellbeing. J. Biomed. Inform. 77, 120–132 (2018).

    Article  PubMed  Google Scholar 

  66. Moshe, I. et al. Predicting symptoms of depression and anxiety using smartphone and wearable data. Front. Psychiatry 12, 625247 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  67. Vasudevan, S., Saha, A., Tarver, M. E. & Patel, B. Digital biomarkers: convergence of digital health technologies and biomarkers. npj Digit. Med. 5, 36 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  68. Asare, O. K. et al. Predicting depression from smartphone behavioral markers using machine learning methods, hyperparameter optimization, and feature importance analysis: exploratory study. JMIR mHealth uHealth 9, e26540 (2021).

    Article  Google Scholar 

  69. Fried, E. I., Rieble, C. & Proppert, R. K. K. Building an early warning system for depression: rationale, objectives, and methods of the WARN-D study. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/9qcvs (2022).

  70. Lattie, E. G. et al. Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review. J. Med. Internet Res. 21, e12869 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  71. Torous, J. et al. Smartphones, sensors, and machine learning to advance real-time prediction and interventions for suicide prevention: a review of current progress and next steps. Curr. Psychiatry Rep. 20, 51 (2018).

    Article  PubMed  Google Scholar 

  72. Kargl, F., van der Heijden, R. W., Erb, B. & Bösch, C. in Digital Phenotyping and Mobile Sensing (eds Montag. C. & Baumeister, H.) 13–23 (Springer, 2023).

  73. Larsen, M. E. et al. Using science to sell apps: evaluation of mental health app store quality claims. npj Digit. Med. 2, 18 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  74. Klein, R. A. et al. Investigating variation in replicability. Soc. Psychol. 45, 142–152 (2014).

    Article  Google Scholar 

  75. Simblett, S., Birch, J., Matcham, F., Yaguez, L. & Morris, R. A systematic review and meta-analysis of e-mental health interventions to treat symptoms of posttraumatic stress. JMIR Ment. Health 4, e14 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  76. Magnusson, K., Andersson, G. & Carlbring, P. The consequences of ignoring therapist effects in trials with longitudinal data: a simulation study. J. Consult. Clin. Psychol. 86, 711–725 (2018).

    Article  PubMed  Google Scholar 

  77. Johns, R. G., Barkham, M., Kellett, S. & Saxon, D. A systematic review of therapist effects: a critical narrative update and refinement to review. Clin. Psychol. Rev. 67, 78–93 (2019).

    Article  PubMed  Google Scholar 

  78. Rozental, A., Andersson, G. & Carlbring, P. In the absence of effects: an individual patient data meta-analysis of non-response and its predictors in internet-based cognitive behavior therapy. Front. Psychol. 10, 589 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  79. Mander, J. et al. The therapeutic alliance in different mental disorders: a comparison of patients with depression, somatoform, and eating disorders. Psychol. Psychother. 90, 649–667 (2017).

    Article  PubMed  Google Scholar 

  80. Mechler, J. et al. Sudden gains and large intersession improvements in internet-based psychodynamic treatment (IPDT) for depressed adolescents. Psychother. Res. 31, 455–467 (2021).

    Article  PubMed  Google Scholar 

  81. von Klipstein, L., Riese, H., van der Veen, D. C., Servaas, M. N. & Schoevers, R. A. Using person-specific networks in psychotherapy: challenges, limitations, and how we could use them anyway. BMC Med. 18, 345 (2020).

    Article  Google Scholar 

  82. van den Bergh, R. et al. The content of personalised network-based case formulations. Preprint at PsyArXiv https://doi.org/10.31234/osf.io/yan4k (2022).

  83. Olthof, M. et al. Destabilization in self-ratings of the psychotherapeutic process is associated with better treatment outcome in patients with mood disorders. Psychother. Res. 30, 520–531 (2020).

    Article  PubMed  Google Scholar 

  84. Schiepek, G. et al. Real-time monitoring of psychotherapeutic processes: concept and compliance. Front. Psychol. 7, 604 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  85. Hasselman, F. & Bosman, A. M. T. Studying complex adaptive systems with internal states: a recurrence network approach to the analysis of multivariate time-series data representing self-reports of human experience. Front. Appl. Math. Stat. https://doi.org/10.3389/fams.2020.00009 (2020).

  86. Wallot, S. Recurrence quantification analysis of processes and products of discourse: a tutorial in R. Discourse Process. 54, 382–405 (2017).

    Article  Google Scholar 

  87. Myin-Germeys, I. & Kuppens, P. The Open Handbook of Sampling Methodology. A Step-by-Step Guide to Designing, Conducting, and Analyzing ESM studies (The Center for Research on Experience Sampling and Ambulatory Methods Leuven, 2021).

  88. Trull, T. J. & Ebner-Priemer, U. Ambulatory assessment. Annu. Rev. Clin. Psychol. 9, 151–176 (2013).

    Article  PubMed  Google Scholar 

  89. Insel, T. R. Digital phenotyping: a global tool for psychiatry. World Psychiatry 17, 276–277 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  90. Ebner-Priemer, U. W. et al. Digital phenotyping: towards replicable findings with comprehensive assessments and integrative models in bipolar disorders. Int. J. Bipolar Disord. 8, 35 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  91. Myin-Germeys, I. et al. Experience sampling methodology in mental health research: new insights and technical developments. World Psychiatry 17, 123–132 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  92. Fortea, L. et al. Development and validation of a smartphone-based app for the longitudinal assessment of anxiety in daily life. Assessment https://doi.org/10.1177/10731911211065166 (2021).

  93. Wrzus, C. & Neubauer, A. B. Ecological momentary assessment: a meta-analysis on designs, samples, and compliance across research fields. Assessment https://doi.org/10.1177/10731911211067538 (2022).

  94. Geldhof, G. J., Preacher, K. J. & Zyphur, M. J. Reliability estimation in a multilevel confirmatory factor analysis framework. Psychol. Methods 19, 72–91 (2014).

    Article  PubMed  Google Scholar 

  95. Kockler, T. D., Santangelo, P. S. & Ebner-Priemer, U. W. Investigating binge eating using ecological momentary assessment: the importance of an appropriate sampling frequency. Nutrients 10, 105 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  96. Ottenstein, C. & Werner, L. Compliance in ambulatory assessment studies: investigating study and sample characteristics as predictors. Assessment https://doi.org/10.1177/10731911211032718 (2021).

  97. Lecomte, T. et al. Mobile apps for mental health issues: meta-review of meta-analyses. JMIR mHealth uHealth 8, e17458 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  98. Linardon, J., Cuijpers, P., Carlbring, P., Messer, M. & Fuller-Tyszkiewicz, M. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry 18, 325–336 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  99. Fleming, T. et al. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J. Med. Internet Res. 20, e199 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  100. Torous, J. et al. The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry 20, 318–335 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  101. Chien, I. et al. A machine learning approach to understanding patterns of engagement with internet-delivered mental health interventions. JAMA Netw. Open 3, e2010791 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  102. Torous, J., Lipschitz, J., Ng, M. & Firth, J. Dropout rates in clinical trials of smartphone apps for depressive symptoms: a systematic review and meta-analysis. J. Affect. Disord. 263, 413–419 (2020).

    Article  PubMed  Google Scholar 

  103. Richards, D. & Richardson, T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin. Psychol. Rev. 32, 329–342 (2012).

    Article  PubMed  Google Scholar 

  104. Alon, N., Stern, A. D. & Torous, J. Assessing the Food and Drug Administration’s risk-based framework for software precertification with top health apps in the United States: quality improvement study. JMIR mHealth uHealth 8, e20482 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  105. Stern, A. D. et al. Advancing digital health applications: priorities for innovation in real-world evidence generation. Lancet Digit. Health 4, e200–e206 (2022).

    Article  PubMed  Google Scholar 

  106. Lagan, S. et al. Mental health app evaluation: updating the American Psychiatric Association’s framework through a stakeholder-engaged workshop. Psychiatr. Serv. 72, 1095–1098 (2021).

    Article  PubMed  Google Scholar 

  107. Stoyanov, S. R. et al. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth uHealth 3, e27 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  108. Ramos, G., Ponting, C., Labao, J. P. & Sobowale, K. Considerations of diversity, equity, and inclusion in mental health apps: a scoping review of evaluation frameworks. Behav. Res. Ther. 147, 103990 (2021).

    Article  PubMed  Google Scholar 

  109. Carlo, A. D., Hosseini Ghomi, R., Renn, B. N. & Areán, P. A. By the numbers: ratings and utilization of behavioral health mobile applications. npj Digit. Med. 2, 54 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  110. Lagan, S. et al. Actionable health app evaluation: translating expert frameworks into objective metrics. npj Digit. Med. 3, 100 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  111. Szinay, D. et al. Influences on the uptake of health and well-being apps and curated app portals: think-aloud and interview study. JMIR mHealth uHealth 9, e27173 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  112. Roberts, A. E. et al. Evaluating the quality and safety of health-related apps and e-tools: adapting the Mobile App Rating Scale and developing a quality assurance protocol. Internet Interv. 24, 100379 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  113. Huber, M. et al. How should we define health? BMJ 343, d4163 (2011).

    Article  PubMed  Google Scholar 

  114. Wasil, A. R., Gillespie, S., Shingleton, R., Wilks, C. R. & Weisz, J. R. Examining the reach of smartphone apps for depression and anxiety. Am. J. Psychiatry 177, 464–465 (2020).

    Article  PubMed  Google Scholar 

  115. Gentili, A. et al. The cost-effectiveness of digital health interventions: a systematic review of the literature. Front. Public Health 10, 787135 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  116. Mitchell, L. M., Joshi, U., Patel, V., Lu, C. & Naslund, J. A. Economic evaluations of internet-based psychological interventions for anxiety disorders and depression: a systematic review. J. Affect. Disord. 284, 157–182 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  117. Wac, K. in Digital Health (eds Rivas, H. & Wac, K.) 83–108 (Springer, 2018).

Download references

Author information

Author notes

  1. These authors contributed equally: Caroline Seiferth, Lea Vogel.

  2. A full list of members and their affiliations appears in the Supplementary Information.

Authors and Affiliations

  1. Institute of Psychology, University of Bamberg, Bamberg, Germany

    Caroline Seiferth

  2. National Centre for Early Prevention, German Youth Institute, Munich, Germany

    Lea Vogel & Ansgar Opitz

  3. Department of Psychology, LMU Munich, Munich, Germany

    Lea Vogel & Benjamin Aas

  4. sysTelios Think Tank, sysTelios Health Center, Siedelsbrunn, Germany

    Benjamin Aas

  5. Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital of Psychiatry and Psychotherapy, Tuebingen, Germany

    Isabel Brandhorst, Annette Conzelmann, Marlene Finkbeiner, Karsten Hollmann, Heinrich Lautenbacher, Tobias J. Renner & Johanna Löchner

  6. DZPG (German Center of Mental Health), Tuebingen, Germany

    Isabel Brandhorst, Annette Conzelmann, Marlene Finkbeiner, Karsten Hollmann, Heinrich Lautenbacher, Tobias J. Renner & Johanna Löchner

  7. Department of Psychology, Stockholm University, Stockholm, Sweden

    Per Carlbring

  8. PFH – Private University of Applied Sciences, Department of Psychology (Clinical Psychology II), Goettingen, Germany

    Annette Conzelmann

  9. Department of Psychology, Faculty of Education and Psychology, Shahid Beheshti University, Tehran, Iran

    Narges Esfandiari

  10. Medical Faculty, University of Tuebingen, Tuebingen, Germany

    Heinrich Lautenbacher

  11. Department of Education and Health Research, Institute of Sport Science, University of Tuebingen, Tuebingen, Germany

    Edith Meinzinger & Sebastian Wolf

  12. Sir Henry Wellcome Building for Mood Disorders Research, College of Life and Environmental Sciences, University of Exeter, Exeter, UK

    Alexandra Newbold

  13. Medical Psychology and Medical Sociology, Faculty of Medicine, University of Freiburg, Freiburg, Germany

    Lasse Bosse Sander

  14. Department of Behavioural and Cognitive Sciences, University of Luxembourg, Esch-sur-Alzette, Luxembourg

    Philip S. Santangelo

  15. LMU Munich, Munich, Germany

    Ramona Schoedel

  16. GLAM, Imperial College, London, UK

    Björn Schuller

  17. Institute of Behavioral Science and Technology, University of St. Gallen, St. Gallen, Switzerland

    Clemens Stachl

  18. Department of Clinical Psychology and Psychotherapy, University of Ulm, Ulm, Germany

    Yannik Terhorst

  19. Beth Israel Deaconess Medical Center, Department of Psychiatry, Harvard Medical School, Boston, MA, USA

    John Torous

  20. Quality of Life Lab, Geneva School of Economics and Management, University of Geneva, Geneva, Switzerland

    Katarzyna Wac

  21. Black Dog Institute, UNSW, Sydney, New South Wales, Australia

    Aliza Werner-Seidler

Authors
  1. Caroline Seiferth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lea Vogel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benjamin Aas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Isabel Brandhorst
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Per Carlbring
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Annette Conzelmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Narges Esfandiari
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Marlene Finkbeiner
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Karsten Hollmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Heinrich Lautenbacher
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Edith Meinzinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Alexandra Newbold
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Ansgar Opitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Tobias J. Renner
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Lasse Bosse Sander
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Philip S. Santangelo
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Ramona Schoedel
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Björn Schuller
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Clemens Stachl
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Yannik Terhorst
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. John Torous
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Katarzyna Wac
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Aliza Werner-Seidler
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Sebastian Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Johanna Löchner
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

sysTelios Think Tank

  • Benjamin Aas

Contributions

J.L., C. Seiferth and L.V. conceptualized the study and supervised the writing process. J.L., L.V. and C. Seiferth wrote the first draft. The sections were written in the following writing groups: ‘Where to start’: K.H., I.B., J.L. and C. Seiferth; ‘Intervention content development’: C. Seiferth and J.L.; ‘UCD and participatory approaches’: L.V., L.B.S., A.W.-S. and J.L.; ‘Managing suicidality’: L.B.S. and K.H.; ‘Data protection and data security’: H.L., I.B. and A.C.; ‘AI in assessment and intervention’: B.S. and J.L.; ‘Sensing and wearables’: K.W., Y.T., R.S. and C. Stachl; ‘Efficacy evaluation, RCTs and other methods’: A.O., B.A., S.T.T. and A.C.; ‘EMA’: P.S.S. reand M.F.; ‘Transfer into (clinical) practice’: S.W., B.A., E.M. and SysTelios Think Tank; ‘AEFs’: J.T. All authors commented on the first and final draft. P.C., T.J.R., A.N. and N.E. reviewed the final version particularly. All authors share responsibility for the final version of the paper.

Corresponding author

Correspondence to Johanna Löchner.

Ethics declarations

Competing interests

P.C. has received speaker fees from Angelini Pharma, Lundbeck and Koa Health within the past 3 years. J.T. is a scientific advisor for precision mental wellness. L.B.S. reported receiving personal fees from Psychotherapy Training Institutes, Health Insurances and Clinic Providers in the context of e-mental health but outside the submitted work. The other authors declare no competing interests.

Peer review

Peer review information

Nature Mental Health thanks Ana Catarino and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Table 1

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Seiferth, C., Vogel, L., Aas, B. et al. How to e-mental health: a guideline for researchers and practitioners using digital technology in the context of mental health. Nat. Mental Health 1, 542–554 (2023). https://doi.org/10.1038/s44220-023-00085-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s44220-023-00085-1

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing