Introduction

In the UK, 25% of 17–19 year-olds experience significant levels of mental illness often accompanied by self-harm and sometimes escalating to suicide1. As 50% of mental health problems are established by the age of 14 and 75% by the age of 242, prevention and early intervention in children and young people are of critical importance3. Preventive interventions include those that support children and young people to develop skills to maintain mental health, that target pre-clinical risk factors, or respond to early signs of distress4. This review will apply the Gordon framework of prevention to mental health problems: universal (targeting whole populations, regardless of current mental health status), selective (targeting specified risk factors) and indicated levels (targeting early sub-clinical signs and symptoms)5. Tertiary prevention, targeting existing mental health disorders, is not within the scope of this review.

The rapid growth of digital technologies (e.g. smartphones, wearables) has created the potential for predictive prevention: the use of data to personalise preventive interventions6. The ubiquity of digital technologies offers an opportunity to support increased access to mental health interventions for children and young people7. To date, there is little evidence to demonstrate the successful implementation and subsequent impact of evidence-based digital mental health interventions for children and young people at scale8. This scoping review aims to further our understanding of the challenges to implementation faced by digital health interventions (DHIs) addressing the prevention and early intervention of mental health problems in children and young people by examining the reporting of factors that improve opportunities for successful adoption into real-world contexts. These include features related to the development, evaluation and implementation of DHIs9,10.

Previous studies have identified several challenges to the use of DHIs in routine service delivery including technical difficulties and low awareness of data standards and privacy11, as well as low engagement and retention rates amongst users12. Moreover, several gaps have been identified in research such as the lack of economic evaluations and implementation studies11,13. A scoping review of mental health apps for young people documented several advantages of apps including ubiquity, flexibility, and timely communication, but these were challenged by technical difficulties, poor adherence, and few studies that addressed privacy or conducted an economic evaluation14. The large numbers of publicly available digital mental health interventions (over 10,0008) highlight the growing gap between research and evaluation as well as practice and implementation. For instance, a recent review found only 2 of 15 evidence-based mental health apps were available to download despite their acceptability15 and the clear need for effective DHIs in routine care16.

Efficacious DHIs for mental health have been reviewed across the age-span of children and young people7, including university students17,18, and in relation to those with anxiety disorders19 and/or depression17,19,20,21,22,23,24. However, generalising efficacy of DHIs outside research settings is constrained if interventions are not sufficiently appropriate or appealing, challenging not only the engagement of children and young people25 but potentially also those who support them (e.g. parents and teachers). Engaging stakeholders in the development, implementation and evaluation of technologies is one of the pillars for responsible research and innovation (RRI) and a crucial element to develop new digital innovations in a socially desirable and acceptable way26. For example, researchers must consider the wider societal implications (e.g., workforce issues, training/skills) as well as consider how real-world uptake might differ from usage within a trial (e.g., limited mobile data) as engagement is a key barrier to the effectiveness of DHIs in mental healthcare8. Co-production with young people, and where relevant their parents25, as well as specialist technical and psychological input20, is important in identifying and potentially mitigating these problems.

There is clearly potential for DHIs to be effective in prevention and early intervention for children and young people but there is little research addressing the opportunities and challenges of their adoption into real-world contexts. This review examines factors related to successfully sustained DHIs9,10 and applies these to evidence-based interventions designed to prevent mental health disorders in children and young people. These include mapping the characteristics of studies, their participants, design elements, and features related to implementation. It aims to identify potential influences of successful adoption within the existing evidence-base and highlights factors that should be addressed by researchers in the design and reporting of the development, evaluation and implementation of DHIs. A scoping review approach was chosen as it is more suitable than a systematic review where the purpose of the review is to identify knowledge gaps, scope a body of literature to investigate the adequacy of research design and reporting27.

Methods and analysis

This scoping review uses the framework proposed by Arksey and O’Malley28 and is informed by PRISMA guidelines29.

Search strategy

Five electronic databases (ACMDL, PubMed, PsycInfo, Embase and Cochrane Library) were searched in June 2019 for all full-text publications in English published from 2013 to 19/06/2019. Reviews were searched when identified and any relevant articles included. A previous systematic review by Clarke et al.30 reported on 28 studies conducted between 2000 and early 2013. These studies were followed up so as to map whether these interventions had been subject to further research.

Study screening

Eligibility criteria were as follows:

  • peer-reviewed studies

  • DHIs for mental health disorders

  • children and young people aged 0–25 years old

  • universal, selective or indicated prevention

  • participants were not included in studies on the basis of a clinical diagnosis (e.g. depression or anxiety)

  • DHI delivered in any location

Screening was conducted by four researchers (AB, EBD, EPV and MK).

Data Extraction

The data extraction chart was developed iteratively using the Joanna Briggs Institute template31 by three researchers (AB, EPV and EBD), with input from other authors, to reflect categories identified within the previous review30 and the literature surrounding digital therapies and prevention for children and young people. Thirty-four categories were identified ranging from demographic information to key elements of prevention, implementation and involvement. Three researchers co-ordinated to ensure that the coding framework was appropriate and consistently applied. Four main categories were identified – study characteristics, user characteristics, usability and engagement, and implementation – from the data extraction chart.

Results

Study selection

Thirty studies were identified from 791 studies (Fig. 1).

Fig. 1
figure 1

PRISMA flow diagram outlining the search process and the exclusion of ineligible articles.

Full size image

Study characteristics

The 30 studies included relate to 21 different interventions. These include 12 providing universal prevention (Table 1), 3 selective (Table 2) and 15 indicated prevention (Table 3). The majority of studies reported on interventions accessed via the internet (n = 21) while a small number used instant messaging (n = 2) or social media elements (n = 1). Some used an app (universal = 2, selective = 1), whilst those available only on a desktop or laptop computer (n = 8) mainly used game elements (n = 5).

Table 1 Study characteristics for universal prevention studies.
Full size table
Table 2 Study characteristics for selective prevention studies.
Full size table
Table 3 Study characteristics for indicated prevention studies.
Full size table

The majority of participants were recruited from secondary education (n = 19; universal = 9, selective = 1, and indicated = 9). This was followed by primary healthcare (n = 4; selective = 1, and indicated = 3), universities (n = 3; universal = 2, and indicated = 1) and via the media (n = 3; universal = 1, and indicated = 2). One selective study identified potential participants through court documents.

User characteristics

The majority of studies did not report participant characteristics that are known risk-factors for mental health disorders (n = 16). These included living status, education of participants and/or parents, technology ownership, and rural-urban classification (Tables 4–6). Selective studies identified those with parents who were divorced, or had addictions or mental illness, and those at risk of depression. Just under half of the studies did not report information about participants’ ethnicity (n = 15). Those that did were inconsistent in how this was measured, with seven simply identifying ‘local’ and ‘non-local’ participants.

Table 4 Individual characteristics, usability and engagement, and implementation features within reported universal prevention studies.
Full size table
Table 5 Individual characteristics, usability and engagement, and implementation features reported within selective prevention studies.
Full size table
Table 6 Individual characteristics, usability and engagement, and implementation features reported within indicated prevention studies.
Full size table

Universal interventions were aimed at young people whose ages ranged from 9 to 25 years old (mean = 17.35, SD = 3.97) whilst selective interventions were for between 11 and 25 years old (mean = 17.47, SD = 3.75). Indicated interventions included the broadest age range from 7 to 25 years old (mean = 16.15, SD = 3.27). The majority of interventions targeted participants between the ages of 15–16 years (n = 22, mean = 16.73, SD = 3.59). No interventions were aimed at children aged ≤6 years. Universal and selective primary prevention interventions were also not available to children aged ≤9 years.

Usability and engagement

Most studies reported capturing user experience through questionnaires (n = 9) that range from a series of questions around helpfulness and acceptability, to a single question asking if the user is satisfied. Only one study reported using multiple validated measures—the System Usability Scale and the Usefulness, Satisfaction and Ease of use (USE) questionnaire32. Other methods used included interviews (n = 3), focus groups (n = 2) and surveys (n = 1). Only six interventions reported on user involvement in the development of interventions. This included collaborative working on the content and design of the intervention with young people and other users, involvement in usability testing, and engagement of young people with lived experience to feedback on materials.

Only 8 studies identified completion criteria. Some used automated methods, such as the number of logins, to assess engagement (n = 7). Others used self-report (n = 2) or relied on attendance (e.g. interventions delivered within schools). Because most interventions did not report on how much of an intervention needed to be completed, it was difficult to identify whether participants have been sufficiently engaged. Reasons provided for dropout are linked to setting (e.g. school absence) and technical issues (e.g. logging in difficulties). Other factors included older age, higher and lower levels of anxiety, speaking a language other than English, feeling better or too unwell, being busy, or responding badly to the intervention.

Implementation

Most universal studies (n = 11/12) recruited all those that chose to participate who were within the age range. Only one study screened participants using a clinical interview for depression within the general school population, for the purpose of excluding those in the clinical range and identifying changes within a clinical score. Universal studies were delivered at home (n = 7), within schools (n = 4), and a lab (n = 1). Selective studies targeted specific at-risk populations (e.g. children of divorced parents) and were all delivered within a home setting (n = 3). Six of the fifteen indicated studies used clinical assessments to identify participants and these interventions were delivered in school (n = 1), multiple settings (n = 3) and home (n = 2). Nine used self-reported assessments and were delivered within the home (n = 5), multiple settings (n = 3) and schools (n = 1).

All indicated (n = 15) and one selective study included participants with a score indicating mental health symptomology. Studies variously excluded those with a mental health diagnosis, low levels of comprehension, those currently in treatment, those with scores that were too high or low on mental health symptomology, those not exclusively attracted to the opposite sex, and those with current suicidality.

The barriers and facilitators to implementation were different for those programmes delivered within schools, at home and across multiple settings. Within schools, completion was often challenged by absences. Relevance was a key issue across all settings with involved stakeholders needing to understand the programme and its outcomes, and feel that it was relevant to their experience. It was also necessary for the preventive DHI to be provided through an easily accessible and appropriate device. For instance, within schools it may be more difficult to implement app-based interventions due to restrictions on students’ smartphone use or it may create inequalities for those students without access to one. Technical issues across all settings challenged use; this was overcome when support was provided by researchers to those delivering (e.g. teachers) the intervention. Much of the data focused on implementation was around the acceptability, usability and content of the programme, rather than challenges and opportunities in real-world dissemination. Consent was challenging in two studies where parental consent was needed for those disclosing their sexuality, and another, delivered remotely, that noted some participants did not realise they were engaged in a research study. Despite the potential for DHIs to be delivered without clinical oversight only a small number of studies (n = 3) reported adverse events or potential negative impact in detail.

The research was funded by fellowships, granted to researchers, and national health research grants. Funding was also received from accelerator programmes, charitable organisations, and was mainly from research, private and health funding with a small amount of business funding. Some funding was received for developing the intervention, and others received funding for delivery of the research, but it was mainly unclear what this funding was provided for. Although the reports include statements that there were no conflicts of interest, the founders and developers of interventions were at times involved in the research.

Only six studies reported the cost of their intervention within the study. The current availability of interventions was difficult to assess as many were difficult to search because they had names associated with other websites (e.g. ‘stressbusters’) or were not named within the research. Only 38% (n = 8/21) are publicly available. Of those accessible online some were only available in certain countries (e.g. SPARX is currently not available in the United Kingdom) or specific languages (e.g. Kopstoring is only available in Dutch). Intervention costs were at times reported online (e.g. MoodGYM) but it was difficult to ascertain how to access these interventions as an individual because they were often country-specific or required a significant amount of personal information (e.g. e-Couch).

Discussion

DHIs offer the opportunity to provide children and young people with evidence-based interventions that prevent mental health disorders at an early stage. However, our scoping review suggests that DHIs are not yet meeting their potential and the design and reporting of research does not generally support real-world implementation. Whilst they are being delivered within several settings, studies most frequently recruited from secondary schools, colleges or universities. Younger children and those who do not engage in school or are often absent are not being included within research designs. Other known mental health risk-factors are not reported such as socio-economic status and ethnicity, and this limits the generalisability of the findings to populations where there are most need and potential benefit. This is particularly significant as ethnic minorities are less likely to be referred to or access formal mental health services33,34,35. Reporting participants’ demographic characteristics are essential to know if research is engaging with groups hardly reached (e.g., sexual and gender minorities) which often are at increased risk for mental health problems, including suicide36. Demographic information can also signal if equality, diversity and inclusion strategies have been applied to understand how results may generalize to different groups; thereby supporting adoption and implementation.

DHIs are designed mainly to deliver cognitive behavioural therapy (CBT) for indicated prevention to target depression and/or anxiety symptoms. This is unsurprising considering CBT is the recommended treatment for depression37 and anxiety38 in children and young people and the most studied intervention offline for anxiety and depression39. Fewer interventions aimed to improve general wellbeing, which suggests researchers may be more likely to develop interventions based on existing clinical guidelines despite the effectiveness of tackling general wellbeing in non-digital prevention interventions24.

Preventive DHIs can be flexible in how they are offered; whether in terms of when, where or how they are accessed. However, different settings may offer unique challenges and opportunities that prevent individuals or those delivering from benefiting. Our review found that few studies reported on factors related to implementation and this represents an important gap in understanding how DHIs are best adopted to support the prevention of mental health disorders in children and young people. Our findings suggest there are two key areas that must be addressed: systemic understanding of how technical issues can be solved/supported and ensuring that the programme content is not only relevant to children and young people but also to those supporting its use (e.g. parents and teachers). It is also important to consider the real-world contexts into which these DHIs might be implemented. For instance, when delivered remotely it is not possible to apply consistent eligibility criteria and yet only three studies address adverse events or the negative impact of the DHI. Co-production of content, design and usability can increase the likelihood of successful implementation within DHIs and RRI frameworks recommend user involvement in the design of new technologies. However, within this review, only 10 interventions reported on user experience and five interventions reported that users had been involved in the research. It is clear that there is a significant lack of consensus as to how the user experience should be captured or involvement reported.

Regarding sustainability, there is a need to design, develop and test interventions within an implementation framework; i.e. with the pipeline of adoption firmly sitting as the foundation of the work. Real-world accessibility of interventions was difficult to ascertain as many are only available in specific countries or languages and the costs of access are not always clear. While these DHIs hold great potential to be disseminated and used widely, our review indicates that too often software is not updated, and existing interventions do not take advantage of newer developments that have the potential to improve the predictive capacity of preventive interventions, such as sensors and wearables, machine learning or natural language processing methodologies40.

Another issue highlighted is that programme completion and reasons for dropout need to be addressed more clearly as it is difficult to assess how many modules or how engaged a user needs to be for it to be considered complete. Outcomes are potentially impacted by the type of technology used and the way in which data are collected. Many DHIs still rely on self-report, which is less objective and considered scientifically less rigorous, despite the potential for automated measurements. One study demonstrated high rates of follow up (MEMO—98% post-intervention and 92.5% at 12 months) that were attributed to strategies they developed to reduce dropout41. Another (MoodGYM, delivered nationally) had extremely low rates of engagement (8.5% logged into the first module)42. However, this may be due to differences in how this was measured and the method of delivery. The latter was done so automatically through logging access to the website whilst the former relied on self-report. Whether either can accurately capture engagement remains to be seen.

Strengths and limitations

This scoping review examined 30 studies representing 21 different DHIs for the prevention of mental health disorders amongst children and young people delivered in various settings. Our results provide an updated summary of factors related to the adoption of interventions into real-world contexts, reporting on stakeholder involvement, engagement and users’ experience. We have also expanded on past reviews by identifying the potential for these prevention interventions to be disseminated and used widely. This review has identified tools currently used to address user experience and engagement within these interventions and highlighted the gaps in the design and reporting of research (e.g., reporting risk factors, gaps in age range covered by interventions). We have mapped barriers and facilitators to real-world implementation including differences between research and practice (e.g., exclusion criteria, implementation, funding and costs). The findings of this review highlight that there is more work needed to address better research design and reporting of development, evaluation and implementation of DHIs for the prevention of mental health disorders in children and young people if their potential is to be fully realised.

The strength of this scoping review is that it looked broadly at mental health prevention across multiple settings, including user experience and involvement, implementation or the real-world uptake of interventions. However, it did not include grey literature and there may have been co-design and involvement within studies that went unreported in publications. An agreed DHI taxonomy would be beneficial to identify common core components between interventions. Their clinical, technical and evaluative mechanisms are reported in several ways, which challenges this reviews clarity.

Recommendations

More research is needed examining factors related to the successful adoption of preventive DHIs for children and young people within mental health and how they can be encouraged; addressing risk factors, ethical issues including consent processes for remote delivery, and younger age groups. We recommend that researchers report the amount of their programme that must be completed (minimum dose) and identify the availability and accessibility metrics of their intervention including costs (e.g. through an economic evaluation). Real-world implementation is imperative to consider8, and more research should address this within different settings and technologies. DHI research would also benefit from an agreed taxonomy for reporting the clinical, technical and evaluative components. The impact of these interventions, including negative reactions or the exclusion of certain populations, must also be addressed in future research. Finally, we recommend that funding is provided that ensures the sustainability of research-based DHIs from development through to real-world dissemination.