Main

Although common mental health conditions such as depression and anxiety entail intense emotional distress and have a substantial impact on social and occupational functioning, a striking proportion of these conditions remain undiagnosed or untreated1,2. The estimated costs of mental health disorders are high: across Europe in 2010, mood disorders cost €113.4 billion (£97.3 billion), of which around 37% was attributed to direct medical costs3,4. Further costs are incurred through the use of other healthcare services: people suffering from depression and anxiety are more likely to use primary and secondary healthcare services, such as general practitioners and other community provisions5,6. These costs vary by severity and are estimated to be above £750 per person treated for outpatient care7. There is well-established literature on the global economic cost of mental health disorders and the health economic value of mental healthcare provision8,9,10,11,12. However, although there is a wide range of treatment options available for these conditions, from pharmacotherapy to psychotherapy, and a combination of both8,9,10,11, we lack studies comparing the cost-effectiveness of different treatment interventions. For example, a recent systematic review of economic evaluations for internet-based mental health interventions noted that most are compared to wait-list control groups rather than to other types of intervention12. Furthermore, the way variables such as treatment costs, clinical effectiveness, waiting times, treatment times and natural recovery rates interact to drive health and economic outcomes also remains poorly understood.

Here we investigate the drivers of health and economic costs in mental health conditions based on large-scale real-world health-systems data that capture information about the treatment pathways, duration and clinical outcomes for a range of interventions. We use data from the NHS Talking Therapies (NHS TT) program, formerly known as Improving Access to Psychological Therapies (IAPT), a large-scale initiative aimed at increasing access to evidence-based psychological therapy within the English National Health Service (NHS)13. The NHS TT program provides a standard framework for the collection, monitoring and evaluation of clinical outcomes, which promotes value-based care using a stepped-care approach. This approach is founded on the principle that patients should be offered the least intensive interventions appropriate for their needs first, from step 1 (least severe) to step 4 (most severe)14.

More than 1.5 million patients referred to NHS TT services each year are offered a range of behavioral therapies and interventions such as counseling, cognitive behavioral therapy (CBT) and group therapy15. Within these, CBT is one of the more common types of therapy and is delivered to approximately one-third of NHS TT patients16. With proven clinical effectiveness, structured CBT models are also amenable to delivery via online materials, in either self-guided or therapist-led format17,18. Online CBT approaches have been implemented in NHS TT services to help overcome common barriers to accessing mental health treatment, including enabling out-of-hours access to therapy and reducing waiting times19,20. Furthermore, online-delivered psychological therapies may play an important role in reducing the cost of mental healthcare, with a number of recently published studies comparing the cost-effectiveness of online therapy with that of traditional therapy12,21,22,23.

This study extends this knowledge and investigates the principal cost drivers for treatment of common mental disorders, by comparing a form of internet-delivered CBT with general NHS TT services. Using a modeling approach grounded in real-world health-systems data, we conduct a cost-effectiveness analysis to understand the health and economic impact of different types of care and discuss the potential implications our results have for policy and clinical practice.

Results

Base-case models

We constructed health economic models as health-state-transition (Markov) processes that captured a range of costs associated with different severities of anxiety or depression. Specific costs and utilities were assigned to different health states, with costs and benefits accrued while the patient remained within each state in the model. Transitions between different health states were updated in monthly cycles, and we modeled costs and benefits over a 2 year time horizon. Costs were estimated based on data from (1) Dorset Healthcare University NHS Foundation Trust (DHC), (2) ieso, and, where costs were not directly available, (3) published literature.

Results from the health economic modeling showed a difference in costs and health-related quality of life between the two data samples, with internet-delivered CBT showing small improvements in quality-adjusted life years (QALYs) and lower costs for both anxiety and depression, across all severity bands, relative to NHS TT services in general (Table 1). This means that internet-delivered CBT results in increased health benefits at a lower cost, relative to NHS TT services in general, and can therefore be said to have a ‘dominant’ incremental cost-effectiveness ratio (ICER). The ICER is a standard outcome measure in health economic analysis; it communicates the extra cost of accruing one extra unit of health benefit from using an intervention versus a comparator treatment. In this case, it represents the financial cost (in £) of accruing one extra QALY when using internet-delivered CBT instead of NHS TT services in general. This suggests that at any severity level, internet-delivered CBT is likely to be considered a cost-effective intervention.

Table 1 Deterministic base-case results for anxiety and depression over a 2 year horizon
Full size table

As the data consisted of real-world data samples, they were not matched for symptom severity and demographic characteristics (Extended Data Fig. 1). To test the robustness of the base-case results, we used propensity score matching and inverse probability weighting to balance covariate distribution between the two treatment groups, based on their age and symptom severity at the start of treatment24. Accounting for these differences between the two data samples did not alter the findings (Supplementary Tables 1–3).

Probabilistic sensitivity analysis

To understand the robustness of the modeling results to variations in the input parameters, we performed Monte Carlo simulations. In particular, we randomly varied input parameters based on published estimates of variability25 (when available) or by 15% around the mean26,27,28 and recomputed the model estimates. For both models, the ICER points were compared with a cost-effectiveness threshold of £20,000 per QALY29. We found that our findings were robust to variation in the input parameters, with internet-delivered CBT showing a ‘dominant’ ICER relative to standard NHS TT services, across both conditions (depression and anxiety) and all severity bands (Table 2). Specifically, for the depression population, the probability of internet-delivered CBT being considered cost-effective relative to NHS TT services averaged across all impairment bands was 99.9% (mild, >99.9%; moderate, 99.7%; moderate–severe, 99.9%; severe, 99.9%), with 91.8% of simulated ICER points averaged across all impairment bands located under the cost-effectiveness threshold if ICERs with an incremental QALY loss are excluded (mild, 99.1%; moderate, 89.5%; moderate–severe, 91.6%; severe, 87.1%). For the anxiety population, the probability of internet-delivered CBT being cost-effective relative to NHS TT services averaged across all impairment bands was 90.4% (mild, 78.8%; moderate, 92.4%; severe, >99.9%), with 85.7% of simulated ICER points averaged across all impairment bands located under the cost-effectiveness threshold if ICERs with an incremental QALY loss are excluded (mild, 70.0%; moderate, 87.0%; severe, >99.9%) (Extended Data Figs. 2 and 3).

Table 2 Probabilistic results for anxiety and depression over a 2 year horizon
Full size table

Deterministic sensitivity analysis and scenario analysis

It is notable that the incremental difference in costs is larger for patients with depression, and those with a higher starting severity, relative to patients with anxiety and milder starting symptom severity (Table 1). To understand the behavior of the model in more detail and to identify the principal drivers of cost in the model, we systematically manipulated the various model inputs in a deterministic sensitivity analysis (DSA). Results show that although treatment effectiveness, treatment cost, and utilities of the starting severity health state are the main drivers of the model for milder severity bands, time from referral to end of treatment increases in importance for more severe starting states, more so for depression than for anxiety (Figs. 1 and 2).

Fig. 1: DSA for depression.
figure 1

a–d, Tornado diagrams showing how net monetary benefit is affected in mild (a), moderate (b), moderate–severe (c) and severe (d) depression, relative to the base-case model, by a variation of 20% in each of the model parameters (20% increase in yellow, 20% decrease in green). Model parameters with the greatest spread are those to which net monetary benefit is most sensitive. NTT, NHS Talking Therapies.

Full size image
Fig. 2: DSA for anxiety.
figure 2

a–c, Tornado diagrams showing how net monetary benefit is affected by mild (a), moderate (b) and severe (c) anxiety, relative to the base-case model, by a variation of 20% in each of the model parameters (20% increase in yellow, 20% decrease in green). Model parameters with the greatest spread are those to which net monetary benefit is most sensitive.

Full size image

Time from referral to end of treatment is made up of a combination of waiting time (from referral to start of treatment) and treatment time (from start to end of treatment). As waiting times are one of the biggest challenges faced by mental healthcare services globally, we conducted a sensitivity analysis on NHS TT services data to explore the impact of delays in treatment on costs to the healthcare system and patient health, across all severity bands (Fig. 3). We found that principal drivers of costs from delay resulted from the increase in background costs (that is, use of other healthcare services) and reductions in the patient’s quality of life. These cost increases are most notable for the patient’s health-related quality of life (assuming a QALY is worth £20,000) (ref. 29), particularly so for more severe health states at the start of treatment. Results for internet-delivered CBT are similar and can be found in Supplementary Fig. 4.

Fig. 3: Mean economic cost of mental health conditions by waiting and treatment time, for depression and anxiety.
figure 3

a–g, Economic cost per patient for NHS TT services across all severity bands: mild (a), moderate (b), moderate–severe (c) and severe (d) depression, and mild (e), moderate (f) and severe (g) anxiety. Costs are split by background costs and QALY losses in monetary value over the 2 year time horizon, for waiting plus treatment times ranging from 1 to 12 months. QALYs were converted into a monetary value using the cost-effectiveness threshold of £20,000 (that is, assuming that a QALY is worth £20,000).

Full size image

Another model input that directly relates to the cost-effectiveness of mental healthcare services is treatment effectiveness. We conducted a sensitivity analysis on NHS TT services data to explore the impact of variations in treatment effectiveness on net costs per patient, where a proportion of patients in each severity band moved to less severe health states (positive change in effectiveness) or more severe health states (negative change in effectiveness) (Fig. 4). Results again showed impact on both background costs (which are inversely related to treatment effectiveness) and large changes in costs related to the patient’s health-related quality of life, particularly for more severe health states (assuming a QALY is worth £20,000) (ref. 29). Results for internet-delivered CBT can be found in Supplementary Fig. 5.

Fig. 4: Economic cost of mental health conditions by treatment effectiveness, for depression and anxiety.
figure 4

a–g, The economic cost per patient of mental health conditions for NHS TT services: mild (a), moderate (b), moderate–severe (c) and severe (d) depression, and mild (e), moderate (f) and severe (g) anxiety, split by background costs and QALY losses in monetary value over the 2 year time horizon. Costs are presented for treatment efficacies ranging from −20% (that is, 20% less effective than average NHS TT services) to +20% (that is, 20% more effective than average NHS TT services). QALYs were converted into a monetary value using the cost-effectiveness threshold of £20,000 (that is, assuming that a QALY is worth £20,000) (ref. 29).

Full size image

Discussion

Here we have modeled healthcare costs based on two large data samples of the real-world delivery of mental healthcare treatments. We did this to understand the principal drivers of cost within mental healthcare provision, and our findings highlight the importance of reduced waiting and treatment times for both anxiety and depression, particularly for more severe presentations. Although we take a standard approach of computing the relative costs between two services (NHS TT in general versus internet-delivered CBT), sensitivity and scenario analyses show that the principal benefits accrued by internet-delivered CBT relate to the ability to treat patients sooner with a program that delivers similar rates of recovery under a shorter treatment time—from patient referral to completion of treatment, internet-delivered CBT can be completed in approximately half the time taken for general NHS TT services (Extended Data Fig. 4), with similar treatment effectiveness. These findings may be explained by the fact that online forms of therapy have been reported to facilitate patient disclosure and enhance self-reflection through writing, which may have an effect on treatment efficacy and duration30,31. Furthermore, internet-delivered CBT cost savings are higher for depression than for anxiety, and for more severe presentations than for mild. This is partly due to the higher background medical costs associated with severe mental health conditions, as well as higher background costs for depression relative to anxiety, meaning the value of completing treatment faster is comparatively greater for these patients7,32. In addition, the proportional difference in treatment costs between internet-delivered CBT and standard NHS TT services is also larger for depression than for anxiety (Extended Data Fig. 5), further increasing the relative cost–benefit of internet-delivered CBT in depression.

Results from sensitivity analyses show that treatment effectiveness, time from referral to end of treatment and treatment cost are the most important drivers of cost in this model, with the importance of waiting and treatment time increasing for more severe starting states. This leads to an observed greater benefit of internet-delivered CBT at higher severity levels. Furthermore, scenario analyses show that increases in waiting and treatment times and decreases in treatment effectiveness have a substantial impact on health-related quality of life, highlighting the value of getting patients into effective treatments more quickly, not only to reduce economic cost but also to alleviate human suffering. In other words, the substantive costs of mental health disorders do not come from treating them, but rather from not treating them.

Real-world data is becoming increasingly influential for understanding and making policy decisions in healthcare33. The large dataset used for this analysis provides insight into the key drivers to both health and economic outcomes, when comparing psychotherapy delivery methods. The strength of this data is that it can track individuals through their treatment with a high level of detail on number of sessions, waiting and treatment time, requirement for repeat treatment and engagement with treatment in a real-world setting, which may be more reflective of behavior in clinical practice than a randomized controlled trial.

The results of this study have particularly important implications for clinical practice, given current global accessibility issues, exacerbated by the global pandemic34,35, in which only a minority of people in need have access to psychological therapy. Internet-delivered CBT and other forms of online therapy have the potential to ameliorate this issue and substantially reduce waiting times, by offering a degree of flexibility that is not available in traditional face-to-face services36. Internet-delivered interventions have been shown to be cost-effective relative to a range of control comparators (wait-list control, treatment as usual, attention control and other psychological and pharmaceutical therapies)12,37,38. Thus, the use of internet-delivered CBT can increase capacity through more efficient service delivery and faster treatment and recovery. These benefits are further bolstered by reducing background medical costs and improving patients’ quality of life.

However, it is important to note that not all online service provision is equivalent. Specifically, therapist-guided therapies show better clinical outcomes and lower drop-out rates relative to self-guided interventions16,17,39. In addition, the health economic evidence for self-guided interventions is less favorable than that for therapist-guided interventions12,38, and there is a paucity of knowledge about the cost-effectiveness of self-guided interventions compared with that of treatment as usual or active controls rather than waiting lists12. Online and computerized self-guided interventions, or those with only minimal therapist involvement, are classed as low-intensity interventions. They have the advantage of being inexpensive and easily scalable, but are only suitable for patients with milder presentations, and show low engagement rates in real-world scenarios10,17,40. By contrast, internet-delivered CBT is an effective type of online therapy, in which patients are offered one-to-one sessions with an accredited CBT therapist. As a result of its one-to-one nature, internet-delivered CBT is classed as high-intensity therapy and suitable for patients with more severe conditions, while retaining other advantages of online therapy, such as facilitating access and enhancing disclosure24.

Finally, it is important to note that the models presented here relate to the specific context of the English NHS, with background costs, treatment costs for NHS TT, utilities and natural deterioration rates conservatively estimated from the literature. Although we see considerable potential to extend the approach to other healthcare settings, costs and key parameters of the model will likely differ (that is, waiting times, effectiveness and costs of treatment may differ). However, it is noted that the granular data that enabled the analyses in the current study are not available for most mental healthcare systems, and routine measurement of clinical outcomes for mental healthcare remains the exception rather than the norm.

Future work based on the analysis of other large healthcare datasets, with accurate cost and healthcare resource utilization data, would reveal a richer picture of the interactions between model parameters that occur across different real-world healthcare settings. This will be important to inform mental healthcare policy and optimize mental healthcare delivery focusing on immediacy of access to care and measurable clinical effectiveness, to reduce both human and economic costs of mental health conditions.

Limitations

This study explores the drivers of health and economic cost in mood and anxiety disorders by comparing a specific form of psychological therapy, internet-delivered CBT, with NHS TT services in general, which include various types of therapy (for example, CBT, counseling, interpersonal therapy, psychodynamic therapy) delivered over a range of modalities (for example, face to face, online, video, or telephone). Although results show that internet-delivered CBT offers similar treatment effectiveness under a shorter treatment time, relative to NHS TT services in general, further research is needed to explore the comparative cost-effectiveness of specific therapy types or modalities (for example, internet-delivered CBT versus face-to-face CBT, or internet-delivered CBT versus internet-delivered counseling). Furthermore, it is important to acknowledge that the current economic evaluation only captured direct medical costs to the NHS, including treatment costs and background medical costs. Given the substantial indirect costs of mental health conditions, particularly related to unemployment and productivity losses41, it is recommended that future economic evaluations of different therapy types consider modeling indirect costs alongside direct costs.

Another limitation affecting the current study is the potential for bias, given the statistically significant baseline differences between the two treatment groups (Extended Data Fig. 1). Although we attempted to mitigate this through propensity score matching and inverse probability weighting, the risk of self-selection bias may remain. Factors such as patients’ socioeconomic status and age group may affect their digital literacy, and patients who choose internet-delivered CBT may be more likely to respond to treatment, relative to those who receive standard NHS TT treatment. Although this risk is further mitigated by the fact that patient preference is recognized as an important factor for driving clinical outcomes in standard NHS TT services42, information about whether or not a patient received their preferred treatment type or modality (for example, CBT, counseling, face to face and online) is not systematically recorded. This is a trade-off in using real-world healthcare data: although we have been able to learn from a very large sample, we face the limitation of not having data available to control for a variety of important factors (for example, patient-preferred treatment choice, digital literacy, socioeconomic status, mental health comorbidities and others) in the way that would be possible in a randomized control trial. Enriching data collection around these variables would enhance future research.

It is also important to recognize that certain model inputs were estimated based on the literature (for example, NHS TT treatment costs, natural recovery and symptom deterioration rates). Although care was taken to conduct a thorough review of the existing literature, in some cases, the patient population from which model inputs were estimated did not closely match the patient demographics in the current study (for example, natural recovery and symptom deterioration rates are estimated from published research on a geriatric population of patients with depression in the United States)43. As these model inputs are assumed to be the same for both internet-delivered CBT and standard NHS TT treatment, they are unlikely to meaningfully affect model results.

Finally, mortality was not included within the economic models, primarily because mortality data were not captured in the context of our study. Although we recognize that there is extensive literature demonstrating increased mortality risk associated with mental health disorders44, some studies suggest that this excess mortality is primarily driven by multimorbidity and frailty45. Given the short-term time horizon for the current model, and the fact that patients with comorbid chronic physical health conditions were excluded, the impact on the model results if mortality rates were applied is expected to be small. Nevertheless, this is a limitation for the current economic analysis and future work should consider the possibility of incorporating mortality data into analyses where possible.

Conclusion

We have used large robust real-world healthcare data to understand the costs associated with treating mental health conditions. Our analysis highlights the key healthcare and human costs from waiting times and treatment duration. Treatment costs are generally outweighed by the increased draw on other medical services, and in the loss of health-related quality of life that comes from poor mental health. This highlights that the substantive costs of mental health conditions are driven by not intervening in an adequate or timely manner. Internet-delivered CBT provides one means of reducing waiting times and quickening the pace of a patient’s recovery, with similar treatment effectiveness.

Methods

Ethical statement

This study complies with ethical regulations set by the NHS TT program (https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/improving-access-to-psychological-therapies-data-set/iapt-data-set-fair-processing-guidance), under which both DHC and ieso operate. The NHS TT program is a large-scale national initiative aimed at increasing free-of-charge access to evidence-based psychological therapy for common mental health disorders within the English NHS13. The information captured through the dataset of NHS TT is intended to support the monitoring of the implementation and effectiveness of national policy and legislation, policy development, performance analysis and benchmarking, national analysis and statistics and national audit of NHS TT services. At registration, patients agree to the services’ terms and conditions, including the use of deidentified data for research and audit purposes, including academic publications or conference presentations. As this is a health economics evaluation using routinely collected NHS TT data, it is classed as a service evaluation and therefore not subject to formal ethical review46,47. Before initiation, the study was reviewed internally at both ieso and DHC following their respective standard procedures for service evaluation studies.

Study design and data source

This study is an economic evaluation of real-world mental healthcare-systems data, using routinely collected quantitative clinical-outcomes data from patients receiving care for a mood or an anxiety disorder within the UK National Healthcare Service. Deidentified real-world data were obtained from records of 83,110 patients receiving either standard treatment through NHS TT services or internet-delivered CBT, discharged from treatment between January 2018 and December 2020. All patients received treatment under a stepped-care approach, in which patients are offered the least intensive interventions appropriate for their needs first14. Under the stepped-care model, patients with milder presentations are signposted to low-intensity interventions (step 2), whereas patients with more severe symptoms or more complex needs are signposted to high-intensity interventions (step 3).

Standard NHS TT treatment data were provided by DHC, for patients in Dorset who received outpatient treatment for a mental health condition using a wide range of treatment interventions, including CBT, counseling, guided self-help, and group therapy (Supplementary Table 6). Internet-delivered CBT data were provided by ieso, for patients across England who received outpatient CBT for the treatment of a mental health condition, delivered over the internet using a commercial package provided by ieso (https://www.iesohealth.com/). Patients self-referred or were referred by a primary healthcare worker directly to the service. Upon registration, patients were assigned to a qualified CBT therapist and National Institute for Health and Care Excellence (NICE)-approved disorder-specific CBT treatment protocols10 were delivered during scheduled sessions in an online therapy room, via one-to-one real-time written conversation.

Given that most of the literature covering the health economics of mental healthcare focuses on depression and anxiety, only patients with a primary diagnosis of depression (International Classification of Diseases, Tenth Revision codes F32, F33 and F34.1) or generalized anxiety disorder (International Classification of Diseases, Tenth Revision code F41.1) were included (45,065, 54% of patients referred to NHS TT services or internet-delivered CBT). Clinical diagnosis was determined by a qualified clinician after a one-to-one assessment with the patient.

For simplicity, patients who attended only assessment sessions, but no treatment sessions, or patients with a long-term physical condition such as diabetes or chronic pain, were also excluded from the final model. However, we acknowledge that the overlap between long-term physical conditions and mental health is substantial and likely to incur greater resource use from declining mental health5,16. Additional results for this group of patients are presented in Supplementary Table 7.

The final model was populated with real-world data from 27,540 patients receiving standard treatment through NHS TT services (16,790, mean age 35.7, 65.7% female) or internet-delivered CBT (10,750, mean age 32.6, 73.6% female) (Extended Data Fig. 1). As ieso operates within the NHS TT program, there is a small proportion of DHC patients who received internet-delivered CBT provided by ieso (687 of 16,790, 4.1%). Removing the small proportion of patients who received internet-delivered CBT from the DHC dataset did not meaningfully affect the results of the final model. To make the results representative of the complete service offering provided by DHC, we therefore opted not to exclude these patients from the dataset.

To test the robustness of the final model, we used propensity score matching to create an adjusted model in which covariate distributions were balanced between treatment groups to reduce the risk of selection bias. A logistic regression model was fitted to estimate the propensity score for each patient, with age and symptom severity at the start of treatment as covariates24. Patients receiving standard treatment through NHS TT services were matched with patients receiving internet-delivered CBT, using a nearest neighbor algorithm and a 1:1 ratio. An additional adjusted model used inverse probability weighting to adjust for potential confounding variables. We calculated the probability of a given patient belonging to the standard NHS TT versus internet-delivered CBT, based on their age and symptom severity at the start of treatment24. A weight was assigned to each patient based on the inverse of this probability, with model inputs calculated from this weighted sample. The matching analyses were performed in R v4.2.0 (ref. 48).

Model inputs

Data used to populate the models included waiting time, treatment time, clinical diagnosis, symptom severity before and after treatment measured using the Patient Health Questionnaire-9 (PHQ-9) (ref. 49) or the Generalized Anxiety Disorder Assessment (GAD-7) (ref. 50), number of treatment sessions and recurrence rates (calculated from patients discharged from NHS TT services between October 2015 and September 2020). Inputs relating to internet-delivered CBT were taken from data provided by ieso, and inputs relating to NHS TT services in general were taken from data provided by DHC. A pragmatic search of the literature was used to supplement the model with relevant costs, clinical data and utilities.

Clinical effectiveness data

A summary of clinical effectiveness data used as input for the model can be found in Extended Data Fig. 4. Transition probabilities from pretreatment to posttreatment were calculated from DHC and ieso data for each severity level of anxiety and depression (measured using the GAD-7 and PHQ-9 questionnaires, respectively), for patients who engaged with treatment (Supplementary Tables 8 and 9). Following NHS TT guidelines, patients who complete a minimum of two treatment sessions are considered engaged, as this is the minimum number of sessions a patient must attend such that pre- and posttreatment data are available and clinical change can be evaluated51. Until a person had completed treatment, they were assumed to stay within their starting severity health state. Those who did not engage with treatment were assumed to naturally deteriorate or improve relative to their starting severity; hence, they were able to transition immediately to better or worse health states. The proportion of patients who did not engage with treatment, mean waiting times and mean treatment times were calculated for NHS TT services in general and internet-delivered CBT.

Clinical effectiveness data calculated from DHC and ieso datasets were supplemented with additional data from the literature, including rates of symptom deterioration in each cycle and natural recovery rates, that is, the likelihood that the condition is alleviated without treatment43,52. Values extracted from the literature are assumed to apply equally to NHS TT services in general and internet-delivered CBT. Where separate values for anxiety and depression are not available from the literature (for example, symptom deterioration rates), they are assumed to apply equally to both conditions.

Cost and health-related quality-of-life data

A summary of cost and utilities data used as input for the model can be found in Extended Data Fig. 5. All costs used in the analysis were in UK pounds sterling (2019–20 value). Actual treatment costs for internet-delivered CBT were provided by ieso. All other cost data were sourced from the literature7,32,53 and inflated to 2019–20 value using the Personal Social Services Research Unit inflation indices32. Average treatment costs per patient for internet-delivered CBT and general NHS TT services included the costs for each session, as well as the number of sessions by severity band (that is, mild, moderate, severe) and treatment pathway (that is, step 2, step 3). Total treatment costs were calculated by multiplying the average treatment session cost by the average number of sessions, weighted by the proportion of patients receiving low-intensity (step 2) versus high-intensity interventions (step 3). Full details on treatment costs, number of treatment sessions, and proportion of patients by severity band and treatment pathway can be found in Supplementary Table 10. Background costs comprise medical costs associated with the mental health condition outside of mental health treatment costs (for example, patients with a mental health condition are more likely to attend general practitioner appointments). Background costs used in the final models were based on a previous analysis conducted on behalf of NICE, using costs estimated from the Proudfoot Study7. Background costs were assumed to be the same for internet-delivered CBT and NHS TT services in general.

Utilities for each health state were based on a systematic review and qualitative assessment of health-related quality of life for NHS TT patients with mental health conditions, with additional details provided through communication with the author54,55. Following a systematic review of the psychometric performance of different quality-of-life instruments in 3,512 patients with common mental health problems, a study used the short-form six-dimension health index (SF-6D) to estimate depression and anxiety utility scores54, which we use in the current study. Utility values vary between 0 and 1, with 1 representing a year spent in full health and lower values associated with greater decreases in health-related quality of life. The utilities in the model were not adjusted for aging given the short time horizon of the model and the heterogeneity in average age of those undergoing mental health treatment.

Model structures

Two health economic models were developed to assess the cost-effectiveness of using internet-delivered CBT versus NHS TT services in general, for generalized anxiety disorder and depression. The models were developed in Microsoft Excel 365 for Enterprise (Microsoft Corporation). Before implementing the model, we conducted model scoping to identify published anxiety and depression model structures. These were used to define and validate the model structure used in this study56,57,58, although we note that we did not submit a formal health economic analysis plan. Models were structured as health-state-transition (Markov) models capturing different severity levels of depression or anxiety. Specific costs and utilities were assigned to each health state, and patients incurred costs and accrued benefits for the period of time in which they remained in each health state in the model. Monthly cycles were used in the models to capture the movement between health states, with costs and benefits modeled over a 2 year time horizon for both models (Extended Data Fig. 6). Longer time horizons are usually applicable to chronic conditions associated with ongoing clinical management rather than time-limited treatment episodes, whereas shorter time horizons are applicable to acute conditions. Whereas mental health conditions are often acute, with patients becoming symptom free following a course of treatment, chronicity in the form of symptom recurrence several months or years following treatment is not unusual. Although most economic evaluations for internet-based interventions for mental health use shorter time horizons12, in this study, we selected a 2-year time horizon to model the evolution of anxiety or depression in the short-to-medium term, capturing the balance between the acute versus chronic (recurrent) nature of mental health conditions. As the benefit of internet-delivered CBT accrues over time, a longer time horizon would likely lead to a greater net monetary benefit of internet-delivered CBT relative to standard NHS TT treatment.

Severity levels in the model were defined using GAD-7 for anxiety and PHQ-9 for depression49,50. We capture health states using recognized severity bands of minimal (GAD-7 ≤ 4, PHQ-9 ≤ 5), mild (5 ≤ GAD-7 ≤ 9, 5 ≤ PHQ-9 ≤ 9), moderate (10 ≤ GAD-7 ≤ 14, 10 ≤ PHQ-9 ≤ 14), and severe (15 ≤ GAD-7 ≤ 21, 20 ≤ PHQ-9 ≤ 27) (ref. 50). For depression, we additionally included a moderate–severe band (15 ≤ PHQ-9 ≤ 19) (ref. 49).

Upon referral, patients entered the model in the ‘pretreatment’ state at their starting severity. They remained in that state until treatment was complete, which includes the waiting time for treatment to begin, as well as treatment time itself. This is because data were not consistently available at the end of the waiting time, which would have allowed us to model the health state immediately before the start of treatment. During this time (waiting plus treatment time), patients incurred background medical costs (that is, using other medical resources outside of their mental health treatment, such as general practitioner visits), as well as the QALYs associated with their ‘pretreatment’ health state. This is a simplifying but conservative assumption made to avoid unnecessary complexity in the model—in reality, the majority of patients will see an improvement during treatment, whereas a smaller proportion will experience a deterioration in symptoms before or during treatment. An alternative way to address this would be to assume a constant rate of change from the baseline health state to the health states at the end of the treatment. However, this would add further assumptions to the models, when in reality the true movements over this time period are unknown.

Following treatment, patients transition to ‘posttreatment’ states based on their severity at the end of treatment. The treatment costs were applied at the end of treatment. For the rest of the time horizon (that is, up to 2 years from ‘pretreatment’), patients could transition to higher-severity health states based on deterioration rates or back to the ‘minimal’ health state, based on natural recovery rates. Half-cycle correction was applied in the model to account for the fact that, on average, changes to peoples’ health state occur at the midpoint of each cycle, rather than at the start or end of each discrete cycle.

The model also captured the cost of any additional treatment undertaken within 2 years of initiation of the original treatment. The proportion of patients receiving recurrent treatment for any mental health condition was calculated from patients discharged from DHC NHS TT services between October 2015 and September 2020. A cost was applied for delivering these treatments (at the same level as for the original treatment), but for simplicity, patient benefits (QALYs) and additional background costs accrued during the time period between the original treatment and recurrent treatments were not included. Health outcomes for the recurrent treatment were also not captured, because of the complexities of modeling treatment sequences. These simplifying assumptions were made on the basis that probability of recurrence is relatively low (Extended Data Fig. 4) and changes to how recurring episodes are modeled are unlikely to meaningfully affect model results.

The analyses were conducted in line with the NICE reference case, modeling costs from a UK NHS and personal social services perspective29. Future costs and benefits (QALYs) were discounted at a rate of 3.5% per year59. A cost-effectiveness threshold of £20,000 per QALY was used to assess the cost-effectiveness of internet-delivered CBT, which is recommended by NICE as the maximum amount to be paid for an additional unit of health benefit provided by an intervention29. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS)60 were used as guidance and to inform the preparation and writing of this report (Supplementary Information).

Economic and sensitivity analysis

The cost-effectiveness models generated total costs and QALYs per patient for both NHS TT services in general and internet-delivered CBT. A head-to-head comparison was conducted comparing internet-delivered CBT with NHS TT services generating an ICER, that is, a ratio showing the extra cost per extra unit of health benefit (QALY) for internet-delivered CBT, relative to NHS TT services.

To quantify the level of confidence of the model results in relation to uncertainty in the model inputs, a probabilistic sensitivity analysis was conducted29. A series of ICERs were calculated, with 1,000 Monte Carlo simulations performed for each anxiety and depression severity band (we confirmed that convergence was reached using this number of simulations). To estimate input parameter variation, model parameters including proportion of patients who engaged, symptom deterioration and natural cure probabilities, probabilities of recurrence, and utility values were input into a beta distribution. Variation in completed number of sessions, cost per session of NHS TT services, background costs, recurrence treatment costs, waiting time, and treatment time was estimated using gamma distributions. The cost per session for treatment with internet-provided CBT was set to be fixed as this was costed directly from the study and no variation was expected. Dirichlet distributions were used to estimate the variance in treatment effectiveness and proportion of patients per step, using a conditional beta approach. Any parameters that were sourced from study-reported data were varied by the standard errors and alpha and beta values reported in the study25. For a small number of parameters including natural cure rate, treatment costs for standard NHS TT treatment, background costs and utility values, it was not possible to estimate standard error due to the absence of data on the variability around the sampling distribution of mean values. In these cases, following common practice in economic evaluations26,27,28, the standard error was assumed to be equal to 15% of the mean. A series of ICERs resulting from the probabilistic sensitivity analysis were plotted in a cost-effectiveness plane, with the probability of internet-delivered CBT being cost-effective relative to NHS TT standard care also plotted as a cost-effectiveness acceptability curve. The cost-effectiveness threshold was set at £20,000 per QALY29.

DSA was conducted by varying one input at a time while holding other inputs constant and recording the impact on the model results, to determine the key drivers of the results. All inputs were varied higher and lower by an assumed 20% of the mean in the DSA. This was done to investigate which inputs had an impact on the model results when varied individually. For time from referral to treatment, the value used in the model and the DSA is rounded to the nearest integer to apply the transitions at the end of a cycle. This may result in values that are the same as those of the base case. The results of the DSA are presented using net monetary benefit, which is calculated by multiplying the incremental QALYs by the cost-effectiveness threshold, minus the incremental cost. A positive net monetary benefit indicates that an intervention is estimated to be cost-effective at the chosen threshold.

Finally, scenario analyses were conducted to test the impact of varying key parameters such as waiting times and treatment effectiveness, to understand how these influence the results of the model across the various severity bands.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.