Main

Survival rates in lung cancer in England and the United Kingdom are significantly lower than in countries with similar levels of expenditure on health care (Holmberg et al, 2010; Coleman et al, 2011), and have shown little improvement over the last three decades compared with those seen in other cancers (CRUK, 2012). The importance of diagnosing cancer at an early stage is well understood (Richards, 2009b; DOH, 2012), with many treatment options only effective if used early in the disease course. However, little is known about the best way to achieve earlier diagnosis, with most of the onus being placed on a poor level of understanding of the symptoms of lung cancer, and a reluctance to present at primary care among the general public. Recent government initiatives in England have sought to raise public awareness of the signs and symptoms of cancer to promote earlier presentation to primary care (Richards, 2009a). A range of ‘Be Clear on Cancer’ (BCOC) campaigns have been run since the initiative’s launch in 2011, including regional and national lung cancer early awareness campaigns (Ironmonger et al, 2014). The campaigns have been conducted through a range of advertising mediums including television, radio and leafleting.

A recent publication has presented the results of these campaigns (Ironmonger et al, 2014), finding a shift in the stage of diagnosis of non-small-cell lung cancer (NSCLC) following the awareness campaign and significant increases in levels of public awareness of the symptoms of lung cancer, urgent GP referrals for suspected lung cancer and overall diagnosis. The authors concluded that the results may lead to improved long-term survival, and that the assessment of the cost-effectiveness of the interventions is an important next step. This paper reports on an evaluation of the cost-effectiveness of both the regional pilot and the national BCOC campaign in shifting the stage at diagnosis of NSCLC.

To conduct such an evaluation, it is necessary to understand the natural history of the disease in undiagnosed, and thus untreated, patients, that is, the development and progression of cancers before a diagnosis has been made. This can then be used to provide a prediction of the lifetime health outcomes of the patient population (both with and without an early awareness campaign) by taking into account the stage and age at which diagnosis occurred. To date, there has been no published research estimating the natural history of undiagnosed NSCLC, largely because of the limited evidence available (Black et al, 2006). Therefore, in addition to the estimation of the cost-effectiveness of the early awareness campaigns, this report details the work conducted to characterise the natural history of untreated NSCLC, using calibration techniques (Weinstein, 2006; Whyte et al, 2011), necessary to inform the cost-effectiveness analysis.

Materials and methods

Overview

A decision-analytic model was developed to assess the cost-effectiveness of an early awareness campaign for the signs and symptoms of lung cancer in the English population. The model considers the proportion of patients at each stage of disease, from initial development of the disease before diagnosis, progression and final diagnosis. The long-term outcomes associated with diagnosis by stage of disease are expressed in terms of quality-adjusted life years (QALYs) through a consideration of the mortality risk and health-related quality of life (HRQoL) of NSCLC patients. Costs are considered from the perspective of the UK NHS and Personal Social Services, using 2012 prices, with costs and benefits discounted at 3.5% per year, consistent with the approach taken by the National Institute for Health and Care Excellence (NICE) to value future costs and benefits at current rates (NICE, 2013).

The cost-effectiveness of the regional and national campaigns was estimated by applying the observed shift in the incidence and distribution of NSCLC by stages at diagnosis to the analytical model and considering the resultant change in long-term outcomes and costs. The same model structure was applied to both the regional and national campaigns. To facilitate a direct comparison of the results of the two campaigns, and to account for the lack of an estimate of the direct cost of the regional campaign, the results of the regional pilot were extrapolated to a national population by weighting the results by the size of the population covered. Clearly, this approach relies on the regional campaign approach being similar to the national campaign and that the results of the regional campaign can be extrapolated to a national level.

Model structure

To estimate the prognostic implications of the awareness campaigns, it was necessary to model the natural history of the disease, the structure of which is shown in Figure 1. The model represents a Markov-type structure, consisting of four main health states: no disease (specifically no NSCLC), prediagnostic NSCLC, postdiagnostic NSCLC and mortality.

Figure 1
figure 1

Schematic of the natural history model.

Full size image

Solid black lines represent transitions that can be estimated directly from available data. Dashed grey lines denote transitions that are unobservable in practice owing to their occurrence before diagnosis. Consequently, these are parameters that must be estimated through methods of calibration, which is discussed in the following section. The cycle length of the model, the minimum time in which patients can move from one state to the next, is set at 1 month to account for the potentially rapid development of the disease while not requiring an excessive level of computation.

Within each of the cancer states (pre- and postdiagnosis), there are three stages of disease severity representing limited, advanced and extensive disease. For the aggregated TNM staging system, these are assumed to represent stages I and II for limited disease, stage IIIa for advanced disease and stages IIIb and IV for extensive disease. The development of NSCLC was not assumed to be linear, such that the disease can develop directly from limited to extensive stages. Although the BCOC campaigns were designed to target those over the age of 50 years, it is clear that all ages will be exposed to the campaign. Therefore, the evaluation considered the entire English population over the age of 30 years, an age selected because of the small incidence of NSCLC below it, and to reduce excessive computational requirements.

Mortality was considered separately for NSCLC-related deaths and other-cause deaths. NSCLC mortality was assumed to occur only in the postdiagnosis stages of the disease under the assumption that all patients would be diagnosed with NSCLC before death since the incidence of lung cancer diagnosis at post mortem without clinical diagnosis is relatively rare (estimated to be <5% of the total incidence from the 2011 National Lung Cancer Audit Dataset (LUCADA)) (NHSIC, 2011).

Early awareness campaign

The BCOC regional campaign was conducted in the Midlands between October and December 2011, using the rest of England as a control area. The regional campaign collected data on the incidence of NSCLC diagnosis by stage, in the campaign and control areas, before and during the campaign from LUCADA, which collects the total incidence of NSCLC in England (NHSIC, 2011). This allowed for a ‘difference in difference’ analysis to be undertaken, which measures the effect of the campaign on incidence of NSCLC, by stage, while controlling for underlying changes in the diagnoses of disease not related to the campaign by adjusting the change in diagnoses in the intervention area by the change observed in the control area.

The full national campaign took place between May and July 2012. Control data were collected for all English trusts for patients first seen between May and July 2011 and this was compared with the campaign period. To allow for the lack of a direct control area, this was compared with the trend in incidence in the period February to April 2011 and February to April 2012. The full details are available elsewhere (Ironmonger et al, 2014).

Tables 1 and 2 report the raw and percentage changes in the incidence of diagnosis, by stage, for both campaigns in the control and campaign groups, as well as the adjusted relative change in stage (Ironmonger et al, 2014). The impact of the campaigns were ‘smoothed out’ by weighting the incidence at each stage by the respective total observed incidence; this represents the underlying change in the stage at clinical identification. Patients whose diagnosed stage of disease was listed as unknown in LUCADA were excluded from the analysis. This table shows that, in both campaigns, there was a shift toward earlier diagnosis of disease. The national campaign showed a smaller relative reduction in the proportion of patients diagnosed at late stages of disease, but a larger increase in stage I and II relative diagnoses. The results of the regional campaign were scaled up to a national level to allow comparability with the national campaign.

Table 1 Change in incidence and percentage change in the weighted distribution of stage of diagnosis, regional campaign
Full size table
Table 2 Change in incidence and percentage change in the weighted distribution of stage of diagnosis, national campaign
Full size table

The relative changes were applied to the natural history model, presented in the following section, to consider the implications of the estimated NSCLC stage shift generated by the campaigns when compared with no intervention for long-term costs and QALYs. Owing to the immaturity of the data being reported from any of the campaigns, and with no data collected after the end of the campaign after the intervention, it was not possible to consider the duration of effect of any early awareness campaign. The base-case analysis assumes that the campaign only has an effect for the period of the campaign, likely to represent a conservative assumption of the effectiveness of the campaigns.

Model inputs

A major challenge in modelling the natural history of NSCLC is the requirement to understand the transition of patients in prediagnostic stages of disease, which cannot be informed by clinical trials (Black et al, 2006). Therefore, in the absence of observable data, model calibration techniques (Weinstein, 2006; Whyte et al, 2011), together with the elicitation of clinical knowledge, were used to estimate the unobservable transition probabilities. Methods of calibration seek to make use of all available evidence relevant to the disease in the estimation of a set of transitions. This can include expert opinion as a source of prior information, and observable data on the prevalence or incidence of cancer, together with a prior expectation about the structure of the natural history of the disease. Further details of the calibration method as well as reporting guidelines used to structure the model can be found elsewhere (Stout et al, 2009; Vanni et al, 2011). The calibration techniques used in this analysis have been developed and applied elsewhere by Weinstein (2006), Whyte et al (2011, 2012) and Whyte and Harnan (2014).

The technique uses the Metropolis–Hasting algorithm to estimate the natural history parameters by generating multiple sets of parameters from the posterior distribution, informed by observed data (Whyte et al, 2011). The calibration presented here utilised an initial set of prior probabilities elicited from expert clinicians (Doctors Peake, Rogers and Callister acknowledged above), and calibrated it to data on the incidence of NSCLC in England from LUCADA (NHSIC, 2011). The results of the expert elicitation exercise are presented in Table 3, the plausible range reported was used to test the validity of the calibration results. Details of the calibration method and the application of the Stout et al (2009) checklist for reporting of calibration methods are reported in the Supplementary Appendix.

Table 3 Parameter estimates derived from expert elicitation exercise
Full size table

It was assumed that the expected probability of initially developing NSCLC increases with age, and the probability of an individual with prediagnosis NSCLC being clinically identified increases with the advancement of cancer stage. However, owing to a lack of available data to power a more complex model, it was not possible to consider the impact of sub-population effects such as sociodemographic factors and smoking status. Similarly, while age was assumed to impact the probability of initial development of NSCLC, it was not assumed to affect the expected speed of progression. The calibration resulted in the state transitions reported in Table 4 alongside all other model inputs. The table shows that the calibrated estimates fitted well within the expert elicited estimates given in Table 3.

Table 4 Costs, HRQoL, model inputs and impact of the campaigns
Full size table

Mortality

Data on NSCLC survival were provided by the International Cancer Benchmarking Partnership and represents an estimate of net survival from lung cancer for the United Kingdom by age and stage of disease. Data on other-cause age-adjusted mortality were taken from Interim Life Tables for 2008–2010 for England (ONS, 2011) and were adjusted to remove mortality owing to lung cancer for each age group.

Costs

A systematic review of the costs of lung cancer was conducted using the NHS Economic Evaluation Database (EED, 2012). The systematic review identified one published study of relevance, Fleming et al (2008)), which estimated hospital costs relating to lung cancer in Northern Ireland. The authors analysed the case notes of 724 patients diagnosed in 2001 for a period of 12 months following presentation by cell type and extent of disease (limited, advanced and extensive). The severity groupings reported in the study were assumed to be representative of the three stages defined in the model. Costs were adjusted for inflation to a 2012 price level, and are presented in Table 4. Unfortunately, no non-hospital costs were identified in the systematic review, and thus were not available to inform this analysis. It is likely that the estimate of costs used represent a significant underestimate of the total cost to the NHS.

In addition to the costs associated with increased diagnosis and treatment, it is important to consider the cost of the campaign itself. Personal communication with the Department of Health provided a cost estimate for an England-wide campaign of £2.9million (personal communication with clinical advisors: Dr Mick Peake, Dr Matthew Callister, Dr Trevor Rogers; personal communication with Dr Bernard Rachet, London School of Hygine and Tropical Medicine). This cost was used to inform both the scaled up regional pilot and the national campaign as no estimate of total cost of the regional campaign was available. The difficulty in obtaining a more robust estimate or specific references to the cost of the campaign highlight the challenges associated with a full evaluation of such campaigns.

Health-related quality of life

A systematic review of literature on HRQoL associated with lung cancer identified only one publication, which was suitable to inform the model based on the following criteria: (i) estimation of HRQoL in terms of a preference-based weighting suitable for calculating QALYs; (ii) representative of an English setting; and (iii) reported HRQoL by stage of disease (Sturza, 2010). This paper was used to derive the mean HRQoL weights used in the cost-effectiveness model, as shown in Table 4.

These HRQoL values were compared with the age-specific HRQoL weights of the general population from the Health Survey for England (Ara and Brazier, 2009) and applied in the model as decrements for 5 years after initial clinical diagnosis. Long-term survivors of the disease (i.e., beyond 5 years) were assumed to return to the HRQoL level of the general population at that age.

The literature review failed to identify any articles in which the impact of false diagnoses on HRQoL was quantified. Although it is clear that a false diagnosis has the potential to hugely damage HRQoL of a patient over a short period of time, the analysis has not incorporated these effects. The impact of such an omission on the cost-effectiveness results is unclear, as more patients will present the proportion of false diagnosis, which may increase or decrease as a result of the campaign.

Analytical methods

The natural history model was used to generate estimates of long-term expected costs and QALYs with and without a national awareness programme. Cost-effectiveness is expressed in terms of the ICER, a measure of the extra cost per additional QALY gained from the campaign. This can be compared with a cost-effectiveness threshold, which represents health forgone as a result of other services being displaced to fund the campaign from a limited NHS budget. The threshold used by NICE is £20 000–30 000 per QALY gained (NICE, 2013), although recent research suggests that this should perhaps be lower at around £13 000 per QALY (Claxton et al, 2013).

A base-case analysis was undertaken based on the methods outlined above. In addition, two scenarios are considered to reflect uncertainty in the assumptions used:

(1) Size of the ‘worried-well’ population attending their GP and receiving chest X-rays: The base-case analysis assumed that there was no change in the size of the population visiting their GP as a result of the campaign not diagnosing individuals with NSCLC. However, as shown in the clinical results of the BCOC campaigns this is unlikely (Ironmonger et al, 2014). The results of the campaigns showed an increase in the number of presentations at the primary care, with a 63% increase in presentation of patients with a cough in the 8 weeks of the national trial. This scenario considered the impact of an increase in the primary care presentations by estimating the necessary size of the increase in presentation of those without lung cancer that would result in the campaign no longer being considered to be cost-effective and compares it against findings from the campaigns. Each GP consultation is assumed to be clinic based and last 11.7 minutes (costing £53.00) (PSSRU, 2011).

(2) Route of diagnosis: As part of the local BCOC campaigns (DOH, 2012), several local project teams collected data on route to diagnosis of lung cancer over the period of the campaign to investigate whether the route to diagnosis was also affected by the campaign. A reduction in diagnosis at emergency departments is likely to be associated with a significant cost saving due to the large cost per bed day of emergency care compared with presentation at GPs. The scenario assumed a cost per accident and emergency attendance of £129, the upper bound of the average cost per attendance reported by a Foundation Trust Network survey (FTN, 2012), as the only measurable cost saving. The additional cost of a GP consultation reported in the first scenario is also included as the alternative site of presentation.

Results

Natural history estimates

In addition to informing a cost-effectiveness analysis, the natural history model was used to estimate the incidence and prevalence of undiagnosed cancers and the expected rate of cancer development and diagnosis in England. Table 5 reports the predicted annual incidence and prevalence of undiagnosed NSCLC cases in England, as well as the percentage of the English population that this represents. The age group 85+ years was excluded from the analysis owing to a poor calibration performance of the model in this age group, details of the model fit to the calibrated data can be found in the Supplementary Appendix. These estimates indicate that the model predicted a relatively high prevalence of undiagnosed stage I and II NSCLCs in England (0.24% of the population aged 30–84 years). The apparently low prevalence of undiagnosed stage IIIa disease (1110 of 79 238 undiagnosed cases) is likely explained by the short mean duration of stay in this stage of disease as shown by the high estimated probability of transiting from the state (see Table 4).

Table 5 Estimated incidence and prevalence of prediagnosis NSCLC
Full size table

Base-case cost-effectiveness

Table 6 shows the estimated costs (campaign, diagnosis and treatment), incremental QALYs and resultant ICERs without an intervention and with the regional and national campaigns. The table shows that in both cases the large campaign cost (£2.9 million) was complimented by a relatively small increase in costs of diagnosis and treatment.

Table 6 Cost-effectiveness of the campaigns, relative to no intervention
Full size table

The mortality and HRQoL implications of the campaigns can be combined to consider the QALY implications. In the regional campaign (scaled up to a national level), an incremental gain of 289 QALYs was estimated, and 178 QALYs as a result of the national campaign, as shown in Table 6. These results give ICER of £13 660 per QALY gained for the regional campaign and £18 173 for the national campaign, which are below NICE’s threshold range of £20 000–30 000 per QALY, suggesting that the early awareness campaign was potentially cost-effective to the UK NHS (NICE, 2013). The inclusion of the unstaged patients from the LUCADA data in the stage IIIb and IV patient group had only a small impact on the ICER.

Scenario analysis

As discussed above, two scenario analyses were considered to evaluate the implications of the limitations of the base-case analysis. The first scenario found that, over the period of the campaigns, an increase in unnecessary GP consultations of roughly 18 000 in the national and 35 000 in the regional analyses would have had to be caused by the campaigns to make them not cost-effective at a threshold of £20 000. Recent published results show that, across 486 GP practices (of almost 8000 in England), 21 930 additional GP presentations for coughs occurred during the course of the national campaign (Ironmonger et al, 2014). Clearly, these results suggest that the campaigns are unlikely to be cost-effective if the costs of the worried well population are taken into account. However, this scenario fails to attribute any health benefit of these worried-well patients attending the GP consultations, and as such must be interpreted carefully and warrants further research.

The second scenario found that a saving in the region of 50 000 emergency admissions would have had to occur as a result of the campaigns for them to become cost neutral, that is, as much was saved from reduced emergency admissions as the additional costs incurred by the campaign to the NHS. This is far more than the total number of diagnoses of lung cancer during this period. Ironmonger et al (2014) reported a small decrease in the rate of presentation at emergency departments of 1.9%; however, this does not appear to have been sustained and this analysis represents an approximate estimation of the factors involved.

Discussion

This evaluation has made a significant step forward in our understanding of the potential cost-effectiveness of campaigns to improve early awareness of persistent cough as a symptom of NSCLC. The base-case results suggest that the national campaign could be seen as cost-effective relative to the current NICE threshold range (NICE, 2013). However, the estimates of the ICERs are above the recently estimated cost-effectiveness thresholds empirically estimated from the NHS data (Claxton et al, 2013).

The authors are, however, keen to highlight significant limitations in the analysis, and areas where further research is vital. First, the analysis represents a deterministic approach, with no empirical consideration of the scale or impact of the uncertainty in the natural history model or the results of the early awareness campaign. This was owing to the inability to quantify uncertainty in the effects of an early awareness campaign. Although the regional and national campaigns represent two realisations of the uncertainty, it is not possible to know with certainty the full range of possible outcomes. Therefore, the results presented represent the most robust mean predictions but should not be considered to be certain.

Second, the model fails to take into account important lifestyle and sociodemographic factors known to be associated with the prevalence of lung cancer. Although the incorporation of such factors will not impact the nationwide conclusions drawn from the analysis, it is likely that specific subgroups are associated with very different transition probabilities, and as such the cost-effectiveness of campaigns targeting such groups will vary.

Finally, significant further research is needed to provide more complete and robust estimates of a range of the parameters used to inform this research, primarily the costs associated with lung cancer diagnosis and treatment, the impact of such campaigns on other diagnosis and presentations, and the duration of effect of informative campaigns.

An inevitable implication of the unobservable nature of the natural history model is the difficulty in assessing the validity of the estimated parameters. This is compounded by the lack of suitable data to inform the calibration method, with a reliance on clinical expertise and data collected by the LUCADA data set. Several sources of additional information were explored including the use of lung cancer screening trials (the DANTE, ITALUNG and DSCST trials), which have the potential to strengthen the estimate of the underlying prevalence of undiagnosed disease. However, as the primary aim of the early awareness campaigns is to reach similar levels of early stage diagnosis as other comparable nations, the use of non-English data to inform the model is likely to introduce significant bias into the natural history model and thus any inference of cost-effectiveness. The use of data from the UK Lung Cancer Screening Trial (UKLS) (Baldwin et al, 2011; Field et al, 2011) could be used in the future to strengthen the estimates presented, subject to challenges associated with the use of data from screening trials.

This analysis does, however, represent the first attempt to define the natural history of the disease in its prediagnosis state. This is informative for future public health initiatives as the estimates provide an idea of the undiagnosed diseased population, and therefore the size, stage distribution and age of the population that has the potential to benefit from future public health initiatives.

Although this study found clear evidence of a positive impact of both campaigns, the significant uncertainty associated with many of the model inputs used makes a definitive statement of cost-effectiveness challenging. Further research is needed around many of the variables used in this study if further campaigns to improve the awareness of the signs and symptoms of lung cancer are to be justified as cost-effective.