Market access and coverage policies for health technologies should ideally be based on clinical trials that assess final outcomes relevant to patients, such as survival, morbidity and health-related quality of life. Nevertheless, growing pressure for faster access to innovative treatments for patients in the past two decades has led to the introduction of various regulatory initiatives intended to facilitate this (Characterizing the US FDA's approach to promoting transformative innovation.Nat. Rev. Drug Discov. 14, 740–741 (2015))1. Consequently, regulatory authorities and payers often have to base their decisions about the use of a technology on surrogate outcomes, which allow trials to be done less expensively with fewer patients in a relatively short period. For example, more than 40% of trials used as the basis for approval of new indications by the US Food and Drug Administration (FDA) between 2005 and 2012 had a primary outcome that was a surrogate end point2. Furthermore, evidence from surrogate end points may not only expedite the regulatory approval of new health technologies but also inform coverage and reimbursement decisions. Over the past decade, between 27% and 50% of submissions to the National Institute for Health and Care Excellence (NICE) in the United Kingdom, the Pharmaceutical Benefits and Advisory Committee in Australia and the Common Drug Review in Canada were based on surrogate end points3.
However, relying on surrogate end points can pose several serious problems for health-care decision makers. First, surrogate end points may not capture the combined risk–benefit profile of a health technology4. Some drugs approved on the basis of surrogate end points have subsequently been associated with serious safety problems and have had to be withdrawn from the market or have their indications substantially restricted. Even if the surrogate lies in the only causal pathway of the disease process, reliance on surrogate end points can often lead to overestimation of the magnitude of the treatment effect on the final end point5.
Second, the use of surrogate end point data requires decision makers to extrapolate beyond the observed findings in order to estimate the expected true benefits to patients and health systems. Thus, clinical superiority on a surrogate end point may not necessarily translate into benefits that are good value for money for health care systems. This can be illustrated by the example of dasatinib, which has been approved by the European Medicines Agency for the treatment of chronic myeloid leukaemia. This approval was based on a trial showing superior confirmed complete cytogenetic response by 12 months for dasatinib versus imatinib (77% versus 66%, p = 0.007)6. However, an assessment of the drug by the NICE concluded that the estimated incremental gain in survival (22.7 years versus 21.3 years) extrapolated from the observed improvement on the surrogate end point came at a patient cost in excess of €200,000 per quality-adjusted life year. As a result of this assessment, NICE did not recommend coverage of the drug6.
In order for regulatory authorities and payers to use a surrogate end point with confidence, a validation process for such end points is needed. Here, we present a three-step framework for the validation and appropriate use of surrogate end points in both licensing and coverage or reimbursement decisions (Fig. 1).
Establish the level of evidence. The first step is to consider the hierarchy of available evidence7. The biological plausibility of the relationship between the surrogate end point and final outcome is necessary but not sufficient. Evidence is considered to be 'level 2' when a strong correlation exists between the surrogate and the final end point across cohorts or at the level of the individual patient. However, individual patient correlations do not provide the highest level of evidence in order to validate surrogate measures, although they may identify good prognostic markers8. 'Level 1' evidence requires demonstration of the relationship between the treatment effect on the surrogate end point and the final outcomes, preferably across multiple randomized trials. Trial-based evidence of a final outcome is usually not available for a new health-care technology for which surrogates are used, so this evidence needs to be sourced from other trials of the same or a similar technology — for example, trials should be of drugs from the same class or, if such evidence is not is available, drugs from a different class.
Assess the strength of the association. The second step is to assess the strength of the association between the surrogate end point and the final outcome. Among several approaches to address this issue, regression-based and meta-analytic approaches dominate the field. The most reliable approach is to perform a meta-analysis using patient-level data from all randomized trials of a treatment. When patient-level data are available, two levels of association can be estimated: the association between the surrogate and the final outcome, and the association between the effect of treatment on the surrogate and the final outcome. Thresholds set to identify good surrogates can be as high as 0.8 for correlation coefficients (ρ) or 0.65 for coefficients of determination (R2), which are particularly strict rules for the acceptability of putative surrogate end points when applied in practice9.
Quantify the relationship between the surrogate and the final outcome. The final step relates to predicting and quantifying the effect on the final outcome based on the observed effect on the surrogate. A quantitative approach has been proposed that consists of estimating the surrogate threshold effect (STE), which is the magnitude of treatment effect on the surrogate that would predict a significant treatment effect on the final outcome10. This is crucial for decisions on coverage and reimbursement. Regulators usually focus on early evidence of safety and efficacy to determine if the balance of benefits and risks is positive when informing the design of registration trials, whereas reimbursement agencies usually consider long-term effectiveness or cost effectiveness. Whether decisions on market access and reimbursement are based on a formal economic evaluation or on the magnitude of the clinical benefit, the effect of the treatment on the surrogate end point needs to be large enough to predict an improvement in the final outcome of interest before the technology can be concluded to be cost effective.
To date, few empirical assessments have investigated the adequacy of evidence for specific surrogate end points or groups of surrogates, particularly in terms of reimbursement policy. One exception is oncology, for which there is a long tradition of using surrogates. With a few exceptions, such as metastatic colorectal or ovarian cancer, the strength of the associations between the surrogate and final outcomes has tended to be relatively low in studies so far.
In conclusion, surrogates can result in market access for technologies that turn out to offer no true health benefit — or even cause harm —and can result in overestimation of treatment effects (and economic value), which can lead to inappropriate decisions on coverage. However, the use of appropriately validated surrogate end points within a consistent framework provides substantial potential to speed up access to innovative technologies that offer important value for patients and healthcare systems and to improve efficiency and equity within the R&D environment.
The views expressed in this article are the personal views of the authors and may not be understood or quoted as being made on behalf of or reflecting the position of the agencies or organizations with which the authors are affiliated.
Downing, N. S., Krumholz, H. M., Ross, J. S. & Shah, N. D. Characterizing the US FDA's approach to promoting transformative innovation. Nat. Rev. Drug Discov. 14, 740–741 (2015).
Downing, N. S., Aminawung, J. A., Shah, N. D., Krumholz, H. M. & Ross, J. S. Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005–2012. JAMA 311, 368–377 (2014).
Clement, F. M. et al. Using effectiveness and cost-effectiveness to make drug coverage decisions: a comparison of Britain, Australia, and Canada. JAMA 302, 1437–1443 (2009).
Fleming, T. R. & DeMets, D. L. Surrogate end points in clinical trials: are we being misled? Ann. Intern. Med. 125, 605–613 (1996).
Ciani, O. et al. Comparison of treatment effect sizes associated with surrogate and final patient relevant outcomes in randomised controlled trials: meta-epidemiological study. BMJ 346, f457 (2013).
Ciani, O. et al. Complete cytogenetic response and major molecular response as surrogate outcomes for overall survival in first-line treatment of chronic myelogenous leukemia: a case study for technology appraisal on the basis of surrogate outcomes evidence. Value Health 16, 1081–1090 (2013).
Bucher, H. C., Guyatt, G. H., Cook, D. J., Holbrook, A. & McAlister, F. A. Users' guides to the medical literature: XIX. Applying clinical trial results. A. How to use an article measuring the effect of an intervention on surrogate end points. Evidence-Based Medicine Working Group. JAMA 282, 771–778 (1999).
Buyse, M., Sargent, D. J., Grothey, A., Matheson, A. & de Gramont, A. Biomarkers and surrogate end points — the challenge of statistical validation. Nat. Rev. Clin. Oncol. 7, 309–317 (2010).
Ciani, O. et al. Validation of surrogate endpoints in advanced solid tumours: systematic review of statistical methods, results, and implications for policy makers. Int. J. Technol. Assess. Health Care 30, 13 (2014).
Burzykowski, T. & Buyse, M. Surrogate threshold effect: an alternative measure for meta-analytic surrogate endpoint validation. Pharm. Stat. 5, 173–186 (2006).
O.C. is funded by a postdoctoral research fellowship by the University of Exeter Medical School.
M.B. declares an association with the International Drug Development Institute Inc. and CluePoints Inc., E.D.S. declares an association with Dendrix Ltd.
Rights and permissions
About this article
Cite this article
Ciani, O., Buyse, M., Drummond, M. et al. Use of surrogate end points in healthcare policy: a proposal for adoption of a validation framework. Nat Rev Drug Discov 15, 516 (2016). https://doi.org/10.1038/nrd.2016.81
This article is cited by
Impact of Managed Entry Agreements on availability of and timely access to medicines: an ex-post evaluation of agreements implemented for oncology therapies in four countries
BMC Health Services Research (2022)
A perspective on life-cycle health technology assessment and real-world evidence for precision oncology in Canada
npj Precision Oncology (2022)
European union regulation of health technology assessment: what is required for it to succeed?
The European Journal of Health Economics (2022)
Meta-regression of randomized control trials with antithrombotics: weak correlation between net clinical benefit and all cause-mortality
Scientific Reports (2021)