Drug regulatory agencies have traditionally assessed the quality, safety and efficacy of drugs, and the current paradigm dictates that a new drug should be licensed when the benefits outweigh the risks. By contrast, third-party payers base their reimbursement decisions predominantly on the health benefits of the drug relative to existing treatment options (termed relative efficacy; RE). Over the past decade, the role of payers has become more prominent, and time-to-market no longer means time-to-licensing but time-to-reimbursement. Companies now have to satisfy the sometimes divergent needs of both regulators and payers, and to address RE during the pre-marketing stages. This article describes the current political background to the RE debate and presents the scientific and methodological challenges as they relate to RE assessment. In addition, we explain the impact of RE on drug development, and speculate on future developments and actions that are likely to be required from key players.
This is a preview of subscription content
Subscribe to Journal
Get full journal access for 1 year
only $4.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Centers for Medicare & Medicaid Services. National health expenditure projections 2008–2018. CMS website [online], (2009).
Edmonds, P., Glynn, D. & Oglialoro, C. Access to important new medicines. Eur. Bus. J. 12, 146–157 (2000).
European Commission. Regulation (EC) No 726/2004 of the European Parliament and of the Council of 31 March 2004 laying down community procedures for the authorisation and supervision of medicinal products for human and veterinary use and establishing a European Medicines Agency. EC website [online], (2004).
Rawlins, M. The decade of NICE. Lancet 374, 351–352 (2009).
Centers for Medicare & Medicaid Services. Medicare posts final national coverage determination for the use of erythropoiesis stimulating agents in cancer and related neoplastic conditions. CMS website [online], (2007).
National Pharmacy and Therapeutics Committee. WellPoint Health Technology Assessment Guidelines. Drug submission guidelines for re-evaluation indications, and formulations. WellPointNextRx website [online], (2008).
Institute for Quality and Efficiency in Health Care. Methods for assessment of the relation of benefits to costs in the German statutory health care system. Version 1.1. IQWiG website [online], (2008).
European Mecdicines Agency, Committee for Medicinal Products for Human Use. Reflection paper on the regulatory guidance for the use of health related quality of life (HRQL) measures in the evaluation of medicinal products. EMA website [online], (2005).
Papanicolaou, S., Sykes, D. & Mossialos, E. A. EMEA and the evaluation of health-related quality of life data in the regulatory process. Int. J. Technol. Assess. Health Care 20, 311–324 (2004).
Gottlieb, S. & Klasmeier, C. Comparative effectiveness research: the need for a uniform standard. American Enterprise Institute for Public Policy Research website [online], (2009).
Eichler, H. G., Pignatti, F., Leufkens, H. & Breckenridge, A. Balancing early market access to new drugs with the need for benefit/risk data: a mounting dilemma. Nature Rev. Drugs Discov. 7, 818–826 (2008).
Steinke, S. Are FDA and CMS growing apart? AEI's Gottlieb thinks so. “The Pink Sheet” (Bridgewater, NJ) 27–29 (1 Dec 2008).
European Social Insurance Platform. Position of the Medicine Evaluation (MEDEV) Committee regarding the proposal for a regulation of the European Parliament and of the Council, on medicinal products for paediatric use and amending Regulation (EEC) No 1768/92, Directive 2001/83/EC and Regulation (EC) No 726/2004. COM (2004) 599. ESIP website [online], (2005).
Kraemer, H. C., Glick, I. D. & Klein, D. F. Clinical trials design lessons from the CATIE study. Am. J. Psychiatry 166, 1222–1228 (2009).
Lieberman, J. A. et al. Effectiveness of antipsychotic drugs in patients with chronic schizophrenia. N. Engl. J. Med. 35 1209–1223 (2005).
Kastelein, J. J. et al. Simvastatin with or without ezetimibe in familial hypercholesterolemia. N. Engl. J. Med. 358, 1431–1443 (2008).
O'Riordan, M. Congress sends letters to ACC and AHA over ENHANCE press release and industry funding. TheHeart.org website [online], (2008).
van Luijn, J. C., Gribnau, F. W. & Leufkens, H. G. Availability of comparative trials for the assessment of new medicines in the European Union at the moment of market authorization. Br. J. Clin. Pharmacol. 63, 159–162 (2006).
Barbui, C. & Garattini, S. Regulatory policies on medicines for psychiatric disorders: is Europe on target? Br. J. Psychiatry 190, 91–93 (2007).
Schumock, G. T. & Pickard, S. Comparative effectiveness research: relevance and applications to pharmacy. Am. J. Health Syst. Pharm. 66, 1278–1286 (2009).
Angell, M. The Truth About the Drug Companies: How They Deceive Us and What to Do About It 1–352 (Random House, New York, 2005).
Garatini, S. & Bertele, V. How can we regulate medicines better? BMJ 335, 803–805 (2007).
European Commission. Final conclusions and recommendations of the High Level Pharmaceutical Forum. EC Pharmaceutical Forum website [online], (2008).
Congress of the United States Congressional Budget Office. A CBO Paper. Research on the comparative effectiveness of medical treatments. CBO website [online], (2007).
Hughes, B. The comparative effectiveness challenge. Nature Rev. Drug Discov. 8, 261–263 (2009).
Institute of Medicine of the National Academies. 100 initial priority topics for comparative effectiveness research. Institute of Medicine website [online], (2009).
Institute of Medicine of the National Academies. To err is human: building a safer health system. Institute of Medicine website [online], (1999).
The Centre for Evidence-based Medicine. Oxford Centre for Evidence-based Medicine — levels of evidence (March 2009). CEBM website [online], (2009).
Glenny, A. M. et al. Indirect comparisons of competing interventions. Health Technol. Assess. 9, 1–134 (2005).
McAlister, F. A., Laupacis, A., Wells, G. A. & Sackett, D. L. Users' guides to the medical literature XIX applying clinical trials results, B. Guidelines for determining whether a drug is exerting (more than) a class effect. JAMA 282, 1371–1377 (1999).
Australian Government Department of Health and Ageing. Report of the Indirect Comparisons Working Group to the Pharmaceutical Benefits Advisory Committee: assessing indirect comparisons. Australian Government Department of Health and Ageing website [online], (2009).
European Social Insurance Platform. Response to the European Commission public consultation on the future of pharmaceuticals for human use in Europe. ESIP website [online], (2007).
International Conference on Harmonisation. ICH harmonised tripartite guideline. Choice of control group and related issues in clinical trials E10. Current Step 4 version dated 20 July 2000. ICH website [online], (2000).
European Medicines Agency. European Public Assessment Report: Ketek. EMA website [online], (2009).
Heres, S. et al. Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. Am. J. Psychiatry 163, 185–194 (2006).
Song, F., Harvey, I. & Lilford, R. Adjusted indirect comparison may be less biased than direct comparison for evalurating new pharmaceutical interventions. J. Clin. Epidemiol. 61, 455–463 (2008).
European Mecdicines Agency, Committee for Medicinal Products for Human Use. Guideline on the choice of the non-inferiority margin. EMA website [online], (2005).
Schwartz, D. & Lellouc, J. Explanatory and pragmatic attitudes in therapeutic trials. J. Clin. Epidemiol. 62, 499–505 (2009).
Tunis, S. R., Stryer, D. B. & Clancy, C. M. Practical clinical trials. Increasing the value of clinical research for decision making in clinical and health policy. JAMA 290, 1624–1632 (2003).
Thorpe, K. E. et al. A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J. Clin. Epidemiol. 62, 464–475 (2009).
Luce, B. R. et al. Rethinking randomized clinical trials for comparative effectiveness research: the need for transformational change. Ann. Intern. Med. 151, 206–209 (2009).
Bucher, H. C., Guyatt, G. H., Griffith LE & Walter, S. D. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J. Clin. Epidemiol. 50, 683–691 (1997).
Lumley, T. Network meta-analysis for indirect treatment comparisons. Stat. Med. 21, 2313–2324 (2002).
Lu, G. & Ades, A. E. Combination of direct and indirect evidence in mixed treatment comparisons. Stat. Med. 23, 3105–3124 (2004).
Gartlehner, G. & Moore, C. G. Direct versus indirect comparisons: a summary of the evidence. Int. J. Techn. Assess. Health Care 24, 170–177 (2008).
Cipriani, A. et al. Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis. Lancet 373, 746–758 (2009).
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A. & Rosenthal, R. Selective publication of antidepressant trials and its influence on apparent efficacy. N. Engl. J. Med. 17, 252–260 (2008).
Sridharan, L. & Greenland, P. The importance of negative studies. Arch. Intern. Med. 169, 1022–1023 (2009).
Song, F. et al. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ 3 April 2009 (doi:10.1136/bmj.b1147).
Editorial. A double-edged sword. Nature Rev. Drug Discov. 7, 275 (2008).
Vandenbroucke, J. Observational research, randomised trials and two views of medical science. PLoS Med. 5, e67 (2008).
Schneeweiss, S. Developments in post-marketing comparative effectiveness research. Clin. Pharm. Ther. 82, 143–156 (2007).
McMahon, A. D. Observation and experiment with the efficacy of drugs; a warning example from a cohort of nonsteroidal anti-inflammarory and ulcer-healing drug users. Am. J. Epidemiol. 154, 557–562 (2001).
The National Collaborating Centre for Chronic Conditions. Hypertension. Management in adults in primary care: pharmacological update. National Institute for Health and Clinical Excellence website [online], (2006).
US Food and Drug Administration, Center for Drug Evaluation and Reasearch. Application Number 22-055. Medical Review: Altabax. FDA website [online], (2007).
European Medicines Agency, Committee for Medicinal Products for Human Use. Guideline on the evaluation of anticancer medicinal products in man. EMA website [online], (2005).
Scottish Medicines Consortium. Guidance to manufacturers for completion of new product assessment form (NPAF) (Revised June 2007). SMC website [online], (2007).
Peters, A. German health insurers launch study to compare Avastin to Lucentis in AMD. AMP Health Europe website [online], (2007).
Horsley, W. for North East Treatment Advisory Group. Bevacizumab (Avastin®) and ranibizumab (Lucentis®) in the management of non-AMD choroidal neovascular disease. NETAG website [online], (2009).
Bach, P. B. Limits of Medicare's ability to control rising spending on cancer drugs. N. Engl. J. Med. 360, 626–633 (2009).
The ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group. Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs diuretic. JAMA 288, 2981–2997 (2002).
European Commission. Commission Directive 2003/63/EC of 25 June amending Directive 2001/83/EC of the European Parliament and of the Council on the Community code relating to medicinal products for human use. European Medicines Agency website [online], (2003).
Kasper, A. Payers having more impact on R&D, Pfizer says: asenapine is early casualty. “The Pink Sheet” (Bridgewater, NJ) 10 (18 Dec 2006).
Sean, R. Tunis and Steven, D. Pearson. Coverage options for promising technologies: Medicare's 'coverage with evidence development'. Health Affairs 25, 1218–1230 (2006).
Centers for Medicare & Medicaid Services. Guidance for the public, industry, and CMS staff national coverage determinations with data collection as a condition of coverage: coverage with evidence development. Document issued on July 12, 2006. CMS website [online], (2006).
Garber, A. M. & Tunis, S. R. Does comparative–effectiveness research threaten personalized medicine? N. Engl. J. Med. 360, 1925–1927 (2009).
Thomson Reuters. Thought Leadership Slides. R&D Expenditure 1997–2007 (slide 30 of 49). Thomson Reuters website [online], (2009).
Avorn, J. In defense of pharmacoepidemiology — embracing the Yin and Yang of drug research. N. Engl. J. Med. 357, 2219–2221 (2007).
Eisenstein, E. L. et al. Reducing the costs of phase III cardiovascular clinical trials. Am. Heart J. 149, 482–488 (2005).
von Elm, E. et al. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 335, 806–808 (2007).
US Food and Drug Administration. FDA's Sentinel Initiative. FDA website [online], (2010).
European Medicines Agency. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). ENCePP website [online], (2010).
European Medicines Agency. Press Release: EMEA-coordinated PROTECT project has been accepted for funding by the Innovative Medicines Initiative Joint Undertaking. EMA website [online], (2009).
Miller, F. G. & Steven, D. Pearson, S. D. Coverage with evidence development. Ethical issues and policy implications. Medical Care 46, 746–751 (2008).
European Commission. High Level Pharmaceutical Forum. Core principles on relative effectiveness. EC Pharmcacuetical Forum website [online], (2008).
International Network of Agencies for Health Technology Assessment. INAHTA Health Technology Assessment (HTA) glossary. INAHTA website [online], (2010).
US Department of Health and Human Services. Draft definition of comparative effectiveness research for the Federal Coordinating Council. HHS website [online], (2010).
Sox, H. C. & Greenfield, S. Comparative effectiveness research: a report from the Institute of Medicine. Ann. Int. Med. 151, 203–205 (2009).
Center for Medical Technology Policy. Comparative effectiveness definitions. CMTP website [online], (2009).
Bureau Européen des Unions de Consommateurs (BEUC). EMEA Working Group with Patient Organisations. Beuc input to individual sub-groups. Transparency/informtion and dissemination of information. European Medicines Agency website [online], (2009).
European Commission. Regulation (EC) no 141/2000 of the European Parliament and of the Council of 16 December 1999 on orphan medicinal products. EC website [online], (2000).
European Commission. Commission Regulation (EC) No 847/2000 of 27 April 2000 laying down the provisions for implementation of the criteria for designation of a medicinal product as an orphan medicinal product and definitions of the concepts 'similar medicinal product' and 'clinical superiority'. EC website [online], (2000).
European Medicines Agency. ICH Topic E 9 statistical principles for clinical trials. Step 5. Note for guidance on statistical principles for clinical trials (CPMP/ICH/363/96). EMA website [online], (1998).
National Instruments. FDA validation of medical devices with National Instruments hardware and software — FAQ. NI website [online], (2010).
US Department of Health and Human Services, Centers for Disease Control. Appendix D: Glossary of HIV prevention terms. CDC website [online], (2006).
Bauer, P. & Kieser, M. Combining different phases in the development of medical treatments within a single trial. Stat. Med. 18, 1833–1848 (1999).
Bretz, F., Koenig, F., Brannath, W., Glimm, E. & Posch, M. Adaptive designs for confirmatory clinical trials. Stat. Med. 28, 1181–1217 (2009).
European Medicines Agency, Committee for Medicinal Products for Human Use. Reflection paper on methodological issues in confirmatory clinical trials planned with an adaptive design. EMA website [online], (2007).
The authors declare no competing financial interests.
About this article
Cite this article
Eichler, HG., Bloechl-Daum, B., Abadie, E. et al. Relative efficacy of drugs: an emerging issue between regulatory agencies and third-party payers. Nat Rev Drug Discov 9, 277–291 (2010). https://doi.org/10.1038/nrd3079
Nature Reviews Clinical Oncology (2022)