Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Analysis
  • Published:

Efficacy outcomes in phase 2 and phase 3 randomized controlled trials in rheumatology

Nature Medicine volume 26, pages 974–980 (2020)Cite this article

Subjects

Abstract

Phase 3 trials are the mainstay of drug development across medicine but have often not met expectations set by preceding phase 2 studies. A systematic meta-analysis evaluated all randomized controlled, double-blind trials investigating targeted disease-modifying anti-rheumatic drugs in rheumatoid and psoriatic arthritis. Primary outcomes of American College of Rheumatology (ACR) 20 responses were compared by mixed-model logistic regression, including exploration of potential determinants of efficacy overestimation. In rheumatoid arthritis, phase 2 trial outcomes systematically overestimated subsequent phase 3 results (odds ratio comparing ACR20 in phase 2 versus phase 3: 1.39, 95% confidence interval: 1.25–1.57, P < 0.001). Data for psoriatic arthritis trials were similar, but not statistically significant (odds ratio comparing ACR20 in phase 2 versus phase 3: 1.35, 95% confidence interval: 0.94–1.94, P = 0.09). Differences in inclusion criteria largely explained the observed differences in efficacy findings. Our findings have implications for all stakeholders in new therapeutic development and testing, as well as potential ethical implications.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1
Fig. 2: Descriptive comparison of ACR response rates of phase 2 and phase 3 studies.
Fig. 3: Mixed-model analysis for comparison of phase 2 versus phase 3 clinical effects in rheumatoid arthritis.
Fig. 4: Results of psoriatic arthritis trials confirm data obtained in rheumatoid arthritis.
Fig. 5: Exploratory analysis of efficacy bias between phase 2 and phase 3 clinical trials in rheumatoid arthritis.

Similar content being viewed by others

Data availability

Due to the nature of this analysis, the data that support the findings of this study were published previously. Primary outcomes used for the analyses of this study are shown in the Supplementary Information. All references for the trials included for analyses are also listed in the Supplementary Information.

References

  1. Umscheid, C. A., Margolis, D. J. & Grossman, C. E. Key concepts of clinical trials: a narrative review. Postgrad. Med. 123, 194–204 (2011).

    PubMed  PubMed Central  Google Scholar 

  2. Genovese, M. C. et al. A phase 2 dose-ranging study of subcutaneous tabalumab for the treatment of patients with active rheumatoid arthritis and an inadequate response to methotrexate. Ann. Rheum. Dis. 72, 1453–1460 (2013).

    CAS  PubMed  Google Scholar 

  3. Breneman, D. et al. Phase 1 and 2 trial of bexarotene gel for skin-directed treatment of patients with cutaneous T cell lymphoma. Arch. Dermatol. 138, 325–332 (2002).

    CAS  PubMed  Google Scholar 

  4. Schreiber, S. et al. A randomized, placebo-controlled trial of certolizumab pegol (CDP870) for treatment of Crohn’s disease. Gastroenterology 129, 807–818 (2005).

    CAS  PubMed  Google Scholar 

  5. Comi, G. et al. Effect of laquinimod on MRI-monitored disease activity in patients with relapsing-remitting multiple sclerosis: a multicentre, randomised, double-blind, placebo-controlled phase IIb study. Lancet 371, 2085–2092 (2008).

    CAS  PubMed  Google Scholar 

  6. Heald, P. et al. Topical bexarotene therapy for patients with refractory or persistent early-stage cutaneous T cell lymphoma: results of the phase III clinical trial. J. Am. Acad. Dermatol. 49, 801–815 (2003).

    PubMed  Google Scholar 

  7. Sandborn, W. J. et al. Certolizumab pegol for active Crohn’s disease: a placebo-controlled, randomized trial. Clin. Gastroenterol. Hepatol. 9, 670–678 (2011).

    CAS  PubMed  Google Scholar 

  8. Vollmer, T. L. et al. A randomized placebo-controlled phase III trial of oral laquinimod for multiple sclerosis. J. Neurol. 261, 773–783 (2014).

    CAS  PubMed  Google Scholar 

  9. Tang, M. et al. Correlation of phase 2 trials (Ph2t) results with outcomes of phase 3 trials (Ph3t) of investigational agents (IA) in locally advanced and metastatic pancreas cancer (LAMPC). J. Clin. Oncol. 35, 227–227 (2017).

    Google Scholar 

  10. Zia, M. I., Siu, L. L., Pond, G. R. & Chen, E. X. Comparison of outcomes of phase II studies and subsequent randomized control studies using identical chemotherapeutic regimens. J. Clin. Oncol. 23, 6982–6991 (2005).

    CAS  PubMed  Google Scholar 

  11. Smolen, J. S. et al. Efficacy and safety of tabalumab, an anti-B-cell-activating factor monoclonal antibody, in patients with rheumatoid arthritis who had an inadequate response to methotrexate therapy: results from a phase III multicentre, randomised, double-blind study. Ann. Rheum. Dis. 74, 1567–1570 (2015).

    CAS  PubMed  Google Scholar 

  12. Genovese, M. C. et al. A phase III, multicenter, randomized, double-blind, placebo-controlled, parallel-group study of 2 dosing regimens of fostamatinib in patients with rheumatoid arthritis with an inadequate response to a tumor necrosis factor-α antagonist. J. Rheumatol. 41, 2120–2128 (2014).

    CAS  PubMed  Google Scholar 

  13. Weinblatt, M. E. et al. Effects of fostamatinib, an oral spleen tyrosine kinase inhibitor, in rheumatoid arthritis patients with an inadequate response to methotrexate: results from a phase III, multicenter, randomized, double-blind, placebo-controlled, parallel-group study. Arthritis Rheumatol. 66, 3255–3264 (2014).

    CAS  PubMed  Google Scholar 

  14. Liang, F. et al. Comparison of treatment effect from randomised controlled phase II trials and subsequent phase III trials using identical regimens in the same treatment setting. Eur. J. Cancer 121, 19–28 (2019).

    PubMed  Google Scholar 

  15. Kirby, S., Burke, J., Chuang-Stein, C. & Sin, C. Discounting phase 2 results when planning phase 3 clinical trials. Pharm. Stat. 11, 373–385 (2012).

    CAS  PubMed  Google Scholar 

  16. Wang, S.-J., Hung, H. M. J. & O’Neill, R. T. Adapting the sample size planning of a phase III trial based on phase II data. Pharm. Stat. 5, 85–97 (2006).

    CAS  PubMed  Google Scholar 

  17. Aletaha, D. et al. Reporting disease activity in clinical trials of patients with rheumatoid arthritis: EULAR/ACR collaborative recommendations. Ann. Rheum. Dis. 67, 1360–1364 (2008).

    CAS  PubMed  Google Scholar 

  18. Felson, D. T. et al. American College of Rheumatology. Preliminary definition of improvement in rheumatoid arthritis. Arthritis Rheum. 38, 727–735 (1995).

    CAS  PubMed  Google Scholar 

  19. Maini, R. N. et al. Therapeutic efficacy of multiple intravenous infusions of anti-tumor necrosis factor α monoclonal antibody combined with low-dose weekly methotrexate in rheumatoid arthritis. Arthritis Rheum. 41, 1552–1563 (1998).

    CAS  PubMed  Google Scholar 

  20. Burmester, G. R. et al. Efficacy and safety of mavrilimumab in subjects with rheumatoid arthritis. Ann. Rheum. Dis. 72, 1445–1452 (2013).

    CAS  PubMed  Google Scholar 

  21. Takeuchi, T. et al. Efficacy and safety of mavrilimumab in Japanese subjects with rheumatoid arthritis: findings from a phase IIa study. Mod. Rheumatol. 25, 21–30 (2015).

    CAS  PubMed  Google Scholar 

  22. Burmester, G. R. et al. A randomised phase IIb study of mavrilimumab, a novel GM-CSF receptor α monoclonal antibody, in the treatment of rheumatoid arthritis. Ann. Rheum. Dis. 76, 1020–1030 (2017).

    CAS  PubMed  Google Scholar 

  23. Fleischmann, R. M. et al. A randomized, double-blind, placebo-controlled, twelve-week, dose-ranging study of decernotinib, an oral selective JAK-3 inhibitor, as monotherapy in patients with active rheumatoid arthritis. Arthritis Rheumatol. 67, 334–343 (2015).

    CAS  PubMed  Google Scholar 

  24. Genovese, M. C., van Vollenhoven, R. F., Pacheco-Tena, C., Zhang, Y. & Kinnman, N. VX-509 (decernotinib), an oral selective JAK-3 inhibitor, in combination with methotrexate in patients with rheumatoid arthritis. Arthritis Rheumatol. 68, 46–55 (2016).

    CAS  PubMed  Google Scholar 

  25. Genovese, M. C., Yang, F., Østergaard, M. & Kinnman, N. Efficacy of VX-509 (decernotinib) in combination with a disease-modifying antirheumatic drug in patients with rheumatoid arthritis: clinical and MRI findings. Ann. Rheum. Dis. 75, 1979–1983 (2016).

    CAS  PubMed  Google Scholar 

  26. Genovese, M. C. et al. A phase II randomized study of subcutaneous ixekizumab, an anti-interleukin-17 monoclonal antibody, in rheumatoid arthritis patients who were naive to biologic agents or had an inadequate response to tumor necrosis factor inhibitors. Arthritis Rheumatol. 66, 1693–1704 (2014).

    CAS  PubMed  Google Scholar 

  27. Genovese, M. C. et al. Efficacy and safety of olokizumab in patients with rheumatoid arthritis with an inadequate response to TNF inhibitor therapy: outcomes of a randomised phase IIb study. Ann. Rheum. Dis. 73, 1607–1615 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Takeuchi, T. et al. Efficacy and safety of olokizumab in Asian patients with moderate-to-severe rheumatoid arthritis, previously exposed to anti-TNF therapy: results from a randomized phase II trial. Mod. Rheumatol. 26, 15–23 (2016).

    CAS  PubMed  Google Scholar 

  29. Mease, P. et al. A phase II, double-blind, randomised, placebo-controlled study of BMS945429 (ALD518) in patients with rheumatoid arthritis with an inadequate response to methotrexate. Ann. Rheum. Dis. 71, 1183–1189 (2012).

    CAS  PubMed  Google Scholar 

  30. Weinblatt, M. E. et al. The efficacy and safety of subcutaneous clazakizumab in patients with moderate-to-severe rheumatoid arthritis and an inadequate response to methotrexate: results from a multinational, phase IIb, randomized, double-blind, placebo/active-controlled, dose-ranging study. Arthritis Rheumatol. 67, 2591–2600 (2015).

    PubMed  Google Scholar 

  31. Mease, P. J. et al. Phase II study of ABT-122, a tumor necrosis factor- and interleukin-17A-targeted dual variable domain immunoglobulin, in patients with psoriatic arthritis with an inadequate response to methotrexate. Arthritis Rheumatol. 70, 1778–1789 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. Mease, P. J. et al. The efficacy and safety of clazakizumab, an anti-interleukin-6 monoclonal antibody, in a phase IIb study of adults with active psoriatic arthritis. Arthritis Rheumatol. 68, 2163–2173 (2016).

    CAS  PubMed  Google Scholar 

  33. Abstracts of the American College of Rheumatology 66th Annual Meeting and the Association of Rheumatology Health Professionals 37th Annual Meeting. October 24–29, 2002. New Orleans, Louisiana, USA. Arthritis Rheum. 46, S33–677 (2002).

  34. Weinblatt, M. E. et al. Adalimumab, a fully human anti-tumor necrosis factor-α monoclonal antibody, for the treatment of rheumatoid arthritis in patients taking concomitant methotrexate: the ARMADA trial. Arthritis Rheum. 48, 35–45 (2003).

    CAS  PubMed  Google Scholar 

  35. Fleischmann, R. et al. Efficacy and safety of certolizumab pegol monotherapy every 4 weeks in patients with rheumatoid arthritis failing previous disease-modifying antirheumatic therapy: the FAST4WARD study. Ann. Rheum. Dis. 68, 805–811 (2009).

    CAS  PubMed  Google Scholar 

  36. Emery, P. et al. Efficacy and safety of different doses and retreatment of rituximab: a randomised, placebo-controlled trial in patients who are biological naive with active rheumatoid arthritis and an inadequate response to methotrexate (Study Evaluating Rituximab’s Efficacy in MTX iNadequate rEsponders (SERENE)). Ann. Rheum. Dis. 69, 1629–1635 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  37. Weinblatt, M. E. et al. Treatment of rheumatoid arthritis with a Syk kinase inhibitor: a twelve-week, randomized, placebo-controlled trial. Arthritis Rheum. 58, 3309–3318 (2008).

    CAS  PubMed  Google Scholar 

  38. Mease, P. J. et al. Brodalumab, an anti-IL17RA monoclonal antibody, in psoriatic arthritis. N. Engl. J. Med. 370, 2295–2306 (2014).

    PubMed  Google Scholar 

  39. Baeten, D. et al. Anti-interleukin-17A monoclonal antibody secukinumab in treatment of ankylosing spondylitis: a randomised, double-blind, placebo-controlled trial. Lancet 382, 1705–1713 (2013).

    CAS  PubMed  Google Scholar 

  40. McInnes, I. B. et al. Efficacy and safety of secukinumab, a fully human anti-interleukin-17A monoclonal antibody, in patients with moderate-to-severe psoriatic arthritis: a 24-week, randomised, double-blind, placebo-controlled, phase II proof-of-concept trial. Ann. Rheum. Dis. 73, 349–356 (2014).

    CAS  PubMed  Google Scholar 

  41. Smolen, J. S., Weinblatt, M. E., Sheng, S., Zhuang, Y. & Hsu, B. Sirukumab, a human anti-interleukin-6 monoclonal antibody: a randomised, 2-part (proof-of-concept and dose-finding), phase II study in patients with active rheumatoid arthritis despite methotrexate therapy. Ann. Rheum. Dis. 73, 1616–1625 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  42. Weinblatt, M. E. et al. An oral spleen tyrosine kinase (Syk) inhibitor for rheumatoid arthritis. N. Engl. J. Med. 363, 1303–1312 (2010).

    CAS  PubMed  Google Scholar 

  43. Genovese, M. C. et al. An oral Syk kinase inhibitor in the treatment of rheumatoid arthritis: a three-month randomized, placebo-controlled, phase II study in patients with active rheumatoid arthritis that did not respond to biologic agents. Arthritis Rheum. 63, 337–345 (2011).

    CAS  PubMed  Google Scholar 

  44. Whittaker, S. et al. Efficacy and safety of bexarotene combined with psoralen-ultraviolet A (PUVA) compared with PUVA treatment alone in stage IB-IIA mycosis fungoides: final results from the EORTC Cutaneous Lymphoma Task Force phase III randomized clinical trial (NCT00056056). Br. J. Dermatol. 167, 678–687 (2012).

    CAS  PubMed  Google Scholar 

  45. Furie, R. et al. Anifrolumab, an anti-interferon-α receptor monoclonal antibody, in moderate-to-severe systemic lupus erythematosus. Arthritis Rheumatol. 69, 376–386 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  46. AstraZeneca. Update on TULIP 1 phase III trial for anifrolumab in systemic lupus erythematosus. https://www.astrazeneca.com/media-centre/press-releases/2018/update-on-tulip-1-phase-iii-trial-for-anifrolumab-in-systemic-lupus-erythematosus-31082018.html (2018).

  47. Dechartres, A., Trinquart, L., Boutron, I. & Ravaud, P. Influence of trial sample size on treatment effect estimates: meta-epidemiological study. Brit. Med. J. 346, f2304 (2013).

    PubMed  Google Scholar 

  48. Grayling, M. J., Dimairo, M., Mander, A. P. & Jaki, T. F. A review of perspectives on the use of randomization in phase II oncology trials. J. Natl Cancer Inst. 111, 1255–1262 (2019).

    PubMed  PubMed Central  Google Scholar 

  49. Arnett, F. C. et al. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum. 31, 315–324 (1988).

    CAS  PubMed  Google Scholar 

  50. Aletaha, D. et al. 2010 Rheumatoid arthritis classification criteria: an American College of Rheumatology/European League Against Rheumatism collaborative initiative. Arthritis Rheum. 62, 2569–2581 (2010).

    PubMed  Google Scholar 

  51. Taylor, W. et al. Classification criteria for psoriatic arthritis: development of new criteria from a large international study. Arthritis Rheum. 54, 2665–2673 (2006).

    PubMed  Google Scholar 

  52. Sterne, J. A., Gavaghan, D. & Egger, M. Publication and related bias in meta-analysis. J. Clin. Epidemiol. 53, 1119–1129 (2000).

    CAS  PubMed  Google Scholar 

  53. Gan, H. K., You, B., Pond, G. R. & Chen, E. X. Assumptions of expected benefits in randomized phase III trials evaluating systemic treatments for cancer. J. Natl Cancer Inst. 104, 590–598 (2012).

    PubMed  Google Scholar 

  54. Felson, D. T. et al. American College of Rheumatology/European League Against Rheumatism provisional definition of remission in rheumatoid arthritis for clinical trials. Ann. Rheum. Dis. 70, 404–413 (2011).

    PubMed  Google Scholar 

  55. Fransen, J. et al. Performance of response criteria for assessing peripheral arthritis in patients with psoriatic arthritis: analysis of data from randomised controlled trials of two tumour necrosis factor inhibitors. Ann. Rheum. Dis. 65, 1373–1378 (2006).

    CAS  PubMed  PubMed Central  Google Scholar 

  56. L’Abbé, K. A., Detsky, A. S. & O’Rourke, K. Meta-analysis in clinical research. Ann. Intern. Med. 107, 224–233 (1987).

    PubMed  Google Scholar 

  57. Sterne, J. A. C. et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. Brit. Med. J. 343, d4002 (2011).

    PubMed  Google Scholar 

  58. Kim, Y., Choi, Y. -K. & Emery, S. Logistic regression with multiple random effects: a simulation study of estimation methods and statistical packages. Am. Stat. 67, 171–182 (2013).

    Google Scholar 

  59. Higgins, J. P. T. et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. Brit. Med. J. 343, d5928 (2011).

    PubMed  Google Scholar 

Download references

Acknowledgements

The authors thank B. Bierbaumer for his assistance in building the underlying database for this project.

Author information

Authors and Affiliations

  1. Division of Rheumatology, Department of Internal Medicine III, Medical University of Vienna, Vienna, Austria

    Andreas Kerschbaumer, Josef S. Smolen, Tijen Stefanova & Daniel Aletaha

  2. Department for Emergency Medicine, Medical University of Vienna, Vienna, Austria

    Harald Herkner

  3. University Library, Medical University of Vienna, Vienna, Austria

    Eva Chwala

Authors
  1. Andreas Kerschbaumer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Josef S. Smolen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Harald Herkner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tijen Stefanova
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eva Chwala
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Daniel Aletaha
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

A.K. contributed to development of hypothesis, statistical methods, study screening and selection, data extraction, manuscript preparation and creation of tables and graphs. J.S.S. contributed to development of hypothesis, study selection and manuscript preparation. H.H. contributed to development of hypothesis, statistical methods and manuscript preparation. T.S. contributed to data extraction, manuscript preparation and creation of tables. E.C. contributed to literature search, preparation of the manuscript and creation of tables. D.A. contributed to development of hypothesis, statistical methods, study selection and manuscript preparation.

Corresponding author

Correspondence to Daniel Aletaha.

Ethics declarations

Competing interests

A.K. has received consulting or lecture fees from Bristol-Myers Squibb, Celgene, Eli-Lilly, Merck Sharp and Dohme and Pfizer and non-financial support from Gilead; J.S.S. received grants to his institution from Abbvie, AstraZeneca, Janssen, Lilly, Merck Sharpe & Dohme, Pfizer and Roche and provided expert advice for, or had symposia speaking engagements with, AbbVie, Amgen, AstraZeneca, Astro, Bristol-Myers Squibb, Celgene, Celltrion, Chugai, Gilead, Glaxo, ILTOO Pharma, Janssen, Lilly, Merck Sharp & Dohme, Novartis-Sandoz, Pfizer, Roche, Samsung, Sanofi and UCB; H.H. has no financial relationship to disclose and is Co-ordinating Editor of the Cochrane EC Group; T.S. has no financial relationship to disclose; E.C. has no financial relationship to disclose; D.A. has received grants from AbbVie, Novartis, Roche and SoBi, consulting and lecture fees from Abbvie, Amgen, Celgene, Gilead, Galapagos, Lilly, Merck Sharpe & Dohme, Novartis, Pfizer, Roche, Sandoz and Sanofi/Genzyme.

Additional information

Peer review information Jennifer Sargent was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Detailed exploratory analysis of efficacy bias between phase 2 and phase 3 clinical trials in rheumatoid arthritis.

Heatmaps visualizing the association of the evaluated variables and the study phase (interaction) with the different American College of Rheumatology (ACR) responses (ACR20/50/70) uncorrected (left) and corrected for multiple testing using Bonferroni correction (right). The color code expresses the size of the p-values for interaction term between the respective determinant and study phase, testing the H0: OR = 1; the underlying estimates (beta) and degrees of freedom for the respective analyses are shown in the table in blue and red font color, respectively.

Extended Data Fig. 2 Scatterplot showing number of swollen joints required for study inclusion over time.

Circle colors represent phase 2 (blue) or phase 3 (red) trials, respectively.

Extended Data Fig. 3 Scatterplot showing number of tender joints required for study inclusion over time.

Circle colors represent phase 2 (blue) or phase 3 (red) trials, respectively.

Extended Data Fig. 4 Boxplot showing joint counts used for study inclusion (28/66 joint count) over time.

Box colors represent phase 2 (blue) or phase 3 (red) trials, respectively.

Extended Data Fig. 5 Funnel plots showing plotting the standard error (y-axis) against the odds ratios for ACR responses (x-axis).

Each dot represents a study arm selected for comparison. Study arms of phase 2 studies are shown in blue, phase 3 studies are shown in red.

Supplementary Information

Supplementary Information

Supplementary Tables 1–7, Supplementary References and Study Protocol.

Reporting Summary

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kerschbaumer, A., Smolen, J.S., Herkner, H. et al. Efficacy outcomes in phase 2 and phase 3 randomized controlled trials in rheumatology. Nat Med 26, 974–980 (2020). https://doi.org/10.1038/s41591-020-0833-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-020-0833-4

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing