A methodological framework to enhance the clinical success of cancer immunotherapy

Journal name:
Nature Biotechnology
Year published:
Published online

To the Editor:

Cancer immunotherapy has a rich history spanning more than 100 years. Yet the field has struggled to integrate its knowledge into methodological advances that enable clinical success. The recent approvals of sipuleucel-T (Provenge) for hormone-refractory prostate cancer and ipilimumab (Yervoy) for unresectable and metastatic melanoma mark a notable turning point for the field. The drug approvals, both based on patient survival benefit1, 2, underscore the emergence of immunotherapy as a new treatment modality for cancer and reflect key characteristics of a new methodological framework for future progress. A notable challenge for the development of immunotherapies (defined here to span vaccination, adoptive T-cell transfer and strategies to modulate adaptive immune responses) has been the absence of such a tailored methodological framework distinct from that widely used for chemotherapy.

We have started to systematically address the unique characteristics of immunotherapeutic agents in clinical trials by building a methodological framework to provide the knowledge and tools needed for successful immunotherapy development. Here, we summarize the results of two community-based associations—the Cancer Immunotherapy Consortium (CIC; formerly Cancer Vaccine Consortium, a program of the nonprofit Cancer Research Institute; New York) in the United States, and the Association for Cancer Immunotherapy (CIMT; Mainz, Germany) in Europe over the past seven years. CIC and CIMT conducted various initiatives in partnership with other groups and with the participation of major stakeholders from academia, the biotech and pharmaceutical industries and the US Food and Drug Administration (FDA). The resulting framework promises to define a better path for the development of new therapies and lay the foundation for the clinical subspecialty of immuno-oncology by informing future practitioners in the field3 and enabling reproducible success in the development of cancer immunotherapies.

The new framework comprises several components: (i) a new development paradigm for cancer immunotherapies4, (ii) harmonized use of methods for measuring immune response as a foundation for immune biomarker development5, 6, (iii) improved study designs4 and clinical endpoints7, (iv) immune-related antitumor response criteria8, (v) a publication framework for immune monitoring results from clinical trials9 and (vi) scientific exchange and regulatory interactions to inform guidance document development by regulatory authorities10, 11 (Table 1).

Table 1: Common challenges, proposed solutions and intended outcomes in immuno-oncology clinical development

Despite distinct scientific differences between chemotherapies and immunotherapies, clinical development of immunotherapies has followed the established chemotherapy paradigm. Notably, chemotherapies target the cancer directly whereas immunotherapies target the immune system. This methodological discrepancy may have contributed to the failures of several immunotherapy candidates4, 7, 12. The proposed paradigm recognizes characteristics of immunotherapy development that may differ from those of chemotherapy, such as the following: first, the optimal biologic dose is often not the maximum tolerated dose; second, treatment effect is not proportionally linked to toxicity; third, conventional pharmacokinetics may not determine dose and schedule; fourth, anti-tumor response is not the sole predictor of survival; and finally, clinical effects can be delayed in time and can occur after tumor volume increase (often categorized as progression). The new paradigm divides the development process into two phases—proof-of-principle trials and efficacy trials, where efficacy trials are recommended to be randomized (phase 2 and 3 trials). Furthermore, it offers considerations for toxicity screening in early trials, concepts for measurement of biologic activity, use of immune response assays in clinical trials, dose and schedule investigation, decision points in development, trial design, improved clinical endpoints and combination therapy. Besides providing a systematic approach to the developmental science, much of the value of this paradigm lies in the consensus between all of the main participants involved in cancer immunotherapy development, namely representatives from the academic, industrial and regulatory sectors4, 7, 12.

The unique clinical effects associated with immunotherapies, as opposed to chemotherapies, need to be recognized in a revision of our established concept of clinical endpoints to improve endpoints for immunotherapy trials. Oncologists are very familiar with chemotherapy effects, which occur either soon after treatment starts or not at all. For immunotherapy, however, clinical effects may have a broader spectrum and include early response (similar to chemotherapy), delayed response (after apparent tumor burden increase or progression), or slow changes over time, usually recognized as stable disease8. These response kinetics likely reflect the interplay between the immune system and the tumor. Delayed or slowed clinical effects influence both anti-tumor response and survival as clinical trial endpoints7 and require adjusted methods to measure this biology.

For anti-tumor response endpoints (complete or partial response, disease control and progression-free survival), principles for the development of immunotherapy response criteria were derived from community workshops4 and translated into applicable criteria based on clinical data from the ipilimumab (a cytotoxic T lymphocyte–associated antigen 4 (CTLA-4) targeting fully human antibody) immunotherapy program conducted by Bristol-Myers Squibb (Princeton, NJ, USA) and Medarex (Princeton) encompassing 487 patients with advanced melanoma. Four patterns of response were identified: first, immediate response; second, durable stable disease with possible slow decline in tumor burden; third, response after tumor burden increase (possible lymphocyte infiltration); and fourth, response in the presence of new lesions. The resulting immune-related response criteria are generally based on the World Health Organization (WHO; Geneva) and RECIST (response evaluation criteria in solid tumors) criteria, describe tumor burden as a continuous variable over time, and account for new lesions in the overall tumor burden8. Current data suggest an association of such response patterns with favorable survival, indicating that immune-related response criteria identify patients who have derived previously unrecognized benefit8. These criteria present an additional tool for investigating immunotherapies and are currently being prospectively validated.

For the survival endpoint, differences between chemotherapy and immunotherapy in randomized trials can be seen in the form of a delayed separation of Kaplan-Meier curves12, which for immunotherapies may occur months after treatment start and may reduce the statistical power to differentiate the curves in their entirety7. Conventional statistical methods do not have a provision for a delayed separation of curves, but rather assume a constant hazard ratio over time (proportional hazards), where the separation of curves occurs shortly after treatment start. In immunotherapy trials, a delayed separation of curves months after treatment start is expected, and events before the separation do not contribute to the differentiation between curves. These conditions need to be compensated for to avoid loss of statistical power. Consequently, alternative statistical methods should be considered when computing the required number of events for final analysis under a delayed separation assumption7. Importantly, any early interim and futility analysis should be carefully considered, as a delayed separation will increase the chances of a negative result at a time when curves have not yet parted.

The relevance of these observations is illustrated by the development of anti-CTLA-4 antibodies in metastatic melanoma13, 14 through two independent development programs by Pfizer (tremelimumab) and Bristol-Myers Squibb (ipilimumab). The tremelimumab program conducted an early interim analysis for survival in its phase 3 study and could not observe a survival benefit, resulting in study termination for futility as recommended by the data monitoring committee. Two years later, extended follow-up on the study population revealed a separation of survival curves14. Conversely, the scientific approach for ipilimumab development, based on the new clinical paradigm, shifted away from response-based endpoints and led to the change of the primary endpoints for its two phase 3 studies in metastatic melanoma from response rate and progression-free survival to overall survival with no interim analyses13. A mature final survival analysis of the first phase 3 study of ipilimumab in pretreated metastatic melanoma patients demonstrated a delayed separation of curves at four months and the first survival benefit in the history of advanced melanoma clinical investigation (hazard ratio of 0.66 or 34% risk reduction for death; ref. 2). The second phase 3 trial in untreated advanced melanoma also met its survival endpoint with the same characteristics15.

Immune biomarker development depends on the effective management of data variability resulting from immune assays. Activation of the immune system is the first biologic event after treatment with immunotherapy. Consequently, measurement of the immune response (T-cell or antibody response) for biomarker development is of particular interest to describe effects of therapy before reaching clinical endpoints. Immunological biomarkers, if reliably and reproducibly measured through immune monitoring assays, may fulfill several applications, from determining whether an immune intervention achieved its biological effect to predicting clinical outcomes as surrogates for clinical benefit. Current T-cell immune response assays, such as the enzyme-linked immunosorbent spot (ELISPOT) assay, intracellular cytokine staining and human leukocyte antigen–peptide multimer staining, are scientifically sound but tend to be methodologically inconsistent if not performed by specially trained laboratories. Unless properly controlled, they yield highly variable data and have contributed to the field's inability to define biomarkers for the above clinical applications7. A possible solution has been outlined by a series of international proficiency panels (quality control experiments across multiple centers) conducted by CIC and CIMT with >120 participating laboratories from 14 countries, encompassing the academic, nonprofit, biotech and pharmaceutical sectors, the US Department of Defense and the German regulatory agency Paul-Ehrlich-Institute (Langen, Germany). The results demonstrate that assay harmonization can substantially reduce variability5, 6 and may help to build a framework for assay use in multicenter clinical trials similar to that of The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use-Good Clinical Practice (ICH-GCP) for clinical protocols. Harmonization supports the elimination of factors that cause major variability from assay conduct through the adoption of standard operating procedures by all laboratories conducting the respective assay. This process does not require standardization of assay protocols. Wide implementation of the assay harmonization concept will likely increase the quality of immunological monitoring and thereby enable use of assay results to guide the clinical development of new immunotherapies and provide a better understanding of exact therapeutic mechanisms of action. Furthermore, success in managing data variability for immune monitoring assays would contribute to methodological validation and clinical qualification in biomarker development. This would allow their integration into clinical development plans and possibly create regulatory utility. Tools, such as assay harmonization, should be used judiciously so they do not stifle the scientific creativity needed for new assay development.

Data reporting in publications presents another challenge for immune monitoring in clinical trials. The community has not yet created the mechanism for open and consistent reporting of results. To support reproducible biomarker development, such a mechanism is needed to present results in scientific publications in a way that allows full disclosure of relevant experimental details. On the basis of the concept championed by the Minimum Information About Biological and Biomedical Investigations initiative16, CIC and CIMT together with the Human Immune Monitoring Center at Stanford University (Stanford, CA, USA) started the Minimal Information About T-Cell Assays (MIATA) project9. MIATA aims to provide guidelines for the publication of results from T-cell assays performed in clinical trials, which are based on community consensus and find broad recognition among scientists conducting such assays. The project includes wide outreach through public consultation and is planned to be completed late in 2011.

On the basis of the above, the first regulatory guidance was developed by the FDA. The US agency participated in several of the community workshops described, and in 2007 hosted its own workshop where the above topics were reviewed. In 2009, FDA issued a guidance document on Clinical Considerations for Therapeutic Cancer Vaccines10, including many of these topics. The guidance underwent public consultation and is currently in the process of finalization. In mid-2010, the European Medicines Agency (EMA) released a concept paper to request public feedback for revision of its guidance on “evaluation of anticancer medicinal products in man” with a specific aim to address clinical endpoints for biologics and including a section on cancer vaccines11, which received feedback from CIC and CIMT. The continued interactions between community-based associations and regulatory authorities may foster the expansion of regulatory guidance to better serve new immunotherapy development.

In conclusion, an obvious weakness of the past has been the absence of a tailored methodological framework for immunotherapy development that is distinct from that widely used for chemotherapy. The framework described here offers new tools, development principles and structure and has the potential to increase the credibility of the field overall. It defines a better path for development of new therapies and creates the foundation for a clinical subspecialty of immuno-oncology.

It should be noted that past failures in the clinical translation of immunotherapeutic strategies can be attributed in part to, aside from methodological limitations, incomplete scientific understanding of tumor immunology, including limited knowledge of the mechanisms that determine the interaction of the immune system with the tumor17, 18. The incorporation of novel approaches to address tumor-induced immune suppression, pathways of immune modulation, such as CTLA-4 or PD-1 (programmed death 1), the tumor microenvironment or the optimization of clinical effects through tailored combination therapies19 will also play a crucial role in the future clinical successes of immunotherapies.

Although many open questions remain, the outlook for immuno-oncology has substantially improved over the past two years. The framework we describe will continue to expand with the emerging field. Using the CIC and CIMT examples, continued progress may be accelerated through wide collaboration among stakeholders.


  1. Kantoff, P.W. et al. N. Engl. J. Med. 363, 411422 (2010).
  2. Hodi, F.S. et al. N. Engl. J. Med. 363, 711723 (2010).
  3. Goldman, B. & DeFrancesco, L. Nat. Biotechnol. 27, 129139 (2009).
  4. Hoos, A. et al. J. Immunother. 30, 115 (2007).
  5. Janetzki, S. et al. Cancer Immunol. Immunother. 57, 303315 (2008).
  6. Britten, C.M. et al. Cancer Immunol. Immunother. 58, 17011713 (2009).
  7. Hoos, A. et al. J. Natl. Cancer Inst. 102, 13881397 (2010).
  8. Wolchok, J.D. et al. Clin. Cancer Res. 15, 74127420 (2009).
  9. Janetzki, S. et al. Immunity 31, 527528 (2009).
  10. Guidance for industry: Clinical considerations for therapeutic cancer vaccines. (US Department of Health and Human Services, Food and Drug Administration, Center for Biologics Evaluation and Research, September 2009). http://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/default.htm
  11. European Medicines Agency. Concept paper on the need to revise the guidelines on the evaluation of anticancer medicinal products in man. 22 July 2010. EMA/CHMP/EWP/433478/2010. http://www.ema.europa.eu/ema/index.jsp?curl=search.jsp&q=22+July+2010.+EMA%2Fchmp%2Fewp%2F433478%2F2010&murl=menus%2Fregulations%2Fregulations.jsp&mid=
  12. Finke, L.H. et al. Vaccine 25, B97B109 (2007).
  13. Hoos, A. et al. Semin. Oncol. 37, 533546 (2010).
  14. Marshall, M., Ribas, A. & Huang, B. Evaluation of baseline serum C-reactive protein and benefit from tremelimumab compared to chemotherapy in first-line melanoma. Abstract no. 2609, presented at the 2010 annual meeting of the American Society Clinical Oncology, Chicago, IL, June 4–8, 2010.
  15. Robert, C. et al. N. Engl. J. Med. 364, 25172526 (2011).
  16. Taylor, C.F. et al. Nat. Biotechnol. 26, 889896 (2008).
  17. Finn, O.J. N. Engl. J. Med. 358, 27042715 (2008).
  18. Schreiber, R., Old, L.J. & Smyth, M.J. Science 331, 15651570 (2011).
  19. Zitvogel, L., Kepp, O. & Kroemer, G. Nat. Rev. Clin. Oncol. 8, 151160 (2011).

Download references


We thank all participants of the workshops and community-wide initiatives conducted by CIC and CIMT for the contribution of knowledge to this evolving methodological framework.

Author information


  1. Bristol-Myers Squibb, Wallingford, Connecticut, USA.

    • Axel Hoos
  2. Cancer Immunotherapy Consortium (CIC; formerly Cancer Vaccine Consortium) of the Cancer Research Institute, New York, New York, USA.

    • Axel Hoos &
    • Jill O'Donnell-Tormey
  3. Department of Medicine, University Medical Center of the Johannes Gutenberg-University, Mainz, Germany.

    • Cedrik M Britten
  4. Ribological GmbH, Mainz, Germany.

    • Cedrik M Britten
  5. Association for Cancer Immunotherapy (CIMT), Mainz, Germany.

    • Cedrik M Britten &
    • Christoph Huber
  6. Center for Translational Oncology and Immunology (TRON), Mainz, Germany.

    • Christoph Huber
  7. Cancer Research Institute (CRI), New York, New York, USA.

    • Jill O'Donnell-Tormey

Competing financial interests

A.H. is an employee of Bristol-Myers Squibb (BMS), a pharmaceutical company and holds stock in BMS. C.M.B. has 50% employment at Ribological (Mainz, Germany), which is a company developing vaccines to treat cancer. C.H. is cofounder and supervisory board member of Ganymed Pharmaceuticals and BioNtech (Mainz, Germany), both biotech companies.

Corresponding author

Correspondence to:

Author details

Additional data