Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Opinion
  • Published:

Common pitfalls in preclinical cancer target validation

Nature Reviews Cancer volume 17, pages 441–450 (2017)Cite this article

Subjects

Key Points

  • Scientific robustness refers to the ability of a finding to withstand experimental variation. Results that are reproducible, but only under an extremely narrow set of conditions, are unlikely to make predictions that will be true (robust) under real-world conditions, such as in the clinic.

  • Whether an elevated level of a particular protein is associated with a poor prognosis in a given cancer provides very little information as to whether that protein would be a good target in that cancer. Being associated with a poor prognosis is neither necessary nor sufficient to be a good cancer target.

  • The fact that A correlates with B and that it is biologically plausible that A causes B does not formally prove that A causes B. For example, observing that high expression of a gene correlates with poor survival in cancer patients and knowing that that gene regulates malignant cell behaviour would not formally prove that the high expression of that gene is responsible for the poor survival. Similarly, observing that a drug is having its expected pharmacodynamic effect on its intended target and knowing that its intended target is important for cancer cell survival would not formally prove that the cytotoxicity of the drug is on-target.

  • Most of the cellular assays used in cancer pharmacology are 'down' rather than 'up' assays, which is problematic because there are far more uninteresting ways to make a complex system, such as a cell, perform less well than there are to make it work better.

  • Cellular phenotypes caused by a chemical or genetic perturbant should be considered to be off-target until proved otherwise, especially when the phenotypes were detected in a down assay and therefore could reflect a nonspecific loss of cellular fitness. It is only by performing rescue experiments that one can formally address whether the effects of a perturbant are on-target.

  • The basis for the therapeutic indices of the currently available cancer drugs, including cytotoxic and targeted agents, is still poorly understood. Most successful drugs do not inhibit their targets completely and continuously at their therapeutically useful doses and accurately predicting, a priori, the therapeutic index for inhibition of a new cancer target is virtually impossible.

Abstract

An alarming number of papers from laboratories nominating new cancer drug targets contain findings that cannot be reproduced by others or are simply not robust enough to justify drug discovery efforts. This problem probably has many causes, including an underappreciation of the danger of being misled by off-target effects when using pharmacological or genetic perturbants in complex biological assays. This danger is particularly acute when, as is often the case in cancer pharmacology, the biological phenotype being measured is a 'down' readout (such as decreased proliferation, decreased viability or decreased tumour growth) that could simply reflect a nonspecific loss of cellular fitness. These problems are compounded by multiple hypothesis testing, such as when candidate targets emerge from high-throughput screens that interrogate multiple targets in parallel, and by a publication and promotion system that preferentially rewards positive findings. In this Perspective, I outline some of the common pitfalls in preclinical cancer target identification and some potential approaches to mitigate them.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Distinguishing correlation from causation.
Figure 2: Drawing erroneous causal inferences from Kaplan–Meier curves.
Figure 3: Up assays outperform down assays.
Figure 4: Commonly used down assays in cancer biology and cancer pharmacology.
Figure 5: On-target versus off-target effects.
Figure 6: Distinguishing on-target versus off-target effects with rescue experiments.
Figure 7: Designing perturbant-resistant targets for rescue experiments.
Figure 8: Generating drug-resistant protein targets.

Similar content being viewed by others

References

  1. Prinz, F., Schlange, T. & Asadullah, K. Believe it or not: how much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. 10, 712 (2011).

    Article  CAS  Google Scholar 

  2. Begley, C. G. & Ellis, L. M. Drug development: raise standards for preclinical cancer research. Nature 483, 531–533 (2012).

    Article  CAS  Google Scholar 

  3. Frye, S. V. et al. Tackling reproducibility in academic preclinical drug discovery. Nat. Rev. Drug Discov. 14, 733–734 (2015).

    Article  CAS  Google Scholar 

  4. Nosek, B. A. & Errington, T. M. Making sense of replications. eLife 6, e23383 (2017).

    Article  Google Scholar 

  5. Errington, T. M. et al. An open investigation of the reproducibility of cancer biology research. eLife 3, e04333 (2014).

    Article  Google Scholar 

  6. Flaherty, K. T. et al. Inhibition of mutated, activated BRAF in metastatic melanoma. N. Engl. J. Med. 363, 809–819 (2010).

    Article  CAS  Google Scholar 

  7. Harris, C. C. p53 tumor suppressor gene: from the basis research laboratory to the clinic — an abridged historical perspective. Carcinogenesis 17, 1187–1198 (1996).

    Article  CAS  Google Scholar 

  8. Semenza, G. L. Defining the role of hypoxia-inducible factor 1 in cancer biology and therapeutics. Oncogene 29, 625–634 (2010).

    Article  CAS  Google Scholar 

  9. Mellor, H. R. & Harris, A. L. The role of the hypoxia-inducible BH3-only proteins BNIP3 and BNIP3L in cancer. Cancer Metastasis Rev. 26, 553–566 (2007).

    Article  CAS  Google Scholar 

  10. Reiling, J. H. & Hafen, E. The hypoxia-induced paralogs Scylla and Charybdis inhibit growth by down-regulating S6K activity upstream of TSC in Drosophila. Genes Dev. 18, 2879–2892 (2004).

    Article  CAS  Google Scholar 

  11. Brugarolas, J. et al. Regulation of mTOR function in response to hypoxia by REDD1 and the TSC1/TSC2 tumor suppressor complex. Genes Dev. 18, 2893–2904 (2004).

    Article  CAS  Google Scholar 

  12. Keith, B., Johnson, R. S. & Simon, M. C. HIF1alpha and HIF2alpha: sibling rivalry in hypoxic tumour growth and progression. Nat. Rev. Cancer 12, 9–22 (2012).

    Article  CAS  Google Scholar 

  13. Lin, A., Giuliano, C. J., Sayles, N. M. & Sheltzer, J. M. CRISPR/Cas9 mutagenesis invalidates a putative cancer dependency targeted in on-going clinical trials. eLife 6, e24179 (2017).

    Article  Google Scholar 

  14. Gorre, M. et al. Clinical resistance to STI-571 cancer therapy caused by BCR–ABL gene mutation or amplification. Science 293, 876–880 (2001).

    Article  CAS  Google Scholar 

  15. Azam, M., Latek, R. R. & Daley, G. Q. Mechanisms of autoinhibition and STI-571/imatinib resistance revealed by mutagenesis of BCR–ABL. Cell 112, 831–843 (2003).

    Article  CAS  Google Scholar 

  16. Burgess, M. R., Skaggs, B. J., Shah, N. P., Lee, F. Y. & Sawyers, C. L. Comparative analysis of two clinically active BCR–ABL kinase inhibitors reveals the role of conformation-specific binding in resistance. Proc. Natl Acad. Sci. USA 102, 3395–3400 (2005).

    Article  CAS  Google Scholar 

  17. Balbas, M. D. et al. Overcoming mutation-based resistance to antiandrogens with rational drug design. eLife 2, e00499 (2013).

    Article  Google Scholar 

  18. Wacker, S. A., Houghtaling, B. R., Elemento, O. & Kapoor, T. M. Using transcriptome sequencing to identify mechanisms of drug action and resistance. Nat. Chem. Biol. 8, 235–237 (2012).

    Article  CAS  Google Scholar 

  19. Kasap, C., Elemento, O. & Kapoor, T. M. DrugTargetSeqR: a genomics- and CRISPR–Cas9-based method to analyze drug targets. Nat. Chem. Biol. 10, 626–628 (2014).

    Article  CAS  Google Scholar 

  20. Han, T. et al. The antitumor toxin CD437 is a direct inhibitor of DNA polymerase α. Nat. Chem. Biol. 12, 511–515 (2016).

    Article  CAS  Google Scholar 

  21. Forment, J. V. et al. Genome-wide genetic screening with chemically mutagenized haploid embryonic stem cells. Nat. Chem. Biol. 13, 12–14 (2017).

    Article  CAS  Google Scholar 

  22. Emery, C. M. et al. MEK1 mutations confer resistance to MEK and B-RAF inhibition. Proc. Natl Acad. Sci. USA 106, 20411–20416 (2009).

    Article  CAS  Google Scholar 

  23. Nazarian, R. et al. Melanomas acquire resistance to B-RAF(V600E) inhibition by RTK or N-RAS upregulation. Nature 468, 973–977 (2010).

    Article  CAS  Google Scholar 

  24. Losman, J. A. et al. (R)-2-hydroxyglutarate is sufficient to promote leukemogenesis and its effects are reversible. Science 339, 1621–1625 (2013).

    Article  CAS  Google Scholar 

  25. Kaelin, W. G. Jr. Molecular biology. Use and abuse of RNAi to study mammalian gene function. Science 337, 421–422 (2012).

    Article  Google Scholar 

  26. Kittler, R. et al. RNA interference rescue by bacterial artificial chromosome transgenesis in mammalian tissue culture cells. Proc. Natl Acad. Sci. USA 102, 2396–2401 (2005).

    Article  CAS  Google Scholar 

  27. Brignole, C. et al. Immune cell-mediated antitumor activities of GD2-targeted liposomal c-myb antisense oligonucleotides containing CpG motifs. J. Natl Cancer Inst. 96, 1171–1180 (2004).

    Article  CAS  Google Scholar 

  28. Munoz, D. M. et al. CRISPR screens provide a comprehensive assessment of cancer vulnerabilities but generate false-positive hits for highly amplified genomic regions. Cancer Discov. 6, 900–913 (2016).

    Article  CAS  Google Scholar 

  29. Aguirre, A. J. et al. Genomic copy number dictates a gene-independent cell response to CRISPR/Cas9 targeting. Cancer Discov. 6, 914–929 (2016).

    Article  CAS  Google Scholar 

  30. Blouw, B. et al. The hypoxic response of tumors is dependent on their microenvironment. Cancer Cell 4, 133–146 (2003).

    Article  CAS  Google Scholar 

  31. Bridgeman, V. L. et al. Vessel co-option is common in human lung metastases and mediates resistance to anti-angiogenic therapy in preclinical lung metastasis models. J. Pathol. 241, 362–374 (2017).

    Article  CAS  Google Scholar 

  32. Donnem, T. et al. Vessel co-option in primary human tumors and metastases: an obstacle to effective anti-angiogenic treatment? Cancer Med. 2, 427–436 (2013).

    Article  CAS  Google Scholar 

  33. Tybulewicz, V. L., Crawford, C. E., Jackson, P. K., Bronson, R. T. & Mulligan, R. C. Neonatal lethality and lymphopenia in mice with a homozygous disruption of the c-abl proto-oncogene. Cell 65, 1153–1163 (1991).

    Article  CAS  Google Scholar 

  34. Hamilton, A. L. et al. Proteasome inhibition with bortezomib (PS-341): a phase I study with pharmacodynamic end points using a day 1 and day 4 schedule in a 14-day cycle. J. Clin. Oncol. 23, 6107–6116 (2005).

    Article  CAS  Google Scholar 

  35. Kaelin, W. J. Choosing anticancer drug targets in the postgenomic era. J. Clin. Invest. 104, 1503–1506 (1999).

    Article  CAS  Google Scholar 

  36. Kaelin, W. G. Jr. The concept of synthetic lethality in the context of anticancer therapy. Nat. Rev. Cancer 5, 689–698 (2005).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The author thanks J. Losman, D. Nijhawan, W. Sellers and members of the Kaelin Laboratory for careful reading of this manuscript and useful suggestions. He dedicates this article to his late wife, Dr Carolyn M. Kaelin, who survived a breast cancer in 2003 but died of a glioblastoma in 2015.

Author information

Authors and Affiliations

  1. Howard Hughes Medical Institute, Dana-Farber Cancer Institute and Brigham and Women's Hospital, Harvard Medical School, Boston, 02215, Massachusetts, USA

    William G. Kaelin Jr

Authors
  1. William G. Kaelin Jr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William G. Kaelin Jr.

Ethics declarations

Competing interests

The author declares no competing financial interests.

Related links

PowerPoint slides

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaelin, W. Common pitfalls in preclinical cancer target validation. Nat Rev Cancer 17, 441–450 (2017). https://doi.org/10.1038/nrc.2017.32

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrc.2017.32

This article is cited by

Search

Advanced search

Quick links

Nature Briefing: Cancer

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

Get what matters in cancer research, free to your inbox weekly. Sign up for Nature Briefing: Cancer