Common pitfalls in preclinical cancer target validation

Key Points

  • Scientific robustness refers to the ability of a finding to withstand experimental variation. Results that are reproducible, but only under an extremely narrow set of conditions, are unlikely to make predictions that will be true (robust) under real-world conditions, such as in the clinic.

  • Whether an elevated level of a particular protein is associated with a poor prognosis in a given cancer provides very little information as to whether that protein would be a good target in that cancer. Being associated with a poor prognosis is neither necessary nor sufficient to be a good cancer target.

  • The fact that A correlates with B and that it is biologically plausible that A causes B does not formally prove that A causes B. For example, observing that high expression of a gene correlates with poor survival in cancer patients and knowing that that gene regulates malignant cell behaviour would not formally prove that the high expression of that gene is responsible for the poor survival. Similarly, observing that a drug is having its expected pharmacodynamic effect on its intended target and knowing that its intended target is important for cancer cell survival would not formally prove that the cytotoxicity of the drug is on-target.

  • Most of the cellular assays used in cancer pharmacology are 'down' rather than 'up' assays, which is problematic because there are far more uninteresting ways to make a complex system, such as a cell, perform less well than there are to make it work better.

  • Cellular phenotypes caused by a chemical or genetic perturbant should be considered to be off-target until proved otherwise, especially when the phenotypes were detected in a down assay and therefore could reflect a nonspecific loss of cellular fitness. It is only by performing rescue experiments that one can formally address whether the effects of a perturbant are on-target.

  • The basis for the therapeutic indices of the currently available cancer drugs, including cytotoxic and targeted agents, is still poorly understood. Most successful drugs do not inhibit their targets completely and continuously at their therapeutically useful doses and accurately predicting, a priori, the therapeutic index for inhibition of a new cancer target is virtually impossible.

Abstract

An alarming number of papers from laboratories nominating new cancer drug targets contain findings that cannot be reproduced by others or are simply not robust enough to justify drug discovery efforts. This problem probably has many causes, including an underappreciation of the danger of being misled by off-target effects when using pharmacological or genetic perturbants in complex biological assays. This danger is particularly acute when, as is often the case in cancer pharmacology, the biological phenotype being measured is a 'down' readout (such as decreased proliferation, decreased viability or decreased tumour growth) that could simply reflect a nonspecific loss of cellular fitness. These problems are compounded by multiple hypothesis testing, such as when candidate targets emerge from high-throughput screens that interrogate multiple targets in parallel, and by a publication and promotion system that preferentially rewards positive findings. In this Perspective, I outline some of the common pitfalls in preclinical cancer target identification and some potential approaches to mitigate them.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Distinguishing correlation from causation.
Figure 2: Drawing erroneous causal inferences from Kaplan–Meier curves.
Figure 3: Up assays outperform down assays.
Figure 4: Commonly used down assays in cancer biology and cancer pharmacology.
Figure 5: On-target versus off-target effects.
Figure 6: Distinguishing on-target versus off-target effects with rescue experiments.
Figure 7: Designing perturbant-resistant targets for rescue experiments.
Figure 8: Generating drug-resistant protein targets.

References

  1. 1

    Prinz, F., Schlange, T. & Asadullah, K. Believe it or not: how much can we rely on published data on potential drug targets? Nat. Rev. Drug Discov. 10, 712 (2011).

    CAS  Article  Google Scholar 

  2. 2

    Begley, C. G. & Ellis, L. M. Drug development: raise standards for preclinical cancer research. Nature 483, 531–533 (2012).

    CAS  Article  Google Scholar 

  3. 3

    Frye, S. V. et al. Tackling reproducibility in academic preclinical drug discovery. Nat. Rev. Drug Discov. 14, 733–734 (2015).

    CAS  Article  Google Scholar 

  4. 4

    Nosek, B. A. & Errington, T. M. Making sense of replications. eLife 6, e23383 (2017).

    Article  Google Scholar 

  5. 5

    Errington, T. M. et al. An open investigation of the reproducibility of cancer biology research. eLife 3, e04333 (2014).

    Article  Google Scholar 

  6. 6

    Flaherty, K. T. et al. Inhibition of mutated, activated BRAF in metastatic melanoma. N. Engl. J. Med. 363, 809–819 (2010).

    CAS  Article  Google Scholar 

  7. 7

    Harris, C. C. p53 tumor suppressor gene: from the basis research laboratory to the clinic — an abridged historical perspective. Carcinogenesis 17, 1187–1198 (1996).

    CAS  Article  Google Scholar 

  8. 8

    Semenza, G. L. Defining the role of hypoxia-inducible factor 1 in cancer biology and therapeutics. Oncogene 29, 625–634 (2010).

    CAS  Article  Google Scholar 

  9. 9

    Mellor, H. R. & Harris, A. L. The role of the hypoxia-inducible BH3-only proteins BNIP3 and BNIP3L in cancer. Cancer Metastasis Rev. 26, 553–566 (2007).

    CAS  Article  Google Scholar 

  10. 10

    Reiling, J. H. & Hafen, E. The hypoxia-induced paralogs Scylla and Charybdis inhibit growth by down-regulating S6K activity upstream of TSC in Drosophila. Genes Dev. 18, 2879–2892 (2004).

    CAS  Article  Google Scholar 

  11. 11

    Brugarolas, J. et al. Regulation of mTOR function in response to hypoxia by REDD1 and the TSC1/TSC2 tumor suppressor complex. Genes Dev. 18, 2893–2904 (2004).

    CAS  Article  Google Scholar 

  12. 12

    Keith, B., Johnson, R. S. & Simon, M. C. HIF1alpha and HIF2alpha: sibling rivalry in hypoxic tumour growth and progression. Nat. Rev. Cancer 12, 9–22 (2012).

    CAS  Article  Google Scholar 

  13. 13

    Lin, A., Giuliano, C. J., Sayles, N. M. & Sheltzer, J. M. CRISPR/Cas9 mutagenesis invalidates a putative cancer dependency targeted in on-going clinical trials. eLife 6, e24179 (2017).

    Article  Google Scholar 

  14. 14

    Gorre, M. et al. Clinical resistance to STI-571 cancer therapy caused by BCR–ABL gene mutation or amplification. Science 293, 876–880 (2001).

    CAS  Article  Google Scholar 

  15. 15

    Azam, M., Latek, R. R. & Daley, G. Q. Mechanisms of autoinhibition and STI-571/imatinib resistance revealed by mutagenesis of BCR–ABL. Cell 112, 831–843 (2003).

    CAS  Article  Google Scholar 

  16. 16

    Burgess, M. R., Skaggs, B. J., Shah, N. P., Lee, F. Y. & Sawyers, C. L. Comparative analysis of two clinically active BCR–ABL kinase inhibitors reveals the role of conformation-specific binding in resistance. Proc. Natl Acad. Sci. USA 102, 3395–3400 (2005).

    CAS  Article  Google Scholar 

  17. 17

    Balbas, M. D. et al. Overcoming mutation-based resistance to antiandrogens with rational drug design. eLife 2, e00499 (2013).

    Article  Google Scholar 

  18. 18

    Wacker, S. A., Houghtaling, B. R., Elemento, O. & Kapoor, T. M. Using transcriptome sequencing to identify mechanisms of drug action and resistance. Nat. Chem. Biol. 8, 235–237 (2012).

    CAS  Article  Google Scholar 

  19. 19

    Kasap, C., Elemento, O. & Kapoor, T. M. DrugTargetSeqR: a genomics- and CRISPR–Cas9-based method to analyze drug targets. Nat. Chem. Biol. 10, 626–628 (2014).

    CAS  Article  Google Scholar 

  20. 20

    Han, T. et al. The antitumor toxin CD437 is a direct inhibitor of DNA polymerase α. Nat. Chem. Biol. 12, 511–515 (2016).

    CAS  Article  Google Scholar 

  21. 21

    Forment, J. V. et al. Genome-wide genetic screening with chemically mutagenized haploid embryonic stem cells. Nat. Chem. Biol. 13, 12–14 (2017).

    CAS  Article  Google Scholar 

  22. 22

    Emery, C. M. et al. MEK1 mutations confer resistance to MEK and B-RAF inhibition. Proc. Natl Acad. Sci. USA 106, 20411–20416 (2009).

    CAS  Article  Google Scholar 

  23. 23

    Nazarian, R. et al. Melanomas acquire resistance to B-RAF(V600E) inhibition by RTK or N-RAS upregulation. Nature 468, 973–977 (2010).

    CAS  Article  Google Scholar 

  24. 24

    Losman, J. A. et al. (R)-2-hydroxyglutarate is sufficient to promote leukemogenesis and its effects are reversible. Science 339, 1621–1625 (2013).

    CAS  Article  Google Scholar 

  25. 25

    Kaelin, W. G. Jr. Molecular biology. Use and abuse of RNAi to study mammalian gene function. Science 337, 421–422 (2012).

    Article  Google Scholar 

  26. 26

    Kittler, R. et al. RNA interference rescue by bacterial artificial chromosome transgenesis in mammalian tissue culture cells. Proc. Natl Acad. Sci. USA 102, 2396–2401 (2005).

    CAS  Article  Google Scholar 

  27. 27

    Brignole, C. et al. Immune cell-mediated antitumor activities of GD2-targeted liposomal c-myb antisense oligonucleotides containing CpG motifs. J. Natl Cancer Inst. 96, 1171–1180 (2004).

    CAS  Article  Google Scholar 

  28. 28

    Munoz, D. M. et al. CRISPR screens provide a comprehensive assessment of cancer vulnerabilities but generate false-positive hits for highly amplified genomic regions. Cancer Discov. 6, 900–913 (2016).

    CAS  Article  Google Scholar 

  29. 29

    Aguirre, A. J. et al. Genomic copy number dictates a gene-independent cell response to CRISPR/Cas9 targeting. Cancer Discov. 6, 914–929 (2016).

    CAS  Article  Google Scholar 

  30. 30

    Blouw, B. et al. The hypoxic response of tumors is dependent on their microenvironment. Cancer Cell 4, 133–146 (2003).

    CAS  Article  Google Scholar 

  31. 31

    Bridgeman, V. L. et al. Vessel co-option is common in human lung metastases and mediates resistance to anti-angiogenic therapy in preclinical lung metastasis models. J. Pathol. 241, 362–374 (2017).

    CAS  Article  Google Scholar 

  32. 32

    Donnem, T. et al. Vessel co-option in primary human tumors and metastases: an obstacle to effective anti-angiogenic treatment? Cancer Med. 2, 427–436 (2013).

    CAS  Article  Google Scholar 

  33. 33

    Tybulewicz, V. L., Crawford, C. E., Jackson, P. K., Bronson, R. T. & Mulligan, R. C. Neonatal lethality and lymphopenia in mice with a homozygous disruption of the c-abl proto-oncogene. Cell 65, 1153–1163 (1991).

    CAS  Article  Google Scholar 

  34. 34

    Hamilton, A. L. et al. Proteasome inhibition with bortezomib (PS-341): a phase I study with pharmacodynamic end points using a day 1 and day 4 schedule in a 14-day cycle. J. Clin. Oncol. 23, 6107–6116 (2005).

    CAS  Article  Google Scholar 

  35. 35

    Kaelin, W. J. Choosing anticancer drug targets in the postgenomic era. J. Clin. Invest. 104, 1503–1506 (1999).

    CAS  Article  Google Scholar 

  36. 36

    Kaelin, W. G. Jr. The concept of synthetic lethality in the context of anticancer therapy. Nat. Rev. Cancer 5, 689–698 (2005).

    CAS  Article  Google Scholar 

Download references

Acknowledgements

The author thanks J. Losman, D. Nijhawan, W. Sellers and members of the Kaelin Laboratory for careful reading of this manuscript and useful suggestions. He dedicates this article to his late wife, Dr Carolyn M. Kaelin, who survived a breast cancer in 2003 but died of a glioblastoma in 2015.

Author information

Affiliations

Authors

Corresponding author

Correspondence to William G. Kaelin Jr.

Ethics declarations

Competing interests

The author declares no competing financial interests.

Related links

PowerPoint slides

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kaelin, W. Common pitfalls in preclinical cancer target validation. Nat Rev Cancer 17, 441–450 (2017). https://doi.org/10.1038/nrc.2017.32

Download citation

Further reading

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing