Claims that molecular markers can accurately diagnose cancer have recently been disputed; some prominent results have not been reproduced and bias has been proposed to explain the original observations. As new '-omics' fields are explored to assess molecular markers for cancer, bias will increasingly be recognized as the most important 'threat to validity' that must be addressed in the design, conduct and interpretation of such research.
This is a preview of subscription content
Subscribe to Journal
Get full journal access for 1 year
only $4.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Ransohoff, D. F. Developing molecular biomarkers for cancer. Science 299, 1679–1680 (2003).
Stears, R. L., Martinsky, T. & Schena, M. Trends in microarray analysis. Nature Med. 9, 140–145 (2003).
Petricoin, E. F. et al. Use of proteomic patterns in serum to identify ovarian cancer. Lancet 359, 572–577 (2002).
Zhu, W. et al. Detection of cancer-specific markers amid massive mass spectral data. Proc. Natl Acad. Sci. USA 100, 14666–14671 (2003).
Li, J., Zhang, Z., Rosenzweig, J., Wang, Y. Y. & Chan, D. W. Proteomics and bioinformatics approaches for identification of serum biomarkers to detect breast cancer. Clin. Chem. 48, 1296–1304 (2002).
Adam, B. L. et al. Serum protein fingerprinting coupled with a pattern-matching algorithm distinguishes prostate cancer from benign prostate hyperplasia and healthy men. Cancer Res. 62, 3609–3614 (2002).
Petricoin, E. F. et al. Serum proteomic patterns for detection of prostate cancer. J. Natl Cancer Inst. 94, 1576–1578 (2002).
FDA Press Office. Protein patterns may identify ovarian cancer [online], <http://www.fda.gov/bbs/topics/NEWS/2002/NEW00797.html> (2002).
Marcus, A. Testing for ovarian cancer is on the way. Wall Street Journal (New York) D1, D2 (1 Oct 2002).
Pollack, A. New cancer test stirs hope and concern. New York Times (New York) D1, D6 (3 Feb 2004).
Food and Drug Administration. Letter to correlogic systems, Inc. [online], <http://www.fda.gov/cdrh/oivd/letters/021804-correlogic.html> (2004).
Food and Drug Administration. Letter to laboratory corporation of America [online], <http://www.fda.gov/cdrh/oivd/letters/030204-labcorp.html> (2004).
Food and Drug Administration. Letter to quest diagnostics [online], <http://www.fda.gov/cdrh/oivd/letters/030204-quest.html> (2004).
Society of Gynecologic Oncologists. Society of gynecologic oncologists statement regarding OvaCheck [online], <http://www.sgo.org/images/pdfs/policy/OvaCheck_statement.pdf> (2004).
Wagner, L. A test before its time? FDA stalls distribution process of proteomic test. J. Natl Cancer Inst. 96, 500–501 (2004).
Garber, K. Debate rages over proteomic patterns. J. Natl Cancer Inst. 96, 816–818 (2004).
Check, E. Running before we can walk? Nature 429, 496–497 (2004).
Diamandis, E. Analysis of serum proteomic patterns for early cancer diagnosis: drawing attention to potential problems. J. Natl Cancer Inst. 96, 353–356 (2004).
Baggerly, K. A., Morris, J. S. & Coombes, K. R. Reproducibility of SELDI-TOF protein patterns in serum: comparing datasets from different experiments. Bioinformatics 20, 777–785 (2004).
Sorace, J. M. & Zhan, M. A data review and re-assessment of ovarian cancer serum proteomic profiling. BMC Bioinformatics 4, 24 (2003).
Baggerly, K. A., Coombes, K. R. & Morris, J. S. Are the NCI/FDA ovarian proteomic data biased? A reply to 'producers and consumers'. Cancer Informatics (in the press).
Baggerly, K. A., Edmonson, S. R., Morris, J. S. & Coombes, K. R. High-resolution serum proteomic patterns for ovarian cancer detection. Endo. Relat. Cancer 11, 583–584 (2004).
van de Vijver, M. J. et al. A gene-expression signature as a predictor of survival in breast cancer. N. Engl. J. Med. 347, 1999–2009 (2002).
Huang, E. et al. Gene expression predictors of breast cancer outcomes. Lancet 361, 1590–1596 (2003).
Ramaswamy, S. & Perou, C. M. DNA microarrays in breast cancer: the promise of personalised medicine. Lancet 361, 1576–1577 (2003).
Ransohoff, D. F. Gene-expression signatures in breast cancer. N. Engl. J. Med. 348, 1715–1717; author reply 1715–1717 (2003).
Marshall, E. Getting the noise out of gene arrays. Science 306, 630–631 (2004).
Ambroise, C. & McLachlan, G. J. Selection bias in gene extraction on the basis of microarray gene-expression data. Proc. Natl Acad. Sci. USA 99, 6562–6566 (2002).
Baggerly, K. A. et al. A comprehensive approach to the analysis of matrix-assisted laser desorption/ionization-time of flight proteomics spectra from serum samples. Proteomics 3, 1667–1672 (2003).
Mehta, T., Tanik, M. & Allison, D. B. Towards sound epistemological foundations of statistical methods for high-dimensional biology. Nature Genet. 36, 943–947 (2004).
Ransohoff, D. F. Rules of evidence for cancer molecular-marker discovery and validation. Nature Rev. Cancer 4, 309–314 (2004).
Ransohoff, D. F. Discovery-based research and fishing. Gastroenterology 125, 290 (2003).
Hulley, S. B. et al. Designing Clinical Research: An Epidemiologic Approach (Lippincott Williams & Wilkins, Philadelphia, 2001).
Rothman, K. J. Modern Epidemiology (Little, Brown and Company, Boston/Toronto, 1986).
Hennekens, C. H. & Buring, J. E. Epidemiology in Medicine (Little, Brown and Company, Boston, 1987).
Taubes, G. Epidemiology faces its limits. Science 269, 164–169 (1995).
Ransohoff, D. F. Research opportunity at the interface of molecular biology and clinical epidemiology. Gastroenterology 122, 1199 (2002).
Ransohoff, D. F. Evaluating discovery-based research: when biologic reasoning cannot work. Gastroenterology 127, 1028 (2004).
Coombes, R. C. et al. A randomized trial of exemestane after two to three years of tamoxifen therapy in postmenopausal women with primary breast cancer. N. Engl. J. Med. 350, 1081–1092 (2004).
Altman, D. G. et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann. Intern. Med. 134, 663–694 (2001).
The case-control study: consensus and controversy. J. Chronic. Dis. 32, 1–144 (1979).
Sackett, D. L. Bias in analytic research. J. Chronic. Dis. 32, 51–63 (1979).
Roos, N. P. et al. Mortality and reoperation after open and transurethral resection of the prostate for benign prostatic hyperplasia. N. Engl. J. Med. 320, 1120–1124 (1989).
Greenfield, S. The state of outcome research: are we on target? N. Engl. J. Med. 320, 1142–1143 (1989).
Concato, J., Horwitz, R. I., Feinstein, A. R., Elmore, J. G. & Schiff, S. F. Problems of comorbidity in mortality after prostatectomy. JAMA 267, 1077–1082 (1992).
Iezzoni, L. I. et al. Comorbidities, complications, and coding bias. Does the number of diagnosis codes matter in predicting in-hospital mortality? JAMA 267, 2197–2203 (1992).
Iezzoni, L. I. The risks of risk adjustment. JAMA 278, 1600–1607 (1997).
Hulley, S. et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA 280, 605–613 (1998).
Manson, J. E. et al. Estrogen plus progestin and the risk of coronary heart disease. N. Engl. J. Med. 349, 523–534 (2003).
Col, N. F. & Pauker, S. G. The discrepancy between observational studies and randomized trials of menopausal hormone therapy: did expectations shape experience? Ann. Intern. Med. 139, 923–929 (2003).
Rennie, D. How to report randomized controlled trials. The CONSORT statement. JAMA 276, 649 (1996).
Bossuyt, P. M. et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. Ann. Intern. Med. 138, 40–44 (2003).
Bossuyt, P. M. et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann. Intern. Med. 138, W1–12 (2003).
Brazma, A. et al. Minimum information about a microarray experiment (MIAME)—toward standards for microarray data. Nature Genet. 29, 365–371 (2001).
Ransohoff, D. F. Lessons from controversy: ovarian cancer screening and serum proteomics. J. Natl Cancer Inst. (in the press).
Imperiale, T. F. et al. Fecal DNA versus fecal occult blood for colorectal-cancer screening in an average-risk population. N. Engl. J. Med. 351, 2704–2714 (2004).
Prorok, P. C. et al. Design of the Prostate, Lung, Colorectal and Ovarian (PLCO) cancer screening trial. Control Clin. Trials 21, 273S–309S (2000).
Simon, R., Radmacher, M. D., Dobbin, K. & McShane, L. M. Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J. Natl Cancer Inst. 95, 14–18 (2003).
Ntzani, E. E. & Ioannidis, J. P. Predictive ability of DNA microarrays for cancer outcomes and correlates: an empirical assessment. Lancet 362, 1439–1444 (2003).
Friedman, L. M., Furberg, C. D. & De Mets, D. L. Fundamentals of Clinical Trials, 3rd edn (Springer, New York, 1998).
Sullivan Pepe, M. et al. Phases of biomarker development for early detection of cancer. J. Natl Cancer Inst. 93, 1054–1061 (2001).
Thanks to colleagues at the University of North Carolina at Chapel Hill, the National Cancer Institute and elsewhere for reviewing and commenting on earlier versions of the manuscript. Many ideas were developed through participation in activities of the Early Detection Research Network.
The author declares no competing financial interests.
About this article
Cite this article
Ransohoff, D. Bias as a threat to the validity of cancer molecular-marker research. Nat Rev Cancer 5, 142–149 (2005). https://doi.org/10.1038/nrc1550
Nature Reviews Cancer (2021)
BMC Cancer (2019)
A pilot study of non-targeted screening for stimulant misuse using high-resolution mass spectrometry
Forensic Toxicology (2019)
MALDI-TOF peptidomic analysis of serum and post-prostatic massage urine specimens to identify prostate cancer biomarkers
Clinical Proteomics (2018)
Nature Medicine (2018)