Published online 4 December 2007 | Nature | doi:10.1038/news.2007.339


The ghost of research past

Carrots are still taken to prevent cancer, despite clinical results suggesting they don't work.

The public and scientists alike are still keen on early results hinting that β-carotene may protect against cancer.Getty

In 1981, Nature published a review about the possible link between high levels of dietary β-carotene and reduced rates of cancer. The article came with an unusual message from the editor running along the bottom of the first page. “Caution,” the note read. “Unwary readers (if such there are) should not take the accompanying article as a sign that the consumption of large quantities of carrots (or other major dietary sources of β-carotene) is necessarily protective against cancer.”1

The review highlighted promising epidemiological studies, but referred repeatedly to the need for clinical trials. In 1994, the results of those trials were in: β-carotene supplements, given together with vitamin E, did not protect against lung cancer as epidemiological studies had suggested they might. Instead, the supplements may actually have raised risk of developing the disease for male smokers2.

That kind of scientific about-turn, so common in nutritional research, is often blamed for spreading dietary confusion. A new study has found that just as the public continues spend millions each year on β-carotene supplements, some scientists also eschew the clinical-trial data — which in this case are generally held to be more trustworthy — and instead continue to perpetuate the original observational findings.

Selective memory

These findings were paralleled in two other examples in which observational studies and randomized clinical-trial data clashed: the notion that vitamin E might protect against cardiovascular disease, and that oestrogen might increase the risk of developing Alzheimer’s disease. Some journal articles continued to cite the original studies long after the clinical trials were published.

“Across all topics, no matter whether you look early or late after the contradiction, there is a section of the scientific literature that continues to pretty strongly support the original claims,” says John Ioannidis, an epidemiologist at the University of Ioannina School of Medicine in Greece, and an author on the study published in the Journal of the American Medical Association3.

Epidemiological studies that look for an observed link between nutrition and disease are obviously useful in generating testable hypotheses. But they are also potentially misleading. The studies benefit from easy data collection, but are subject to confounding factors and are generally unable to establish a causative relationship.

“I would not say one should not listen to observational studies. They’re a starting point; they give hints about what to pursue further,” says Ioannidis. “But I would not take it for granted that they are the truth.”

Hard to let go

Ioannidis and his colleagues traced the literature in these three chosen case studies where clinical trials were known to have upset early epidemiology. In all three cases, there was a decline in the number of citations of the early observational work once the contradicting clinical trial data was published. But many authors continued to favour the early observational data. This was most pronounced in β-carotene studies in which, over a decade after the clinical trial data were published, some authors referenced the observational data without even mentioning the clinical trial results.

In the other two cases, authors who favourably cited the observational data also referenced the clinical-trial data, but then provided reasons for why they discounted the clinical-trial results. In evaluating the link between vitamin E and cardiovascular disease, for example, some challenged the form of vitamin E administered in the trials; others speculated that the trials needed to have run for longer before the benefits would become apparent. Similar arguments were made in some articles about the effects of hormone-replacement therapy on the risk of Alzheimer’s disease.


While some of these criticisms provide valid starting points for future research, others simply don’t hold up, says Edgar Miller, an epidemiologist at Johns Hopkins University in Baltimore, Maryland, and lead author of an analysis of clinical-trial data that found a link between vitamin E supplements and slightly higher death rates. “Some of these counter-arguments muddy the waters,” he says.

Wishful thinking

Why then do some scientists prefer the earlier data? Ioannidis speculates that this is sometimes due to ‘wish bias’. “People have been pursuing a line of research and they just don’t want to believe that they have followed a path that is incorrect,” he says. “That’s not a nice feeling. It's like defending your baby.”

But Jan Vandenbroucke, an epidemiologist at the Leiden University Medical Centre in the Netherlands cautions that it may be too soon to draw conclusions about bias. Ioannidis and his colleagues should enlist the help of experts in each field to determine how much merit there is in arguments that favor observational research, he says.

Some delay is to be expected before clinical-trial data percolates through the culture, he adds: "It takes some time to turn around a scientific community." 

  • References

    1. Peto, R., Doll, R., Buckley, J. D., & Sporn, M. B. Nature 290, 201-208 (1981). | Article | PubMed | ISI | ChemPort |
    2. The Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group. N. Engl. J. Med. 330, 1029-1035 (1994). | Article | PubMed | ISI |
    3. Tatsioni, A., Bonitsis, N. G. & Ioannidis, J. P. A. J. Am. Med. Assoc. 298, 2517-2526 (2007). | Article |
Commenting is now closed.