Perspective: Does peer review mean the same to the public as it does to scientists?

Nature (2006) | doi:10.1038/nature05009

Even reviewed literature can be cherry-picked to support any argument.

The research community understands that scientific information that has not been peer reviewed should not be taken seriously. As scientists, we discriminate between what is put out on blogs or in press releases and what is published in the formal scientific literature. We also know the difference between a peer-reviewed primary paper or review, and an unreviewed letter to the editor or opinion piece. In other words, we understand the peer-review system, and use it as a filter to sort the wheat from the chaff.

As other contributors to this forum will no doubt opine, the peer-review system has its flaws. My own view is that it's the least-bad system that can be devised, and that, although it might need tinkering with, its fundamentals should remain intact. One way to consider its operating principle is: 'Judgement of the scientists, by the scientists, for the people.' But do the people understand the limitations of the process? I suspect not. Even science writers and journalists who should act as important links between scientists and the public sometimes seem not to appreciate what peer review means.

It's been peer reviewed, so it must be right, right? Wrong! Not everything in the peer-reviewed literature is correct. Indeed, some of it is downright bad science. Professional scientists usually know how to rate papers within their own fields of expertise (all too often very narrow ones nowadays). We realize that some journals are more stringent than others in what they will accept, and that peer-review standards can unfortunately be too flexible. A lust for profit has arguably led to the appearance of too many journals, and so it can be all too easy to find somewhere that will publish poor-quality work.

The public doesn't understand this, how could it? But the term 'peer review' is often equated with 'gold standard'. Hence, the politically motivated, lazy or unscrupulous can use the peer-reviewed literature selectively, to make arguments that are seriously flawed, or even damaging to public policy. Chris Mooney, in The Republican War on Science (Basic Books, 2005), provides several examples of how this operates in the political world.

Professional scientists can see through this tactic. We know that scientific truth evolves on the basis of a mounting consensus, not through an isolated paper that adopts a maverick position, even if it has been 'peer reviewed'. In contrast, politicians all too often cherry-pick the 'facts' they find most convenient to their party's agenda. And politicians are not alone.

In my own field of AIDS research, a small clique of scientists and scientifically ignorant laymen promotes the bizarre view that HIV does not cause AIDS, or, in a particularly dubious variant of the genre, that HIV does not actually exist. These AIDS denialists are experts at selectively using the peer-reviewed literature to superficially bolster their positions. I think they lack the training – or if trained, the integrity – to appreciate two things that are understood by professional scientists. First, that peer-reviewed literature develops over time, so that what was legitimately uncertain 20 years ago is fully understood today. This means that citing decade-old papers and ignoring more recent ones is an unscrupulous tactic. Second, that ignoring every paper bar the one that most conveniently suits a preconceived position could be considered scientific misconduct.

Similar practices can be found in other science-related areas. For example, advertisements claiming that vitamin pills can cure cancer and infectious diseases selectively cite the peer-reviewed literature.

One problem the public faces when trying to understand science is that the peer-reviewed literature is still not generally accessible, despite the efforts of the open-access movement (see 'Access to the literature'). I am often asked to email scientific papers to AIDS activists who cannot easily access publication databases. I shudder to think how frustrating it must be for the true layperson entering an area of research for the first time, without the professional connections to acquire information, let alone interpret it. The publishers need to wise up; the public has a right to see the papers its tax dollars pay for. Otherwise, the public may resort to the Internet to inform them – to the blogs and websites that all too often promote strange, pseudoscientific ideas.

And so, despite professional scientists laughing at the notion that HIV does not cause AIDS, some vulnerable, newly infected people, who would like to believe that they have not just contracted a deadly virus, end up surfing the web for answers. Sadly, so do some science writers and their editors (see www.aidstruth.org for recent examples). This kind of fiasco might be avoided if the public had better access to the peer-reviewed literature, and if bona fide scientists were willing to give the public more assistance in interpreting it properly.

RELATED LINKS:

AIDSTruth.org

John Moore is an AIDS researcher now working at the Weill Medical College of Cornell University in New York (www.med.cornell.edu).

Visit our peer-to-peer blog to read and post comments about this article.

Extra navigation

.
ADVERTISEMENT