Preprints could promote confusion and distortion

The scientific community must take measures to keep preprints from distorting the public’s understanding of science, says Tom Sheldon.
Tom Sheldon is a senior press manager at the Science Media Centre in London.

Search for this author in:

Thousands of papers are submitted every month to the platforms arXiv and bioRxiv, which make manuscripts available before they have been peer reviewed and accepted by a journal. Scientists applaud preprints because they enable researchers to claim priority and make their findings available more quickly, unshackled from sluggish and tyrannical journals.

This might make sense within the scientific community, but as someone who has worked for years with researchers and journalists to ensure responsible coverage of science in the media, I fear that this method of publication holds substantial risks for the broader community — risks that are not being given proper consideration by the champions of preprint. Weak work that hasn’t been reviewed could get overblown in the media. Conversely, better work could be ignored.

Those who run preprint servers, researchers who submit to them and anyone considering publicizing a preprint have a responsibility to take note of these risks. The responses we received to an open letter we posted on 17 July have not entirely reassured me that this is the case. Here, I call on scientists to reflect on — and find ways to limit — the potential of preprints to elicit public confusion and misinformation.

Many people still learn about science the same way they learn about Vladimir Putin, Syria or the World Cup: through news sites, television and radio. The bulk of research reported through these channels is peer reviewed. A few days before a paper is published, the science journal will issue a restricted press release to qualified journalists under an agreement (called an embargo) that no one will report on the paper until a designated time. The embargo system has its flaws, but it does give reporters time to assess the research and gather expert reaction.

Contrast this with preprints. As soon as research is in the public domain, there is nothing to stop a journalist writing about it, and rushing to be the first to do so. Imagine early findings that seem to show that climate change is natural or that a common vaccine is unsafe. Preprints on subjects such as those could, if they become a story that goes viral, end up misleading millions, whether or not that was the intention of the authors.

Many reporters (including those at Nature) have written stories based on preprints. I’ll admit that we do not yet have examples of harm from such stories, but this is probably because — at the moment — only a tiny fraction of preprints cover health-related or controversial fields.

What we have no shortage of are examples in which public understanding has been distorted by media coverage of ambiguous or just downright bad science. Take the (now retracted) paper which claimed that genetically modified maize (corn) gave rats cancer (G.-E. Séralini et al. Food Chem. Toxicol. 50, 4221–4231; 2012) — its carefully orchestrated publicity campaign ensured that journalists could not seek outside scrutiny before running their news stories. The science was poor, but the claims were intriguing. Many journalists gave up the chance to vet information so as not to be late to the story.

Some scientists have pointed out that preprints are similar to conference presentations, which are also not peer reviewed. In my experience, conference proceedings often make the most misleading stories of all. Take a poster from the Pediatric Academic Societies meeting in April–May 2016, which looked at the incidence of autism and pesticide application in New York state. This was extremely early work that could not definitively establish a pattern, let alone a causal link. Still, several mainstream outlets rushed to cover it with some alarming headlines, infuriating scientists. Preprints could create similar pressures, and similarly flawed coverage — but much more often, and on a larger scale.

Another risk is the inverse — and this one could matter more to some researchers. Under the preprint system, one intrepid journalist trawling the servers can break a story; by the time other reporters have noticed, it’s old news, and they can’t persuade their editors to publish.

There have been cases in which a preprint that garnered news stories got a second flurry of coverage when it was published in a journal. But generally, the rule is ‘it has to be new to be news’. Reacting to our blog, Tom Whipple, science editor of The Times in the United Kingdom, tweeted: “I’m not sure how to keep a newspaper in profitable existence that decides to give people news they’ve already read on the BBC.”

It is not enough to shrug and blame journalists, and it is unhelpful to dismiss those journalists who can accurately convey complex science to a mass audience. Scientists need to be part of these debates — with their eyes open to how the media works. Journalists do include appropriate caveats or even decide not to run a story when conclusions are tentative, but that happens only because they have been given enough time and breathing space to assess it. If the scientific community isn’t careful, preprints could take that resource away.

How can we have preprints and support good journalism? Should scientific societies or preprint advocates develop guidelines for what should and should not be posted as a preprint? Should there be a preprint moratorium on any research with public-health implications? Should universities or researchers ever publicize a preprint? Should all preprints be emblazoned with a warning aimed at journalists that work has not been peer reviewed?

Preprints could bring great prizes for science. But these questions must be brought up now, so that public understanding is not damaged as preprints flourish.

Nature 559, 445 (2018)

doi: 10.1038/d41586-018-05789-4
Nature Briefing

Sign up for the daily Nature Briefing email newsletter

Stay up to date with what matters in science and why, handpicked from Nature and other publications worldwide.

Sign Up

Updates & Corrections

  • Correction 25 July 2018: An earlier version of this article omitted to mention that the Science Media Centre is partially funded by Springer Nature, Nature's publisher.

Competing Financial Interests

Funders of the Science Media Centre include Springer Nature, Nature's publisher.