Journal experiments and surveys suggest scientists are coming round to the idea of publishing review reports.
When Kevin Sinclair reported that 13 cloned sheep his lab had studied lived long and healthy lives1, he wanted to be as transparent as possible about what has been a controversial research area. Sinclair had invited journalists to see the sheep while the experiment was in progress at the University of Nottingham’s School of Biosciences in Loughborough, UK. And when his paper was published, in July 2016, he agreed that its peer-review reports should also be made public.
The developmental biologist was taking part in a trial by Nature Communications, in which the journal offered authors the option to have their reviews published. The goal was to find out whether scientists would see the practice as a way to make research more reliable and egalitarian — or as a needless nod to transparency that could harm peer review.
A few journals, such as PeerJ, the BMJ and F1000Research, already embrace open peer review in various different forms. Some forbid it. Other publishers and journals, including Nature Communications, are treating the practice as a frigid swimming pool: they are dipping their toes in the water, but are reluctant to plunge in.
So far, scientists seem willing to give open peer review a try. On 10 November, Nature Communications announced that around 60% of its authors in 2016 had agreed to have their reviews published, and that it would therefore continue to offer scientists the option — although would not make it mandatory. (Reviewers can choose to withhold their names, but cannot otherwise influence the process, besides decline to take part altogether in an 'open review' paper).
Meanwhile, an online survey funded by the European Commission (EC), which is not yet published, has found that more than half of its 3,062 respondents thought that open peer review should become routine, although they expressed some qualms about specifics (see Graphic, 'Opening up peer review'). More than half said that open reports would make peer review better; nearly 20% said that this would make it worse. Almost half worried that making the identity of reviewers known would worsen the process.
One challenge is that the concept of open peer review means different things to different people, says Anthony Ross-Hellauer, an information scientist at the Göttingen State and University Library in Germany, who ran the survey for the EC-funded 'OpenAIRE' project on open science. Some think that it implies only naming the reviewers, but not making their reports available; others think that unsigned reports should be public, and still others argue for conducting the entire process of peer review openly and allowing anyone to contribute. “It makes it really difficult to talk about what works in what circumstances if we’re not using the same language,” Ross-Hellauer says.
Some scientific communities seem to embrace open review reports more than others, notes Joerg Heber, executive editor of Nature Communications. During his journal’s trial, authors on more than 70% of eligible papers in ecology and evolution, molecular biology and Earth sciences adopted open reports, whereas physics papers saw the lowest uptake.
For nearly two years, Dutch publisher Elsevier has published unsigned peer-review reports for five of its titles, including Engineering Fracture Mechanics and the International Journal of Surgery. On the basis of that trial, which 33% of editors said led to better reports, Elsevier plans to bring open review to other journals next year.
And some journals allow reviewers to post their pre-publication reviews at other websites. One of those is Publons.com, which encourages scientists to make their peer reviews public, if journals permit. So far, says the site’s co-founder Andrew Preston, almost 500 journals have chosen to set specific permissions policies for Publons, and 16% of these allow the full text of reviews to be posted there.
Advocates say that the benefits of open peer review are straightforward. “It’s about making the process fairer and more transparent, so people can be held accountable if something goes wrong,” says Jonathan Tennant, a palaeontologist at Imperial College London and communications director at ScienceOpen, an open research publishing network.
But Stephen Heard, an ecologist at the University of New Brunswick in Fredericton, Canada, has misgivings. He worries about scientists posting peer-review reports without the knowledge or permission of reviewers. Heard also says that if he knew his reviews would end up public, it would make the job harder because he would feel obliged to cut back on technical language and jargon to make the review readable to a wider audience. “I would inevitably do fewer reviews,” he says.
It’s also not clear who actually reads the reports once they are made public, Heard says. Tennant says that they’re the first thing he reads after a paper’s abstract, and Heber says that download figures will be assessed by Nature Communications. (The journal is published by Nature's publisher, Springer Nature; Nature’s news and comment team is editorially independent of the publisher’s research editorial teams).
Sinclair has not yet heard from anyone who has read the peer-review report to his paper. And although his lone experience was encouraging — the reports were largely positive, and the paper sailed through review — he is not ready to become an open-peer-review evangelist. “I think most people, and I would include myself, are apprehensive,” Sinclair says. “This has been a new experience. As far as I’m concerned, the jury’s out.”
Sinclair, K. D. et al. Nature Commun. http://dx.doi.org/10.1038/ncomms12359 (2016).
Related links in Nature Research
Let’s make peer review scientific 2016-Jul-05
Peer review: Troubled from the start 2016-Apr-19
Publishing: The peer-review scam 2014-Nov-26
The scientists who get credit for peer review 2014-Oct-09
Related external links
Rights and permissions
About this article
Cite this article
Callaway, E. Open peer review finds more takers. Nature 539, 343 (2016). https://doi.org/10.1038/nature.2016.20969
This article is cited by
Is peer review ergodic?
Nature Physics (2018)