Researchers coy about complete review transparency

Survey reveals reluctance to take open peer review to the limit.

28 February 2018

Ikon Images / Alamy Stock Photo

Publishing reviewers’ reports is likely to become more common, says Tony Ross-Hellauer, the author of a global survey, which found evidence of changing attitudes among scientists towards such practices.

Editors, authors and reviewers who responded to the survey strongly supported open peer review, but only if it is carried out anonymously. The results were published in PLoS ONE and reviewed by two named reviewers.

The online survey was funded by the European Commission as part of Europe’s OpenAIRE project to promote the adoption of open access, and conducted between September and October 2016. It found that 60% of 3,000-plus respondents believe open peer review should be mainstream scholarly practice, and even more, 76%, had taken part in the process, in various forms.

But respondents did not commit to the concept unconditionally. More than half supported direct interaction between authors and reviewers, publication of referee reports together with articles, and contributions by the wider community. The majority, however, also felt that disclosing referees’ identities would be harmful to the process.

Traditionally, scholarly journals have used the single-blind peer reviewing system, where reviewers know the identity of authors but not vice versa. The benefit of such a model is that it helps reviewers contextualise the study with the authors’ previous work. But the downside is that it can lead to bias against less prominent researchers, institutions and nations. Many publishers have therefore begun to experiment with different peer review models.

Post-publication peer review platform Faculty of 1000 (F1000), for instance, ascribes to complete transparency, meaning that reviewers sign their reports, which are freely accessible online. Critics argue, however, that complete openness could result in academics being less critical of senior colleagues’ work. But this problem can be overcome “through stronger credit for researchers contributing to peer review”, argues Rebecca Lawrence, managing editor of F1000. Scholars can claim credit for reviews of F1000 papers on their ORCID profile and at the site Publons, and now some of their reports are being cited, Lawrence says.

The opposite extreme, double-blind peer review, where identities of both authors and reviewers is kept anonymous, has also been tried as an alternative to the single-blind system. In a 2009 survey of 4,000 researchers, 76% said double-blind was an effective reviewing system, compared to only 20% approval rate of open reviews.

Journals under the Nature stable started offering the double-blind system in 2015. But uptake has been relatively low. A manuscript posted on the preprint server arXiv in February 2018 revealed that only 12% of authors of more than 125,000 papers published in Nature journals between March 2015 and February 2017 opted for double-blind. Ross-Hellauer, who studies open scholarship at the Know-Center in Graz, Austria, thinks the double-blind system offers “false confidence” since referees can often accurately predict authors, particularly in smaller fields.

The tables are turning, says Andrea Taroni, chief editor of Nature Physics. “The research community is much more supportive of transparency than we found them to be in the past,” he says. A recent trial at Nature Communications found 60% of authors opting to have their papers’ reviews available online. Another trial at Nature revealed that 54% of referees are happy to be accredited for their reviews, Taroni says.

Nature Index is editorially independent of the research publications section of Nature Research.