Researchers on social media discuss the potential impact of making peer review more transparent.
Many scientists struggle to understand why some grant applications succeed and others fail, perhaps explaining the online popularity of two articles calling for increased transparency in the grant peer-review process. Researchers are also talking about changing peer review at the other end of the research spectrum: after the results are published.
Writing in PLoS Biology1, Daniel Mietchen, an evolutionary biologist at Berlin’s Museum of Natural History, argues that, among other things, all successful proposals and their reviews should be released to the public. Dorothy Bishop, a neuropsychologist at the University of Oxford, UK, tweeted:
But some see a downside to transparency. James Johnson, a diabetes researcher at the University of British Columbia in Vancouver, Canada, tweeted:
Mietchen makes the case that lifting the curtain on the research-funding system could transform science as a whole. Publishing proposals as soon as they are submitted and seeing which ones were rejected could help scientists to see how their work stacks up against that of their colleagues. It could also open up new avenues for collaboration and allow others to build on failed proposals, he wrote. Even small steps — such as publication of all successful proposals years after the projects have ended — could be an important move towards that goal, he says.
Mietchen’s piece was a response to an investigation of transparency in grant funding, also published in PLoS Biology2. A survey of 27 funding agencies worldwide found that none of them published their final assessments of successful proposals. Eighteen out of 27 released scientific abstracts of successful applications. In general, the agencies were much more open about the rules of the grant-review process than about any details of research projects or funding decisions. This secrecy, the authors of the investigation argue, helps to promote inconsistencies and unfairness in the grant evaluation system.
Lisa Mangiamele, a neurobiologist at Smith College in Northampton, Massachusetts, tweeted that the survey was “essential reading for new investigators”. Reached for comment, she says: “The idea that the peer-review process is inconsistent at best — and biased at worst — is worrisome. But with greater transparency, it doesn’t have to be that way.”
Johnson agrees that openness is generally good for science, and he says that he often shares ideas with colleagues at conferences, even before he has started writing his grant proposals. But he is concerned that if all proposals are published, unscrupulous scientists could swoop in and claim ideas as their own. “There are dangers in our extremely competitive research environment,” he says. “I try not to be naive, and I try to set an example of ‘careful openness’ for my fellows and students, for whom the stakes are higher.”
The Wellcome Trust, a major UK biomedical-funding agency, already publishes short summaries of many of its awarded grants, according to Kevin Moses, the organization’s director of science. However, he adds, “We are not planning to publish full proposals as candidates have a right to the confidentiality of their unpublished ideas and data.”
Peer review at the other end
Opening up peer review proved a popular topic on social media, including the type that happens after publication. In an editorial in PLoS Medicine3, Hilda Bastian argued that much of the discussion surrounding published papers is fragmented, hard to access and ultimately lost to history. Bastian, editor of PubMed Commons, a website for posting comments on PubMed research papers, says that “a stronger post-publication culture” — including a greater emphasis on thorough, non-anonymous critiques — could improve the quality of science by quickly bringing errors and weaknesses in research to light.
Lisa Langsetmo, an osteoporosis researcher at McGill University in Montreal, Canada, tweeted that peer review should happen in two phases.
In an interview, Langsetmo says that there should be an expert review board that openly assesses all high-impact articles. It would be especially helpful, she says, if reviewers of health-related studies took a hard look at the leaps from results to conclusions. If those leaps don’t hold up to scrutiny, she says, the public has a right to know.
Mietchen, D. PLoS Biol. 12, e1002027 (2014)
Gurwitz, D. Milanesi, E. & Koenig, T. PloS Biol. 12, e1002010 (2014)
Bastian, H. PLoS Med. 11, e1001772 (2014)
Related links in Nature Research
Related external links
About this article
Cite this article
Woolston, C. What would happen if grant reviews were made public?. Nature 517, 247 (2015). https://doi.org/10.1038/517247f