A guide surveys the range of practices in Europe — and offers suggestions for improvement.
Your cross-disciplinary grant application has been rejected. Do you wonder if specialist reviewers really grasped its scope? Would you feel more, or less, confident in the process if reviewers were paid?
A report from the European Science Foundation (ESF) offers guidance on the fairest ways to evaluate project proposals. The European Peer Review Guide , out last week, maps grant-reviewing practices among European funding agencies and sets out recommendations.
Europe is a kind of laboratory for peer review, with programmes that span disciplines, and countries with varying evaluation criteria, review procedures and incentives for reviewers. There is no single correct model, says co-author Marc Heppener, the ESF's director of science and strategy development. "The report refrains from using the term 'best practice', and refers instead to 'good practice'," he says.
The report lists commonly used options for everything from managing conflicts and confidentiality to organizing documentation. It lays out different methods of selecting experts and provides clear definitions of common terms such as 'excellence' and 'transparency'. And it offers specific advice, such as the need for gender balance among reviewers and for incorporating right-of-reply in the decision-making process. As for paying reviewers, it recommends not doing so unless "really necessary".
Heppener hopes that the guide will be useful to research agencies around the world. But it was commissioned in 2009 for a narrower purpose by the heads of the European research councils. They realized that better-harmonized peer-review systems would make it easier to share grant-proposal reviews, a pressing need as the number of international research programmes grows. "National research agencies were aware that reviews of a proposal made in one country may not be valid in a second country, and this was frustrating," says Heppener.
As well as giving a how-to guide for peer review, the report pays special attention to research projects at the intersections of traditional disciplines. It describes ways to assess the quality of all parts of any 'pluridisciplinary' project, and the value of integrating them, with a minimum of peer-review stages.
The European Peer Review Guide is a rolling report, which will be regularly updated as agencies modify their procedures, adopting options that suit their own cultural constraints. Some agencies in large countries, for example, rely mostly on national reviewers, whereas others in small countries may be obliged by law to include international reviewers.
"It's a useful and important report, particularly for those of us who want to fund global projects," says David Stonner, deputy director of the office of international science and engineering at the US National Science Foundation (NSF), who was a formal observer of the procedures for creating the guide. "We need a common language and this report will get a lot of attention at the highest levels of the NSF."
Related external links
About this article
Cite this article
Abbott, A. A how-to for peer review. Nature 473, 17 (2011). https://doi.org/10.1038/473017a