Does peer review help or hinder science?
How fair is peer review? Is it merely a means for scientists to block the presentation of competitors' research results to the community? Should peer review be more transparent? These are recurrent questions the scientific community and publishing industry face.
Several large surveys over the past 5 years have assessed the attitudes of authors and referees across a broad range of scientific disciplines to the peer review process. The results, published in the 2007 Peer review in scholarly journals: Perspective of the scholarly community—an international study (http://www.publishingresearch.net/documents/PeerReviewFullPRCReport-final.pdf) and Peer Review Survey 2009 (http://www.senseaboutscience.org.uk/index.php/site/project/29/), suggest that peer review does indeed serve an essential role in scholarly communication. In these surveys, which garnered over 3,000 and 4,000 responses, respectively, over 90% of the authors stated their published work had been improved by the peer review process. The length of the review period drew more dissatisfaction from authors: the longer a paper was out to review, the more negative authors tended to be about the review process. For referees, who are authors as well, the willingness to contribute time and effort to review manuscripts came from the view that it serves a vital function in the scientific community. Referees felt their role is to determine the originality and importance of a given study as well as to assess its technical merit, which includes analysis of the experimental design and interpretation of the data. Contrary to views held by the general public, however, referees did not consider rooting out fraudulent data or plagiarism part of their role, although should this be obvious they would draw it to the attention of the editors. So why do questions persist regarding peer review? Does the process need to change?
Nature Immunology and other Nature-branded journals, like many other scientific journals, use a single-blind method of peer review in which author identities are known but the referees serve anonymously. Why not conceal both identities in a double-blind review process to provide fair, objective manuscript assessments? Hints of author identity are often provided in the literature cited or other clues in the manuscript itself, such as the phrase “we previously demonstrated....” In the surveys mentioned above, referees expressed the opinion that it is often too easy to identify authors in a double-blind manuscript review and thus it becomes essentially a single-blind process.
Open peer review, in which the referees are identified, has also been proposed as a way of introducing more transparency into the review process. Similarly, as more content is being viewed online, an open assessment period for papers under review in which community-wide comments can be voiced has become a possibility. In 2006, Nature experimented with an open peer-review approach (http://www.nature.com/nature/peerreview/debate/nature05535.html). During a 4-month trial period, Nature offered authors whose papers had passed the editorial triage stage the option of simultaneously undergoing the standard single-blind review process and having their manuscripts placed onto an open server for community comment. Only 5% of the authors agreed to take part in this experiment, for a total of 71 manuscripts. More telling, however, was the lack of comments received during this process, despite the publicity for the experiment and editorial prodding of key contacts in the community to participate in the open comment review. Web traffic to the open server was similar to that for RSS feeds, which suggests readers were aware of and interested in the experiment, yet only 92 comments were logged during the entire period; 33 papers received no comment at all. The 6 immunology papers involved in this experiment received an average of 1.5 comments apiece. Editorially, the public comments for the most part were not technically substantive and did not influence decisions about the manuscripts. Nature discontinued the project.
However, the peer-review process can be improved. One aspect is timeliness. Nature Immunology asks referees to return their reviews within 10 days of accepting the invitation to review. Author (and editor) frustration occurs when referees do not return their reviews on time. Some authors infer that a hostile referee is holding up their work. However, it is most likely that the manuscript is merely sitting unread on the referee's desktop because of other referee commitments, such as grant writing and lab management. Referees likewise complain that they are overextended. Most referees, given their experience in a given field, are called to act in many other professional capacities. Editors are cognizant of the demands on referees' time. For this reason, editors triage manuscripts rather than sending all manuscripts out to external review and assess manuscript revisions to determine the need for additional review. Both steps diminish the workload placed on referees. However, referees need to know that it is fine to decline a review request if time constraints are an issue.
Other considerations to improve peer review include prioritization of data presentation. The ever-increasing amount of supplementary data has prompted concern. Authors should not use manuscripts as a laboratory notebook 'dump', describing every experiment of the study. Likewise, referees should offer advice on the main points and conclusions that can be drawn from a study and not request data about tangential issues that might interest a handful of experts, and they should refrain from raising the bar for revisions. Referees should identify other papers that compromise the novelty of a study if this might preclude publication.
Rather than viewing it an adversarial contest between authors and referees, most researchers acknowledge this process is fair and beneficial. Peer review does improve the quality of science. Hence, until the community rejects the present peer-review process and provides an acceptable alternative, this constructive means of manuscript evaluation will continue.