Considered and critical assessment of a manuscript is essential to peer review and the publication process, but what makes a good referee report? We highlight the central elements of the ideal referee report.
Commenting on published papers is a useful means to update, validate and discuss the scientific literature. However, despite the growth of various post-publication forums, including Faculty of 1000, scientific blogs and online commenting alongside published papers, peer review remains the foundation of publishing in the life sciences. Certainly, the process is not without flaws and we have written in these pages about steps taken by this journal to ensure that it remains rigorous and constructive.
High quality refereeing is vital for a robust peer review process. So, what constitutes an ideal report? Most importantly, the report should provide an objective and thorough evaluation of the study's potential impact, judged in the context of the published literature, and its technical aspects. Careful scrutiny of the data to determine whether they substantiate the main conclusions of the work, accompanied by specific recommendations to fill gaps is essential. Other key issues to highlight in the report include the data quality and presentation, methodology, statistical analysis, strength of the conclusions, clarity of the text and appropriate citation of the literature.
Concerns regarding the potential impact and novelty of the study must be raised in the first round of review. In general, we ask that referees restrict their comments in subsequent rounds of review to the strength of the revisions and the new data provided; however, serious limitations that weaken the impact of the work and that might have been missed in an initial round should be noted in a subsequent report. Agreeing to review a new submission is taken as a commitment to providing input on revisions of the study, as without this policy it would be difficult to maintain consistency in subsequent review cycles.
An assertive but civil tone in reports is greatly appreciated by editors. Indeed, antagonistic comments that create a perception of referee hostility can undermine the strength of the report. All comments pertaining to the suitability of a study for publication or further consideration should be transmitted to the authors, and the overall recommendations provided to editors should be consistent with the feedback provided for authors. A scathing assessment in confidential comments alongside a more lenient evaluation for the authors that glosses over major deficiencies and seems to be supportive of publication creates confusion for authors in the event that a manuscript is rejected based on these confidential comments. Such discrepancy is a rare occurrence and editors will seek further clarification from the reviewer. Confidential comments to the editor must be reserved for discussion of matters that are of a sensitive nature, including suspicion of fraud, data manipulation or duplication.
Referees are provided feedback on the decision reached by the editors and they receive all the reports associated with a manuscript. They can also provide feedback on colleagues' reports if, for example, there are specific points that they wish to reinforce, or if a colleague's views are perceived as unduly harsh or factually erroneous. In cases where a reviewer is convinced that a study does not provide the requisite advance for this journal, a brief report that does not focus on the experimental limitations in detail but is nevertheless accompanied by more general recommendations for improving the study for publication elsewhere is acceptable. By and large, reports of this nature are rare at this journal as the editorial vetting that manuscripts undergo before peer review increases the likelihood that reviewers receive manuscripts that meet the criteria for consideration at the journal, thus saving referees and authors valuable time.
There are growing concerns that the quality of the peer review process is being eroded as referees are stretched thin by other professional commitments. This can also lead to challenges in finding reviewers in a timely manner, particularly before busy grant deadlines or during conference seasons. How can we continue to ensure that refereeing remains robust in the face of these challenges? At this journal, we are always seeking to expand our referee pool to avoid referee fatigue. Reviewers can co-review manuscripts with senior members of their laboratory; the principle reviewer must inform the editors of the co-reviewer's identity and co-reviewers are obviously held to the same standards of confidentiality. Editors also assess the strength of revisions before contacting referees for re-review. Once again, this practice is intended to eliminate futile cycles of review and to save both reviewers' and authors' time.
Recruiting and retaining high quality referees is a priority for journals, but the peer review process lacks more formalized mechanisms for recognizing the contributions of referees. Refereeing provides an opportunity to participate and contribute to the scientific discourse and access to some of the best work before it is publicly available, and we hope that there is satisfaction to be gained from that. Some options for appreciating referees' efforts include providing modest monetary rewards, generating metrics for referee performance, creating referee awards, and increasing acknowledgement of referee contributions by journals, funding agencies and research institutions. However, none of these avenues has gained sufficient traction to be adopted widely. Some journals publish an annual roster of their reviewers and that may be one simple means of crediting referees. We certainly recognize that providing a thoughtful and detailed report requires time and effort, and we are grateful to all our reviewers who have shared their expertise to strengthen the papers published by this journal. We welcome suggestions and ideas from our readers on ways to provide recognition for referees' contributions.
'Comment' in Nature Cell Biology
In 2011, we plan to launch a new section titled 'Comment' that will feature short essays. They are intended to be provocative and potentially agenda-setting pieces on topical issues pertaining to all aspects of research in cell biology. These essays will largely be written by members of the academic community and will replace the editorial in some issues. We view the creation of this section as a means of providing the research community with a direct voice in this journal.