Peer review of grants and manuscripts is at the heart of a generally robust system of self-policing and quality assurance in scientific research. Publications should represent the endpoint of a body of work and it is essential that the community can rely on the information provided. Although most journals strive to publish high quality papers, widely different editorial criteria apply yet and the resulting publication landscape is increasingly used to quantify performance. Reviews must be equally authoritative as they tend to establish dogma, which is later synthesized into textbook 'facts'. Thus, for both the primary and secondary literature, a rigorous peer review process is essential.

Most scientists are willing to sacrifice a considerable amount of time for peer review. This largely altruistic endeavour is remarkable; or are there tangible benefits in being a referee? Some speculate that referees receive preferential treatment; for the record: there are no kickbacks of any sort. Refereeing does offer benefits that require ethical commitment, given the confidential nature of the process: a referee is exposed to every nook and cranny of a potential competitor's research. It is only natural to use information obtained through peer review indirectly in guiding one's research; however, it is emphatically not acceptable for referees to use the information in a competing project in their own or their colleagues' laboratories or — worse still — to initiate a competing project. Thankfully, we have little evidence that this occurs; when we do, we will not use the referee again (although this remains journal internal information, unless proof of an ethical breach is obtained). The biggest benefit of refereeing is probably to stay engaged with the scientific community in a sort of 'real-impact journal club'. We review less than a quarter of submitted manuscripts, with the aim of involving referees only for studies that seem worthy of publication. This saves time for authors, and referees know that the manuscripts they receive meet a minimum standard, so that refereeing should be an enriching experience. Nevertheless, the time commitment for refereeing is considerable and we are campaigning to ensure that this is taken into account in assessing research performance.

At all Nature journals, manuscripts undergo stringent editorial pre-selection. Editors read each manuscript in its entirety and invest significant time in summarizing the data, assessing novelty and importance of a study, and validating their decisions with colleagues or outside advisors. Referee selection is at the heart of a constructive and ethical review process and we rely on a large pool of experts, ranging in levels of seniority and roughly reflecting the submission demographics to the journal (see Fig. 1). Thus, we are not dominated by individuals, although we strive for consistency in decision-making by recruiting a subset of overlapping referees on related manuscripts. In the March 2008 Editorial, we discussed how post-decision feedback allows for referee cross-evaluation. We hope that this clarifies the editorial process and answers the following recent query in response to a detailed decision letter: “was this decision based on the abstract/summary alone?”

Figure 1
figure 1

Refereeing frequency and geographical distribution of referees (March 2007–2008)

Full size image

We ask referees to declare any financial, personal or scientific conflict of interest, and to consult with the editor to alleviate any ethical concerns, before they review a manuscript. We request a report within two weeks so it is fine to decline (but helpful to suggest alternative referees). Remember that accepting to review also entails the re-evaluation of substantial revisions.

When assessing the manuscript, do so first from an editorial point of view. Summarize the key findings of the paper and outline its significance. Is the conceptual advance over the published literature sufficiently striking? Are the depth, breadth and quality of the data exceptional? Does it settle a longstanding question or a prominent controversy? Are the conclusions of broad interest or better suited for a specialist audience? By all means add editorial recommendations (also on issues of style). Nevertheless, the most important part of the report is assessment of the data: are key experiments or crucial controls missing? Are the data significant and definitive? Are all claims made supported by the data? A black and white model should not be a requirement — in fact it often underestimates biological complexity. Formulate a clear set of recommendations for additions or improvements. Experimental suggestions are important even if an outright rejection is recommended, as it makes for a transparent and constructive report that will allow the author to improve the dataset and select an appropriate target journal. If the overall assessment is positive, do not feel obliged to ask for non-essential experiments. New experiments should add key information, buttress claims or improve existing data, and they must be achievable. Indeed, the perception that referees invariably ask for more data encourages some authors to hold back data in the hope that these will be requested and can then be provided by return post; clearly, this is a futile exercise. Referees and editors alike should take great care that all key issues are raised in the first assessment — draw a line and adhere to it. Subsequent evaluations may raise issues on new data, but raising yet more ways to develop a study is not fair. If a key caveat was overlooked, do raise it though, as it is certainly not in the authors' interests to publish flawed studies. Carefully crafted detailed reports are more compelling; a misplaced word can influence a decision and a hastily assembled review can be galling to an author who has spent years on a project. Confidential comments to the editors allow the referee to outline any less formal or more sensitive issues, but they are not obligatory and it is important not to forward an opinion that is at odds with the referee report.

Thanks to all our referees for their constructive and informed reports — the journal depends on your help. We hope you receive as good as you give when you submit your work.

Further reading: http://www.Connotea.org/user/ncb/tag/peer%20review