Editorial | Published:

Perfecting peer review?

Nature Medicine volume 17, pages 12 (2011) | Download Citation

Subjects

Online science blogs are a valuable forum for commenting on published research, but their present importance lies in complementing rather than replacing the current system of peer review.

A recent article in Science reported the identification of bacteria that could incorporate arsenic into its DNA, suggesting that phosphorus might not be an absolute requirement for life. What followed was a torrent of criticism in the blogosphere of the authors' methods and conclusions and of the review process that led to the acceptance of the article for publication.

With the pervasiveness of the Internet, and the speed of communication it permits, commentary and criticism of research findings can occur almost immediately after their online publication. This medium should be actively embraced by the research community as a dynamic forum for discussion and contribution to scientific deliberation. It can enable anonymous participation in discussion without fear of retribution, and its accessibility equalizes the playing field of scientific discourse—whether you are an undergraduate or a Nobel prize winner, if you have something interesting to say, your views will be read.

But the blogosphere is not without its disadvantages. Although the Internet embraces free speech, it does not impose any requirement of assuming innocence before guilt (or technical sloppiness before fraud), and discussions can range from polite and constructive to openly hostile. Scientists' online discussions should be fostered and ultimately should complement (even if they often don't compliment) published research, but formal peer review nevertheless remains a valuable means of adjudicating what should be published and where.

We are not suggesting, however, that peer review is infallible. Nonetheless, as editors, we hope that anyone accepting an invitation to review a manuscript considers that commitment as being of comparable importance to the other responsibilities of a busy researcher. And although we know that more pressing issues can take precedence over reviewing a manuscript, we still expect that the same level of integrity and objective, critical analysis will be applied to the assessment of the manuscript under review as is applied to the referee's own work.

We know, however, that at times these assumptions are not borne out. A reviewer may have taken on one too many manuscripts to commit the necessary time or level of attention that we, and the authors, anticipate. An inherent bias for or against a particular topic, hypothesis or even author might sway a reviewer, consciously or not, in one direction, independent of the strength of a paper's results.

We hope that such instances are in the minority, but to ensure the goal of objective peer review, it is our job as editors to guard against them. To do so, at Nature Medicine we subject all manuscripts to several stages of evaluation. Like the other Nature research journals, we do not have an external editorial board. Because professional editors have no research commitments or agenda, we feel that the absence of an editorial board minimizes the possibility of any external bias in the initial decisions that we make.

Instead, submissions are allocated to six manuscript editors according to subject areas, broadly defined as cancer, neuroscience, physiology, cardiovascular, basic and applied immunology and infectious disease. Each editor is charged with assessing his or her assigned manuscripts in their entirety and in terms of the conceptual novelty, advance, strength of the findings and the degree to which the work fits the purview of the journal—that is, demonstrating new insight into the mechanisms of human disease or its therapy. We then meet several times each week to discuss the new submissions and determine which best meet the criteria for external review (see http://www.nature.com/nm/journal/v10/n1/full/nm0104-1.html).

These meetings are lengthy, can be highly contentious at times and are a continual test of our own skills of critical analysis as well as the papers we discuss. In this manner, a decision on each manuscript is not meted out by a single editor, but instead is based on the combined input of the six manuscript editors and our chief editor.

For manuscripts that are externally reviewed—all primary research papers, as well as Reviews, Commentaries, Perspectives and most Correspondences—the editors select appropriate experts in the field, evaluate their comments and concerns and draft a final decision in consultation with the other editors. The majority of manuscripts that we publish are subjected to at least two rounds of peer review. Our production editor carries out a final check on figures slated for publication to ensure that no image manipulation exists. The process ultimately subjects a published manuscript to varying degrees of scrutiny by about ten individuals.

Can this elaborate process of peer review nevertheless miss fatal flaws in a paper or, conversely, overlook the importance of a new discovery? It can. And the blogosphere can inform us of such instances. Yet we believe that the multiple stages of independent evaluation of a manuscript minimize these occurrences, and the majority of comments from external referees reflect a careful and detailed analysis of the data and their interpretation. In contrast, most blog-based online comments on a given manuscript do not—at this time—suggest a comparably rigorous and objective assessment, and they are generally not sufficiently constructive to help authors do the right experiments to address the key concerns. Can online review of biological research improve to the point that it supersedes the current journal-based evaluation system? Perhaps. But it will require a commitment on the part of the scientific community to devote the time and courtesy now accorded formal peer review to open online assessment of manuscripts that have not been subjected to any preselection criteria. In the meantime, we recognize online commentary for what it does provide: fresh voices whose insights will help improve the iterative processes of science and peer review.

Do you have comments on this editorial? If so, please go to our blog, http://blogs.nature.com/nm/spoonful/.

About this article

Publication history

Published

DOI

https://doi.org/10.1038/nm0111-1

Authors

    Further reading

    Newsletter Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing