Systems: Reviving a culture of scientific debate

Nature (2006) | doi:10.1038/nature05005

Can 'open peer review' work for biologists? Biology Direct is hopeful.

The advent of immensely powerful means of communication in our information age offers unprecedented opportunities for experimentation with new approaches to scientific publishing. In an attempt to offer the scientific community an alternative to the current peer-review system, we recently launched a new journal, Biology Direct.

In Biology Direct, everything happens in the open: the authors select their own reviewers from the editorial board, and the reviews are not only signed but also published, alongside authors' responses, as an integral part of each article. The reviews can be critical or even outright negative. The only condition of publication is that three members of the Biology Direct editorial board become sufficiently interested in a submission to either review it themselves or to solicit a review from an outside expert. Conversely, a paper is rejected if and only if the author cannot get three reviews. Obviously, the authors can 'self-reject', that is, they can withdraw their manuscript if they are not comfortable publishing it having considered its reviews.

The benefits of an open system

This system overcomes the well-known ills of anonymous peer review and is, we believe, fair to both authors and reviewers. The authors gain the opportunity to discuss their work with reviewers in the open and to deal with reviewers' suggestions on their perceived merit, without fear of rejection affecting the response. The reviewers gain public acknowledgement for their often hard work, with the results made available to the readers. Furthermore, we believe that the Biology Direct system is conducive to publication of innovative, bold papers that might have a hard time getting published in the current system. Perhaps most importantly, the Biology Direct approach to peer review could help to revitalize the culture of scientific debate that is waning in the uneven duel between omnipotent, anonymous reviewer and helpless author.

Of course, there are caveats and dangers as well. It is possible that, in this open model, many reviews will not be as brutally honest as they would have been if anonymous. In the worst-case scenario, mediocre papers with lukewarm reviews would flood Biology Direct. Furthermore, some erroneous papers might appear with reviews that are too polite or cryptic for the reader to see the red flag immediately. But Biology Direct investigates an alternative approach to scientific peer review and is not, by any means, an attempt to replace the existing system.

Biology Direct in practice

So what are our findings since the launch of Biology Direct in January 2006? With the full understanding that far too few papers have been published or even submitted to allow any definitive judgments, we can try to discern some early emerging trends.

First, strong interest in the approach is indicated by the enthusiastic response of many highly reputable scientists who have agreed to join the editorial board in the fields of genomics, evolutionary biology and bioinformatics. Moreover, a parallel and equally impressive editorial board has emerged for the immunology section, purely as a grassroots movement (Neil Greenspan and David Kaplan of Case Western Reserve University in Cleveland, Ohio, are the editors of this section).

At the time of writing, Biology Direct has published ten articles (not counting the launch editorial), and three more have been submitted with reviews (which means acceptance under the Biology Direct model). One of these papers is a review, another is a hypothesis, the rest are research articles. About 15 more papers are currently in various stages of the review process.

The Biology Direct system allows publication of papers with 'no comment' reviews, when a reviewer chooses that option or when they miss the deadline for review submission. However, among the published reviews, only one is a 'no comment' so far. And an informal survey shows that every paper is accompanied by at least one 'genuinely constructive' review, a reasonably long review that includes serious discussion and substantial criticism. Most of the papers have two or three such reviews, and the authors almost always respond in kind, with substantive discussion and, often, counter-arguments.

It is noteworthy that one paper has gathered three reviews that challenge the central conclusions of the work (that is, would certainly qualify as negative by the usual standards), and two other papers have one or two such reviews. Of course, the process of publication in Biology Direct allows for one or more rounds of revision during which not only the manuscript but also the reviews may be modified, in particular to include the reviewers' reaction to the revision of the original manuscript. To our knowledge, all currently published papers, without exception, have been revised, and in a few cases, there has been intense back-and-forth.

From these initial experiences, we are under the strong impression that the review system of Biology Direct works: it offers interesting, constructive and, sometimes, sharply critical discussion of each published paper, adding significant value to the publication, and warning of serious problems perceived by the reviewers. Are the praise and the warnings prominent and sufficiently effective? This is still an open question. It is conceivable that, in the future, amendments to the model, such as explicit ratings, will drive home the message of the reviews for readers.

The principal mechanism of effective rejection of a submission to Biology Direct (apart from self-rejection, which has already occurred once) is when an author cannot find three reviewers for a manuscript and gives up (in an extreme case, having gone through the entire editorial board). In the best-case scenario, this rejection mechanism would reduce the number of routine, if technically sound papers, that make it into Biology Direct, as these might fail to elicit sufficient interest to secure reviews. Less optimistically, such a failure might simply reflect a lack of relevant expertise on the editorial board.

So far, we are aware of only a couple of failed attempts to get a manuscript reviewed for Biology Direct. But several authors submitting research seemingly within the scope of Biology Direct have had considerable difficulty finding reviewers. This is a reminder that the journal will only be as good as its editorial board, and we need to continue the effort to involve research leaders in diverse fields. Also, it is our hope that, as they gain experience, editorial board members will become increasingly active in their role as ad hoc editors soliciting outside reviews.

Perhaps the gravest concern associated with the liberal acceptance model of Biology Direct is that it might, in principle, allow more pseudoscience than traditional journals. To prevent this from happening, Biology Direct has an alert system so that reviewers identifying a submission that does not meet standards of science can alert the editors, who then reject the manuscript. In the single case that we have faced so far, the system worked smoothly, with three referees immediately spotting signs of trouble and notifying the editors.

The future of Biology Direct

We hope that Biology Direct will become a space for the publication of pioneering studies, hypotheses and research papers that might not (yet) pass the strict criteria at the best traditional journals but that offer radical ideas or open up new research directions. We may be seeing the first, tentative signs of this: one published paper is a decidedly unconventional hypothesis, accompanied by interested but duly sceptical reviews; another includes unusually broad speculation in an interpretation of comparative-genomic results. Most papers, however, look like business as usual: they are solid work that might easily have come out in more traditional journals. Perhaps this is a sign that Biology Direct will attract a healthy mixture of good, 'normal' science and bold ideas.

We believe there is cause for cautious optimism about the Biology Direct model. The editors are certainly getting a thrill guiding its baby steps. There is interest from the community (some articles have already been accessed more than 2000 time) and this is supported by strong positive feedback from several authors. We hope the Biology Direct approach has a future as a viable alternative to the current system of peer review.

RELATED LINKS:

Biology Direct

Eugene Koonin is a senior investigator at the National Center for Biotechnology Information, at the National Institutes of Health in Maryland (www.ncbi.nlm.nih.gov/CBBresearch/Koonin/). His long-term research interests are in evolutionary genomics.

David Lipman is director of the National Center for Biotechnology Information. His research interests are in molecular evolution and computational tools for sequence comparison.

Ros Dignon is in-house editor for Biology Direct at BioMed Central, and coordinates the day-to-day running and development of the journal. She can be contacted at editorial@biology-direct.com.

Laura Landweber is an associate professor in the Department of Ecology and Evolutionary Biology at Princeton University, New Jersey. She studies the origin and diversity of genetic systems in microbial eukaryotes.

Visit our peer-to-peer blog to read and post comments about this article.

Extra navigation

.
ADVERTISEMENT