Systems: Online frontiers of the peer-reviewed literature

Nature (2006) | doi:10.1038/nature05030

The Internet is allowing much more interactive science publishing

Online tools can be used to improve the accuracy, transparency and usefulness of the scientific literature by moving away from the traditional emphasis on closed peer review. Given the capability for post-publication amendment of articles, the scientific articles themselves and the peer-review process will soon be profoundly different from today's standard.

The traditional system of peer review entails careful scrutiny of each submitted article by at least two experts in the field, with the reviewers' names withheld from the authors and an editor deciding, in the light of reviewers' comments, whether to offer publication. The Internet is encouraging authors, editors and publishers to experiment with publishing models that deviate from this system.

Innovation in peer review

In addition to assessing scientific accuracy, many reviewers are also asked to judge whether a manuscript reaches the level of interest appropriate for a given journal. Some journals — such as the 60 titles in the BMC series published by BioMed Central — pledge to publish all sound research within the journal's scope, and not to judge by interest level. Does asking reviewers to judge primarily whether the work is sound, rather than filtering from among sound papers, affect how the journal is seen in the field and its success?

Almost all the BMC journals were launched at the same time, share open-access goals, and aim to cover similarly sized portions of the scientific literature, as well as sharing almost identical systems of peer review and having similar acceptance rates among submitted articles, but their 'success' — measured by submission, publication and citation rates — shows significant variability. For example, BMC Bioinformatics and BMC Public Health each receive hundreds of manuscripts each month and publish more than a hundred, whereas BMC Evolutionary Biology and BMC Family Practice receive and publish only tens of articles each per month. Given that the peer-review processes of these journals are similar, what is the reason for the variation?

One key difference in the peer review operated by two categories of BMC journal — 'medical' and 'biological' — is that the medical journals operate 'open' peer review, where reviewers are known to the authors and, in the event of publication, the reviewers' names and reports are published with the article, whereas the biological journals operate more traditional 'closed' peer review, in which reviewers' names are withheld from authors. Our experience with the BMC series bears out the BMJ's previous experience that peer reviewers are more reluctant to serve in an 'open' system. (See article in this debate by Groves. We have found, however, that there can be greater variation in the proportion of invited reviewers who agree to serve between two BMC-series journals that operate open peer review, or two journals that operate closed peer review, than between all open-review and all closed-review BMC-series journals.

A recent study of BMC journals (E. Wager et al. BMC Medicine 4, 13;2006) shows that author-nominated peer reviewers provide reports of similar quality and tone to reports by editor-chosen reviewers (although the former are less likely to recommend rejection during the initial round of review) . In this study, 'quality' was assessed blind by two raters, using seven indicators of review quality (such as 'assessing originality' or 'providing constructive comments'), each rated on a five-point scale.

A further experiment in open peer review, with author-nominated reviewers and entirely open communication between the author and reviewers (with very little intervention by editors), is being undertaken by the journal Biology Direct (see article in this web debate by Koonin et al. )

Use of the Internet also means that peer review need no longer be a one-step process: preprint servers and online comment and reviewing systems allow an article to accrue commentary and analysis after it is published. This ranges from the 'post a comment' option operated by many online journals to the two-stage system used by Atmospheric Chemistry and Physics (see article in this web debate by Koop and Poschel. Further and wider experimentation in this direction has been urged by Van de Sompel and Ackerman elsewhere in this debate.

Definitive version?

But if an article can be amended and commented on after publication, what should be seen as the definitive version? Some journals, like FASEB Journal and Genome Biology, already provide different versions of a single article for print and online audiences. Indeed, any article that includes a database or movie available only online can be considered in this light. Many traditionalists object to the idea of multiple versions of scientific articles, condemning them as duplicate publication of the same data. But users of software are familiar with the idea that version 1 is modified through versions 1.1, 1.2 and so on, until a new version 2 is released. If some data have been wholly reanalysed, or indeed if a research article describes a software tool rather than a data set, is it not reasonable to publish a second article describing progress at some point, or a new version of the first article? For example, Nucleic Acids Research publishes an annual 'Databases' issue that this year included articles describing updates to 68 databases. These articles are seen as wholly new, but each could in fact be described as an update of the previous year's description. Indeed, it might be easier to navigate the literature were such articles to be more explicitly branded as such.

In considering how such new versions and updates should be peer reviewed, most journals would expect to operate stringent peer review for a completely new article describing a whole new version of a piece of software or new analysis of data, but many would use a slightly more relaxed consultation with expert reviewers when an addendum or correction is made, by using advice from only a single reviewer, and/or looking only for exceptional reasons not to publish (see article in this debate by Riley).

The traditional one-version-only view of a research article as a snapshot of the information about a particular problem is also challenged by large-scale studies, such as clinical trials in medicine or genome-wide analyses in biology, descriptions of which almost always constitute a 'work in progress' rather than a definitive and final view.


BioMed Central BMC series


Genome Biology

Nucleic Acids Research

Theodora Bloom has editorial responsibility for biological publishing at BioMed Central. She has a PhD in developmental biology and post-doctoral experience in cell biology. She joined the editorial staff at Nature, then moved to Current Biology, and became publishing manager for Current Biology and its siblings. From Elsevier Science London she rejoined the Current Science Group in 1999, shortly before it launched BioMed Central.

Visit our peer-to-peer blog to read and post comments about this article.

Extra navigation