Where a piece of research is published impacts on funding and career prospects. What is being done to ensure this journal's selection process is fair?
A legitimate question for editors at Nature Cell Biology is whether we are considering alternatives to the status quo of anonymous peer review, in particular, why we do not use a 'double blind' process (which received significant endorsement in a recent survey of the Publishing Research Consortium). The existing process, based on a thorough pre-selection by five full-time editors and subsequent external peer review by carefully selected referees, works well — individual stories of woe notwithstanding. After all, NCB papers attract a respectable amount of attention and citations.
We will always consider additions to the system; however, any fundamental changes must demonstrably improve the process. In the case of blinding referees to authorship, there are some pragmatic considerations: an expert in the field can often pinpoint at least a subset of the authors; even if they have not seen the actual data previously, cell biology is sufficiently specialized to allow identification of laboratories on the basis of approaches taken and systems studied (try it!). Also, acknowledgements or reference lists, which cannot easily be extricated from a manuscript, almost invariably reveal the authors. It is not feasible to review a stripped down version of the data — an important part of peer review is to assess the strength of the claims made and the discussion of the previous literature. A two-part process consisting of an initial double-blind review of the data, and later, review of the full manuscript would increase referee workload and delay the assessment. Another caveat is that withholding authors' names may compromise a referee's ability to declare conflicts of interest. Aside from any personal biases, double-blind review may circumvent the systematic gender or geographical biases in the referee process, suggested by some studies. However, at NCB we have not found compelling evidence for such bias. Acceptance rates can differ slightly between countries, but we see no convincing reason to attribute this to editorial or referee biases (see November 2003 Editorial).
Another interesting suggestion is to open up the referee process to the whole community. A recent Nature trial that allowed the online posting of attributable comments prepublication concluded that no significant enhancement of the information available to the editors was obtained (see January 2007 Editorial). However, alternatives such as posting comments without author attribution, will be worth exploring.
Following are some important aspects of the editorial process and ongoing enhancements:
Referee selection. Referees are carefully selected for their expertise. Differing opinions — naturally a likely cause of alarm for an author — is often the result of diverse expertise. It is the editor's role to assess a referee's report in light of their expertise. We explicitly request that referees declare all conflicts of interest and, if necessary, excuse themselves from review. Although conflicts of interest for authors are much publicized topics, they are even more important at the referee level, as the process is only unilaterally transparent. We certainly take note of cases where we suspect a reviewer's privileges have been misused and will not use such referees again. Note that our editors are also subject to similar conflict of interest rules. Importantly, we honour referee exclusions as far as practicable. We do acknowledge that the ratio of referees to authors from English-speaking countries and continental Europe is slightly higher relative to other countries. On the other hand, the gender distribution of our authorship and referees is similar. We make a point of recruiting referees with a wide range of academic seniority: in our experience this is not a prerequisite for acumen as a referee. We will continue to select referees primarily on the basis of their expertise and proven ability to provide a fair and informed, but incisive evaluation. However, we are increasingly trying to attain a more balanced geographical distribution of our referee pool and certainly welcome nominations.
Arbitration. Within reason, we are open to further discussion with authors after a decision has been made (see August 2006 Editorial). If necessary, we will conduct an additional round of expert arbitration. Importantly, we habitually send anonymous referee reports to all the referees and invite feedback, either before or after making a final decision. This additional level of control often significantly aids the decision-making process. To facilitate this further, we will soon enable referees to see each others' reports once they have been posted, and to comment through our online system.
Transparency. A valid suggestion has been to post the referee reports next to published papers. Obviously, this would not benefit rejected manuscripts, where this information would usefully underline our belief that decisions at the journal are fair. To some extent we have always published critical evaluations alongside papers in the form of News and Views articles, although of course, the authors were not necessarily the referees. However, aside from aiding the reader, publication of all the reports would add transparency and accountability. A purely pragmatic caveat is that, almost invariably, manuscripts at this journal are published after at least two rounds of review; the final set of referee reports are not usually information-rich, whereas the initial reports are not necessarily pertinent to the published paper. As such, these reports would have to be either updated by the referees to make them relevant to the published version, or read with caution. The latter option would serve to highlight how a manuscript is developed during review, in our experience an underappreciated aspect of the process. In fairness to the authors, their response to a report should also be published. Indeed, our online journal Nature Reports Stem Cells has taken a first step in this direction: 'Inside the Paper' contains the first three examples of stem cell papers with associated referee reports and author responses, as well as an editorial summary of the paper. In each case the referees and authors agreed to the release.
Commenting and voting. The Nature news section already allows readers to comment on its articles. A similar system will soon be implemented for the primary papers in the Nature journals, as well as the Molecule Pages database. It remains to be seen whether the level of commenting will be richer than that of the prepublication 'open peer review' trial. We have also started a voting system on some of our primary papers (see, for example, 'journal club' at Nature Reports Stem Cells), although the level of voting remains modest. A key issue is whether comments should be moderated or edited before posting.
We welcome your views on this and any of the other enhancements presented above, or indeed any additional suggestions you may have.
Further reading: http://www.connotea.org/user/ncb/tag/peer-review