Review by one’s peers has been a keystone of scientific progress since before the word ‘scientist’ was coined, but it can be an abrasive and dispiriting experience. How do young career-scientists think it can be improved?
Twitter is becoming an excellent medium for communication among scientists. Following the many plant researchers with active twitter accounts, hashtags such as #plantsci, and organisational feeds like @plantae, @GlobalPlantGPC or @NaturePlants, can help find interesting new studies, eavesdrop on conferences we cannot attend, hear about job vacancies, and generally keep up with the goings on in our community. But you do not have to stray too far from the relatively polite and respectful ‘science twittersphere’ to find a more abrasive, rude and downright bullying environment .
The prevalence of bullying, discrimination and social abuse has been much discussed recently and science is in no way immune. The Max Planck Institute for Astrophysics in Garching, Germany, is currently involved in a high-profile case centred on allegations of bullying by its director (Abbot, A. Nature 559, 159–160; 2018) and there are less prominent examples even within the plant community. The toll this can take on young researchers is extremely high. Rates of depression and anxiety in postgraduate students are six times higher than in the general population (Evans, T.M. et al. Nat. Biotechnol. 36, 282–284; 2018). The situation for post-docs and other early-career scientists is unlikely to be much better.
Peer-review is another place where unkind, unethical and even abusive behaviours can manifest. Most researchers have stories of when peer-review has ‘gone wrong’, where reviewers’ comments have been unreasonable or rude, and grudges have been played out. There have been attempts to change the mechanics of peer review to somehow guard against abuse in the system. Conventional peer-review, in which a small number of ‘expert’ reviewers advise an editor who may understand little of the specific details of the work, is frequently characterised as slow, lacking transparency and conducive of unethical conduct by reviewers, shrouded as they are by a cloak of anonymity. While it is difficult to argue that such criticisms are never justified, there is little agreement on how to solve these problems. Many variations have been tried. For example, in June the journal eLife announced that it will be trialling a system where, once a decision has been made to review a study, its publication is assured if the authors are prepared to have referees’ comments, however critical, published alongside (Patterson, M. & Schekman, R. eLife 7, e36545; 2018).
The problems of peer-review and potential solutions, were the topic of a discussion session held at the recent Frontiers and Techniques in Plant Science course at Cold Spring Harbor Laboratory (CSHL). This course has run annually for over 35 years and provides postgraduate and early postdoctoral researchers a primer in modern plant biology in just three weeks.
This years’ attendees have produced a summary of their thoughts on the problems inherent in peer review (https://go.nature.com/2uZ3PuS). They assembled a pros-and-cons list to guide their exploration of potential improvements or alternatives and immediately focussed on the role of anonymity, which it quickly became apparent could be a double-edged sword. An anonymous referee might feel freer to express candid opinions, but they may also use that anonymity to be inaccurate, unfair or obstructive, safe in the knowledge that this will have no effect on their own work and reputation. Conversely, if reviewers are not anonymous they may be reluctant to be critical of work from close colleagues and friends, or to review such work at all, and could expose themselves to bullying and coercion from unscrupulous authors.
Perhaps blinding authors would help by making the power dynamic between reviewers and authors a little more even. But the collaborative nature of science, and the culture of sharing preliminary results at meetings and with posters, makes true anonymity for authors impossible to achieve. It seems that all variations of anonymity/identification for authors, reviewers and editors has its own problems, and none addresses concerns that peer-review takes too long, impeding careers and the advancement of science as a whole.
The attention of the students then turned to preprint servers and particularly bioRxiv (https://www.biorxiv.org/), which is operated by CSHL. A preprint server provides very fast exposure for a researcher’s work. Studies can be ‘live’ within hours of submission, as only the lightest of editorial filters is needed, but they lack expert review, despite mechanisms in place to allow commenting — at the time of writing, of the 36 studies classified as Plant Biology and posted on bioRxiv in June, only one (a technical note about the use of a diagnostic assay to distinguish isolates of wheat and rice blast fungus particularly from Bangladesh (Gupta, D.R. et al. bioRxiv http://doi.org/csfc; 2018)) has so far received any comments.
This lack of at least some form of review was felt by the CSHL group to be of particular concern to young scientists like themselves. They felt that the imprimatur of peer review was needed to help junior researchers identify competent science. Asking scientists to wait an indeterminate length of time for a community to pass judgement on a paper is an undue burden and gives no guidance to the naïve or inexperienced reader.
Ultimately the group at CSHL concluded that devising new variations on peer review misses the real problem: a small number of bad actors in the system. If there are incidents of discrimination, bullying, cronyism and incompetence within peer review, it is those behaviours and the people who commit them that need tackling. Regardless of its structure, peer review is only as good or bad as its participants.
Peer review should be a collaborative process, concentrating on feedback to improve science as a whole. Reviewing others’ work is not an innate skill but something that needs to be learned. For early career scientists to become the competent and fair reviewers of the future they need to be actively participating in the peer-review process, be explicitly taught analytical writing skills and effectively mentored in how to articulate objective and, above all, helpful criticism.