Main

In high-level scientific journals, manuscripts are published usually after a careful assessment of their quality and suitability for the journal through a system known as peer-review. Although some controversy exists about the rationale of the peer-review process,1 it is the most commonly used method of selecting scientific manuscripts for publication. Given the importance of this issue for the advancement of science, editors of eminent medical journals have supported the notion of an international congress where various topics related to improving the peer-review process can be discussed.2,3 This opinion article discusses real-life information about the peer-review process from the perspective of an author and reviewer for scientific journals, with the aim of improving the peer-review process.

The peer review process

Although peer-review systems may differ slightly across different journals, they all possess a similar structure. Initially, the paper is submitted to a journal's editor-in-chief (EIC), or sometimes an associate editor (AE), for initial assessment of whether the paper should be forwarded for further review, as described below, or rejected immediately (the so-called desk rejection) (Fig. 1).

Figure 1
figure 1

Scheme of the usual peer review process (EIC: Editor-in-Chief; AE: Associate Editor)

Communication between reviewers

Editors normally invite more than one reviewer to evaluate a scientific manuscript. Accordingly, editors may receive more unbiased opinions to inform their decision of whether to accept or reject a paper. Generally, invited reviewers do not know how many peers are involved in the review process and there is no interaction between reviewers. This scenario sometimes leads to reviewers providing diametrically opposed opinions on a manuscript. These divergent opinions create difficulty both for the authors, who need to address sometimes incompatible, opposite reviews; and for the editors, who must decide the fate of the manuscript. One possible solution would be to make reviewers' comments available to all reviewers. In this way, reviewers would be able to discuss the heterogeneities in their evaluations, helping to clarify whether an update of the manuscript by the authors is, in fact, a reasonable task. Moreover, points of strong disagreement could be resolved by consensus among reviewers before the comments are sent back to the authors. A potential disadvantage of this solution would be a longer peer-review process.

Peer-review should be focused on the original idea only

Sometimes reviewers recommend that the authors update a manuscript in the way they 'would do' the manuscript, instead of analysing the quality of the manuscript per se. For example, one of the options for the EIC's decision in some scientific journals is a 'reject and resubmit' option. This option means that the manuscript was rejected because of pivotal flaws, but a new version, usually totally changed (sometimes including new eligibility criteria), would be considered for further review. Moreover, these changes sometimes require further analysis and even the incorporation of new data to please the reviewers/editors. In other words, 'reject and resubmit' means, in many cases, a largely new project. One may ask whether a new project induced by reviewers or editors creates a great source of bias in the future manuscript. Furthermore, one could argue that reviewers who suggest pivotal changes to a study should be acknowledged in the updated manuscript. To avoid this conundrum, editors can recommend that reviewers focus on the quality and relevance of the original project, and not be allowed to request the change of pivotal aspects of the project, such as eligibility criteria. This limitation would prevent the review process from changing the core of the work, and reviewer bias would be reduced. Alternatively, it may be prudent for editors to eliminate the 'reject and resubmit' option from the peer-review process altogether.

No chance for dispute

Journals receive a great many submissions each day or month and it is likely that some manuscripts of good quality will be not evaluated in depth. Some scientific journals do not allow any chance for dispute with the EIC's initial decision being final. The refusal of any chance for dispute may contribute to important research being overlooked.4 Hence, a dispute procedure should be a standard option for any journal. Authors should have the opportunity to explain in detail why they think their manuscript would be suitable for publication in the journal. Obviously the chance for dispute does not give certainty of acceptance, but in some cases the true value of the manuscript would be recognised and the material published, reducing reporting bias.5

Suggesting reviewers

During the online submission process, many journals ask or even require the nomination of preferred and non-preferred reviewers. This procedure is very likely to add bias to the review process. If authors suggest preferred reviewers, they are apt to recommend reviewers they know or reviewers who share their way of thinking. At the very least, they will try to suggest reviewers who may provide positive comments about their manuscript. In contrast, authors will suggest non-preferred reviewers who have some kind of conflict of interest, or reviewers whom authors anticipate will provide potentially negative comments. Journal editors indicate that they are not obligated to accept author recommendations regarding reviewers. Nevertheless, this potential bias in reviewer selection could be removed by journal editors selecting reviewers without any recommendations from authors.

Paper acceptance

Often, it is difficult for authors to understand how a manuscript was accepted or rejected based on reviewers' comments. Sometimes the EIC ignores the recommendations of the reviewers completely and makes a decision that contradicts reviewers' suggestions, for example, in the situation where two reviewers provide positive reviews on the manuscript. In other cases the opinions of reviewers may be contradictory and, therefore, a third reviewer is contacted to resolve the dispute. Nevertheless, an EIC may act as the third reviewer and make the final decision by taking into account his/her personal view on the manuscript. One may argue whether some bias is introduced when the EIC interferes in the 'field' of reviewers by using his/her own review to make the decision. Anyway, if this policy is used by an AE or EIC, it should be clearly stated in the journal´s information to authors and readers.

Single-blind or double-blind peer-review process?

A large study evaluated the opinions of more than 4,000 reviewers on the peer-review process.6 Among other questions, the reviewers were asked about advantages and disadvantages of blinding or double-blinding the peer-review process. Some reviewers answered that, in the single-blind review (when reviewers know the identity of authors, but authors do not know the identity of reviewers), more prestigious group researchers may be favoured in the peer-review process. Nevertheless, some authors argue that there is in fact no true double-blind peer-review because reviewers can 'guess' the identity of the authors by checking the writing style, self-citations etc. I believe that the double-blind approach is the most ethical approach, because the policy of blinding is fair for both sides. One may argue that the identity of the authors will not be recognised in a percentage of submissions. Thus, the evaluation will be less biased, at least when compared to the single-blind review where the identity of the authors is always recognised. Thus, for these cases, where reviewers recognise the identity of authors in the double-blind peer-review process, editors should recommend reviewers to inform them about any potential conflict of interest (COI).

To Open or not to Open the peer-review process?

Making the whole peer-review process public may bring advantages and disadvantages. For example, some junior reviewers could be inhibited to perform a detailed and judicious review to not challenge senior researchers due to their own COIs.6 On the other hand, some evidence suggests that the quality of the review might not be affected by making the signed review public.7 The logistics, however, would be more complicated because an open peer-review process might be associated with a low acceptance rate of potential reviewers. Furthermore, those reviewers who accept the task may take longer to complete their reviews.7 Finally, some reviewers may feel uncomfortable to expose some personal limitations such as poor written English. Therefore, editors should try to identify potential reasons for the low rate of acceptance of reviewers (in the open peer-review process) to provide better assistance and support to reviewers. For instance, in this specific situation, the journal could provide English professional editing for the review to be published along with the paper. In an ethical perspective, an open peer-review would be the best alternative for a more transparent peer-review process.

Conclusions

This opinion article describes some common situations that arise in the peer-review process that have the potential to interfere with peer-review quality. Many of these situations are related to potential conflicts of interest of the parties involved in the process. Some suggestions for dealing with these situations are described.