With regard to my factual reporting of the events, I have made it a principal not to be guided by my own general impressions, but in reports I have checked with as much thoroughness as possible . . . Even so the truth was not easy to discover . . . different witnesses gave different accounts of the same events speaking out of partiality or else from imperfect memories. Maybe my history is less easy to read . . . it will be enough for me that my words are judged useful ... This work was not designed to meet the taste of an immediate public, but was done to last for ever.

Thucydides

History of the Peloponesian War (4 th century BCE)

Peer review of scientific manuscripts affects all scientific activities. It influences publication and establishes a scientific hierarchy. Work published in higher impact journals shapes current thinking and investigator's preferences for research areas. Publication of poor quality work, or nonpublication of important experiments, hinders the progress of any discipline. Peer review also affects our careers and our capacity to obtain the funding necessary to develop our lines of research. High quality peer review is thus an important responsibility of the scientific community.

Peer review could be the ideal scientific interaction. The system gives us access to our most competent colleagues, whose responsibility it is to read our work in detail and make constructive criticism to improve its quality. However, many of the comments received from referees on our manuscripts do not meet this goal. Who hasn't received the review that is but a hasty glance, with sparse scientific arguments and preconceived ideas reigning larger than established facts?

Immunology may be on the verge of a new era, in which the quality of critique could undergo massive deterioration. The number of scientists and published articles has exploded in the last decade. To cover every published work in our discipline we would have to absorb about 1,000 new articles per year. The complexity of the field has also increased considerably, forcing a multidisciplinary approach to master original research.

It is difficult to integrate such an overwhelming amount of information. Many scientists adapt by subspecializing to retain detailed knowledge. To maintain a global overview it is tempting to rely on superficial or second-hand information. This adaptation is characteristic of the human mind, which is designed to integrate incomplete information. However, this bodes poorly for the review process; the dangers of which to the progression of knowledge were already recognized 2,400 years ago in the first scientific text published (quoted above). The scientific method was developed and constructed precisely to counter these dangers.

The review process may also fail because the attribution of expertise (by the editor or by the reviewer) is not valid. The definition of a reviewer's qualifications is now quite complex: it requires a detailed assessment that takes into account methodological expertise and precise lines of research. Unfortunately, inappropriate reviewers may depend on their “imperfect memories”, forgetting one of the most important goals of a scientist, which is to have a precise idea of what one does not know.

Far more manuscripts than can be published are peer-reviewed, which may diminish the pressure for improving the reviewing procedure. Scientists, whether at the bench or situated at journals, may consider criticism of the review process as a lack of appreciation for their activities, rather than as an impetus for the constructive reshaping of the system in response to the new characteristics of scientific life. They may look at the current process as favoring healthy competition, which leads to the selection of the optimal papers for publication. Many scientists accept this latter view. However, the best research will not be selected without appropriate revision. On superficial examination, simplistic concepts and experiments described as solving major problems will have much higher appeal than experimental rigor and respect for complexity. As pointed out by Thucydides, without rigorous scientific scrutiny, we will select demagogy rather than science.

With respect to scientists' attitudes to reviews, they can border on the neurotic. Colleagues have compared the review process to gladiators in a Roman arena, with the editors playing the part of the Roman Emperor, giving a thumbs-up, or a thumbs-down. Although this is a gross exaggeration, we certainly do not look at the review of our manuscripts as a joyous and productive interaction. Our frustrations stem from the experience that scientific argument (our legitimate weapon) is often of no avail: even if proven totally unjustified, a negative review retains its weight. This has repercussions on our own performance as reviewers. Because our work was sometimes not looked at thoroughly, we may feel justified in not investing the time required for a detailed, objective and constructive review.

The increasing complexity and mass of scientific knowledge has induced strains that will only worsen with time. The important question is what can be done about it. I have listed some suggestions below. Personally, I hope this letter will open up a general discussion on how to improve the system we created, which is, after all, our sole responsibility.

Some thoughts for improving peer review: (i) There is no better alternative to peer review for evaluating scientific manuscripts, but the review process must be improved. (ii) As the review process is not perfect, a scientist's right to defend their work must be acknowledged and a format to handle the differences between reviewers and authors should be introduced. (iii) A major problem is the lack of accountability. Reviewers do not benefit from conscientiousness and risk little from shoddiness. Our work as reviewers should have a much higher impact in our careers. Conversely, reviewers could be rated and performances scored by editors and colleagues to prevent total impunity. (iv) Reviews should be logical, objective and constructive. All criticisms must be explicitly justified. “General impressions” and vague statements are not acceptable criteria for evaluation of a manuscript. (v) The pool of reviewers should be expanded. Some respected colleagues with lengthy experience and recognized expertise are not frequently solicited as reviewers and yet could do a very good job. The restricted pool of reviewers overloads those reviewers (frequently the most famous) and may affect the quality of their work. (vi) Improved reviewer choice. Editors may not have enough information to make the best possible selection among potential reviewers. The usual procedure for selection (which relies on a broad definition of the areas of interest or expertise) requires rigorous qualification. (vii) Opportunities for dialog and interactions between editors and reviewers should be increased.

See More about peer review: is it time for double-blind reviews? by John R. Klein