Peer review seems like one of those scientific practices that is so fundamental to the way we do modern science that it feels like it must have been around for centuries. It makes so much sense that scientific output should be verified by expert colleagues before being passed off as fact that it is hard to conceive of any other way that things could be done. It will probably come as a surprise to many that Nature did not regularly contact external referees to assess its academic submissions until the early 1970s. Many other journals were no different. Editors today rely heavily on the advice of expert scientists to make decisions on which submissions to publish; we at Nature Research recognize that the process needs to evolve to ensure that it does not burden participants unequally and it does not propagate or amplify bias.

At Nature Astronomy, we usually ask our referees to return a report on a submission within two weeks. Some other journals give a shorter window, while others allow slightly longer. Within the review period we expect the referee to dedicate a day or so to the scientific scrutiny of the manuscript, and we may ask them to participate in a further round or two of review, if needed, usually requiring less time than the first. We aim not to overburden our referees — we typically appoint two or three referees to a paper to reduce the workload per referee (and to reduce individual bias) — and try not to employ the same referee more than once in a year. In order to grant greater recognition for this service, over the past 18 months we have been asking our referees if they wish to be thanked by name in the published paper. Almost 40% of referees have accepted.

In academic astronomy and planetary science, the practice of peer review is not limited to the publication of scientific papers. The procedure might vary slightly for different applications, but the concept is the same: grant requests, job applications and telescope or facility time applications almost always pass before a committee or panel that must judge them according to some criteria, selecting those that fit best. Peer review is thus a critical part of the modern scientific process, and given its importance and widespread nature, participation in a panel or committee or indeed as a referee is often seen as a service to the wider community and considered an inherent part of the academic role.

When it comes to applying for telescope time, competition for a fixed amount of observing time can be fiercer than it is for publishing a paper. For example, in the recent Cycle 7 call for Atacama Large Millimeter/submillimeter Array (ALMA) time, 1,773 proposals were submitted for roughly 4,300 hours of available telescope time. The success rate was 22%. The ALMA time allocation committee (TAC) is considerable in size. It is thematically split into five scientific categories, with each category containing a number of panels and ~6 people sitting on each panel on average. In the most recent completed cycle, there were 25 panels involving ~160 people. The different panels read different numbers of proposals. A recent member of one panel read, assessed and issued feedback on 90 proposals. A standard proposal comprises three pages of scientific justification; a large proposal comprises six pages. That is more than 300 pages of scientific argument to parse and respond to within a three-week period. In ALMA Cycle 6, the TAC members were only given two weeks to process a similar volume of proposals, and there were only 18 panels. Reading and commenting on proposals is not the only duty of the TAC: there is usually also a face-to-face meeting, lasting a week or so including travel time, where notes are compared and the final proposal rankings assembled over long working days. Thus a TAC member might easily dedicate four weeks a year to the peer-review process. And members are expected to contribute over multiple years: three years for the ALMA TAC, which is a huge time burden, especially considering additional academic duties such as teaching, marking, and so on.

The current issue of Nature Astronomy contains an Article reporting on a procedure aimed at reducing the incredible burden of peer review for TAC members. The scheme, called distributed peer review, was put into action at the European Southern Observatory (ESO) in 2019 alongside the standard peer-review process. The ‘distributed’ part of the peer review references the fact that rather than a gathered panel of experts assessing proposals, the task of ranking and providing feedback was carried out by the collected group of proposers — the principal investigators (PIs) or their nominees, in fact. This action immediately led to an order of magnitude reduction in workload, from ~70 proposals per person down to eight. Proposals were automatically assigned to reviewers using a text-processing algorithm, thus reducing a source of bias. ALMA will implement a similar distributed peer review system for most proposals in Cycle 8, currently scheduled for late 2021.

Another TAC process undergoing reform is that for time on another highly competed telescope, the Hubble Space Telescope. The three most recent cycles were assessed under dual-anonymous peer review, where committees scrutinize proposals without knowing the identity of the proposers. This led to an evident change in the demographics of successful PIs, not only increasing the fraction of successful women PIs (such that the percentage of successful women PIs came within a point or two of successful male PIs), but also increasing the fraction of first-time PIs, suggesting that not only was there a historical gender bias but also a seniority bias. As a result of this pilot programme, dual-anonymous review will shortly be rolled out to other NASA-operated telescopes, such as Swift, Fermi, NICER and NuSTAR. The ESO trial did not employ this dual-anonymous system, so that conflicts of interest between panel members and proposals could be identified.

Given the critical role of peer review in the professional lives of academics, from the publication of journal articles to the awards of grant money or telescope time, we should be continually striving to ensure that the system works in a fair and unbiased way. Reform should be straightforward, now that we are becoming increasingly aware of where the problems lie. It remains for those of us who manage these systems (journal editors, TAC chairs, and so on) to be proactive, equitable and promote best practice. Let’s keep up the dialogue and continue to review peer review.