Quality and equality in review

    Scrutiny from every angle, by a diverse set of reviewers, improves the peer review process and the papers that we publish.

    Peer review is a relatively modern addition to the scientific process. Go back 45 years and the systematic assignment of reviewers had only just begun at Nature1. Now, journals are experimenting with options from single- and double-blind peer review (both available at Nature Geoscience2), to open and post-publication review. Whatever the system, passing manuscripts by experts for comment and response has become a cornerstone of scientific publishing. This Peer Review Week (16–20 September 2019) focusses on ‘Quality in Peer Review’, a topic close to our hearts and something we continuously aim to improve.

    Good reviewer reports discuss the extent to which a manuscript is based on robust methods, data and reasoning. Peer review can verify that a paper meets community standards and provides editors with the information necessary to decide if the conclusions advance the geosciences sufficiently to interest our broad audience. We hope that our reviewers will point out flawed logic and unfounded assumptions, and highlight areas that require increased clarity. Some go further, and tease out implications the authors themselves had not noted. Evaluation by experts (and careful revision by authors) can transform a good paper to a great paper — or it can send the authors back to the drawing board.

    Despite ambitious aspirations, there are limits to what peer review can achieve. Contrary to what some may think, reviewers cannot be expected to replicate research or investigate the authenticity of data and materials. Therefore, peer review cannot confirm a study as ‘correct’. Of course, reviewers can, and should, highlight figures or data that seem suspicious. But they must take on good faith what the authors say they have done and found. As replication is the only way to confirm scientific results as correct, we mandate data and code availability statements3, and provide more space for detailed Methods to improve the reproducibility of our papers.

    For rigorous assessment, we need a review panel with the technical expertise to assess every aspect of the paper4. However, in order to avoid the pitfalls of groupthink and confirmation bias, we also prioritize the inclusion of a range of perspectives. In 2017, we called for our authors to better represent their research community when suggesting peer reviewers5. We challenged ourselves to do the same.

    Two years ago, we compared the diversity of our authors on submitted manuscripts with the experts that authors suggested as reviewers, and with the reviewers we actually assigned. In order to chart our progress, we have taken another data snapshot for 2019 (Fig. 1). Here, we compare author and reviewer data for papers that were sent out to review between 1 November 2018 and 1 April 2019, with those sent out between 13 January and 12 May 2017. In 2019, women made up 25% of reviewers at Nature Geoscience, compared with 23% in 2017. At the same time, 20% of reviewer suggestions from authors were for women (compared with 17% in 2017). Despite this improved representation of women in review, 48% of manuscripts still had an all-male review panel (compared with 53% in 2017). See the Supplementary Information for more details.

    Fig. 1: Geographic and gender distributions of Nature Geoscience authors and reviewers.
    figure1

    A comparison between the authors and reviewers of manuscripts sent out to review at Nature Geoscience in 2017 (13 January to 12 May) and 2019 (1 November 2018 to 1 May 2019). Author data are for corresponding authors and can be compared with the corresponding authors on all submitted manuscripts (for 2017 data only, plotted as the innermost pie-chart).

    Geographic inclusion may be even more important than gender when it comes to critical assessment in the geosciences. In 2017, the geographic data were conclusive: we were overly reliant on reviewers based in North America at the expense of those in Asia5. Our efforts to include more reviewers based in Asia are paying off; the share of reviewers in Asia has risen from 4% in 2017 to 10% in 2019. At the same time, suggestions for reviewers based in Asia have remained unchanged at 4%. Corresponding authors based in Asia, perhaps one benchmark for a fair proportion, made up 12% of the manuscripts we sent out to review in the 2019 period (16% in 2017), so there is more to be done.

    Every year, well over 1,000 individuals serve as reviewers for Nature Geoscience, and we are grateful for the time and knowledge they contribute. As editors, we are well aware that invitations to review can be both a burden and an opportunity. Sharing the responsibility of peer review across the research community is the right thing to do not only in pursuit of equality but also because a variety of viewpoints improves the quality of our papers.

    References

    1. 1.

      Csiszar, A. Nat. News 532, 306–308 (2016).

      Article  Google Scholar 

    2. 2.

      Nat. Geosci. 6, 413 (2013).

    3. 3.

      Nat. Geosci. 7, 777 (2014).

    4. 4.

      Nat. Geosci. 9, 261 (2016).

    5. 5.

      Nat. Geosci. 10, 615 (2017).

    Download references

    Supplementary information

    Supplementary Information

    Supplementary data tables for author and reviewer gender and geography.

    Rights and permissions

    Reprints and Permissions

    About this article

    Verify currency and authenticity via CrossMark

    Cite this article

    Quality and equality in review. Nat. Geosci. 12, 681 (2019). https://doi.org/10.1038/s41561-019-0449-9

    Download citation

    Further reading

    Search

    Nature Briefing

    Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

    Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing