Nature | Comment

Journals invite too few women to referee

Corrected:

Jory Lerback and Brooks Hanson present an analysis that reveals evidence of gender bias in peer review for scholarly publications.

Article tools

Subject terms:

Kelsey Vance

US geoscientists on an expedition in Wyoming as part of the Bearded Lady Project, which raises awareness about inequality in science.

Bias — explicit and implicit — is an important cause of the under-representation of women and minorities in science, technology, engineering and mathematics (STEM)1. Women and minorities are disadvantaged in hiring or promotion decisions, awarding of grants, invitations to conferences, nominations for awards, and forming professional collaborations2. These scholarly activities are crucial for career advancement and job retention.

Another career-building activity is serving as a peer reviewer for publications. This develops writing skills and expertise through exposure to other manuscripts, and fosters relationships with fellow scholars and scientific leaders. Such activities are especially important for young scientists.

But most publishers do not collect gender, age or any other relevant demographic information from authors or reviewers. So biases here have been harder to pin down. Most studies of gender inequality in publishing have assigned gender to authors3, 4 but have lacked information on age. This is important because many fields have only recently seen increases in participation of women.

Here we present evidence that women of all ages have fewer opportunities to take part in peer review. Using a large data set that includes the genders and ages of authors and reviewers from 2012 to 2015 for the journals of the American Geophysical Union (AGU), we show that women were used less as reviewers than expected (on the basis of their proportion of membership of the society and as published authors in AGU journals). The bias is a result of authors and editors, especially male ones, suggesting women as reviewers less often, and a slightly higher decline rate among women in each age group when asked.

These findings underline the need for efforts to increase female scientists' engagement in manuscript reviewing to help in the advancement and retention of women in science.

The data set

With 20 journals and nearly 6,000 papers published per year, the AGU is the largest society publisher of Earth and space science. Like many other publishers, it does not ask authors and reviewers to report their age or gender. But the AGU is the largest Earth and space society in the world and it has asked its members to self-report this information since 2013. Overall, 79% of members from 2013 to 2015 (59,316 people) reported both. We call this the AGU membership data set.

We have gender and age data on a further 38,115 individuals who have AGU accounts and who engaged in AGU activities after 2011 (see Supplementary Information). We refer to this group combined with the membership data set as the AGU ‘all accounts’ data set. Checks on random samples indicate that the gender distribution of members choosing not to disclose this information is similar to that of members who did, and that the gender and age information were reported accurately.

“Reviewing history is important in recognition in societies or appointments as editors.”

To reveal gender and age data on authors and reviewers between 2012 and 2015, we merged data on their activities (the ‘editorial’ data set) with the all-accounts data set using unique e-mail information. This merged data set represents the ages (at time of activity) and genders of 106,305 authors across all submitted manuscripts, 24,368 distinct authors, 62,552 reviews and 14,919 distinct reviewers, as well as 97,083 reviewer suggestions by authors and 118,873 reviewer requests by editors. The data set represents 35% of all distinct authors, 57% of distinct reviewers, 50% of all authors and 69% of all reviews completed.

These data represent a significant reference sample of engaged, practising and aspiring Earth and space scientists, even though they do not capture all author and reviewer interactions. Membership of the AGU or an affiliated society is required to submit an abstract to the annual meeting, one of the largest scientific conferences worldwide (22,000 presentations and 25,000 attendees in 2015), and all attendees must create an AGU account. AGU membership is dominated by academic and government scientists (77%, with just 5% in industry) and by scientists in the United States (59%), Japan (5%), the United Kingdom (4%) and the rest of Europe (15%). Proportions are similar for the all-accounts data set.

In short, the AGU makes a good proxy for STEM demographics in the United States. From 2013 to 2015, 28% of members, on average, were women. This is close to the ratio of scientists and engineers employed in the United States (29% in 2013)5 and US-employed Earth and space scientists (25% in 2014)6. The gender ratio of AGU membership decreases with age. The older cohorts have fewer women, reflecting historically higher barriers to inclusion of women and the greater rate of attrition of women from Earth and space science careers (known as the leaky pipeline)2. The youngest age group in 2015 (20s), by contrast, is 45% female. We have not extrapolated our analysis to the complete editorial data set because we have no age data for these additional participants (even if we could assign gender).

Submissions and Publications

We matched the gender and age of 7,196 distinct first authors who submitted 22,067 manuscripts in 2012–15. In the geosciences, first-authored papers are particularly important to career development, hiring and promotion decisions. In most cases, the first author is also the corresponding author (89% in our data set). Women made up 26% of submitting first authors (they accounted for 23% of all submitting authors). Overall, women participated less as authors relative to the make-up of the membership and all-accounts data sets in aggregate and across all age cohorts. Female first authors (n = 2,859) on average also submitted 0.79 fewer papers per person in the four-year period than male first authors (n = 8,098).

The acceptance rate for female first-authored papers was higher than for male ones (61% compared with 57%). Negligible differences have been found by other studies in biology and medicine7. In our data, female first authors had higher acceptance rates across all age groups, and the differences were significant at P < 0.05 for all age cohorts under 60. This difference in acceptance rates pertained (although not always with significance) across papers with any size of author group, including single-author papers. Acceptance rates were similar regardless of the gender of the editor or reviewers.

One might interpret the difference in acceptance rates in two ways. It could imply that female AGU authors are enjoying ‘reverse sex discrimination’, as suggested by recent studies (see, for example, ref. 8). Or — and this is the possibility we favour — the higher acceptance rates reflect the authors’ more considered approach when submitting manuscripts, including better targeting of papers to a journal. Several studies have shown that groups who are expecting obstacles put greater effort into preparation or take fewer risks9. And our data show that women submitted fewer papers than men. A double-blinded reviewing process could shed more light on these factors.

Peer review

The gender ratio of reviewers should be similar to that of published authors of similar age and to the AGU membership. But women constituted 20% of all of the individual reviewers in 2012–15, substantially less than the proportion of distinct published female first authors (27%), published authors (23%) and AGU members (28%) or all-accounts holders (29%; see ‘Unequal opportunities’).

Source: Original analysis by J.L. and B.H.

These lower proportions of female reviewers were evident across most ages and were statistically significant in the 50s cohort. Thus, the lower participation by women as reviewers is not just because editors were seeking more-senior reviewers, who are predominantly male. Furthermore, men (n = 11,876) did more reviews per person (2.1 reviews) than did women (n = 3,043; 1.8 reviews).

There are two possible reasons for women’s lower participation rates. First, authors and editors (or both) might have nominated fewer women in each age group to review (at the AGU, authors suggest reviewers at submission, and editors prepare a final list). Second, women might have declined invitations more often than men.

We found that the first reason contributed more to the discrepancy. Female first authors suggested female reviewers 21% of the time (n = 13,279); men suggested women just 15% of the time (n = 47,236). Unsurprisingly, whatever their gender, authors suggested reviewers in their 20s only 1% of the time.

AGU editors invited just under one-third (29%) of the author-recommended reviewers. Female editors recommended female reviewers 22% of the time versus 17% for male editors. Editors of both sexes were slightly more likely to invite the youngest reviewers (3% of all recommendations). Of these young invitees, only 32% asked by female editors and 31% asked by male editors were women. This is lower than for the AGU membership or distinct published first authors. Recall: these differences did not translate into any significant differences in the decisions on manuscripts. Why authors and editors, especially men, don’t suggest women to review may reflect a number of factors, but it is often attributed at least in part to implicit bias1, 10.

Women of almost every age declined their (fewer) invitations at a slightly higher rate than men in the same age group (except those over the age of 80). The differences were statistically significant for all ages except the 60s and 80s cohorts. For example, the decline rate for women was 22%, versus 17% for men aged 20–30. Decline rates for both genders increased with age, although older cohorts included fewer women and proportionally more older men were asked to review. Therefore, the overall decline rate did not differ significantly. Thus, we do not believe that this results in a negative feedback loop, in which editors don’t invite female reviewers because of they expect them to say ‘no’.

The main reason scientists gave for refusing was their workload. The differences apparent in certain age groups could arise from the fact that women are more likely to be involved in other service activities at their institutions and to shoulder a greater burden of family and other social duties. The differences may also reflect varying approaches to risk or confidence2.

What next

The AGU is similar to many other large scientific societies. Thus we suggest that these results are broadly representative of gender and age differences across scholarly publishing, as well as grant evaluation. Other societies, funders and publishers should perform similar audits.

Participation as a reviewer for papers and grants has many benefits, particularly for early-career scientists. It is a chance to develop a relationship and make a positive impression with an editor, review-panel member or programme manager, who are typically senior scientists and are in turn likely to be involved in evaluating the reviewer’s future papers and grants. The same relationships can be built with authors if the review is not anonymous (many at AGU are not) or the reviewer identifies themselves later to the authors. Reviews can also foster later collaborations (something not possible with double-blind reviews).

Reviewing also offers an important learning experience and a view of unpublished science, and helps improve communication and thinking. It provides positive feedback that a scholar is respected and participating in their field and fosters self-confidence, all of which lead to increased retention of women in geoscience2. Increasingly, there is a call to appreciate the time and effort that goes into reviewing. Hence reviewing history is important in recognition in societies or appointments as editors.

Hiring more female editors has helped the AGU to mitigate the disparity in recommendations. Publishers should consider further ways to train and educate their staff in combating bias. Encouraging authors and editors to invite more women to review, especially younger women, would be a start11. Funding agencies should similarly examine their practices. We encourage organizations and individuals to consider how the structures that propagate gender bias might affect other under-represented groups.

Journal name:
Nature
Volume:
541,
Pages:
455–457
Date published:
()
DOI:
doi:10.1038/541455a

Corrections

Corrected:

An earlier version of the graphic in this Comment mislabelled the bar for the acceptance rate for male first authors in their 70s.

References

  1. Kaatz, A., Gutierrez, B. & Carnes, M. Trends Pharmacol. Sci. 35, 371373 (2014).

  2. Holmes, M. A., O'Connell, S. & Dutt, K. Women in the Geosciences: Practical, Positive Practices toward Parity (Wiley, 2015).

  3. Filardo, G. et al. Br. Med. J. 352, i847 (2016).

  4. Buckley, H. L., Sciligo, A. R., Adair, K. L., Case, B. S. & Monks, J. M. NZ J. Ecol. 38, 335339 (2014).

  5. US National Science Foundation. Women, Minorities, and Persons with Disabilities in Science and Engineering (NCSES Scientists and Engineers Statistical Data System Publication, 2015); available at http://go.nature.com/2ie6agn

  6. US Bureau of Labor Statistics. Women in the Laborforce: A Databook Report 1059 (2015).

  7. Primack, R. B., Ellwood, E., Miller-Rushing, A. J., Marrs, R. & Mulligan, A. Biol. Conserv. 142, 24152418 (2009).

  8. Williams, W. M. & Ceci. S. J. Proc. Natl Acad. Sci. USA 112, 53605365 (2015).

  9. Freeman, R. B. & Huang, W. J. Labor Econ. 33, S289S318 (2015).

  10. Dutt, K., Pfaff, D. L., Bernstein, A. F., Dillard, J. F. & Block, C. J. Nature Geosci. 9, 805808 (2016).

  11. Fox, C. W., Burns, C. S. & Meyer, J. A. Funct. Ecol. 30, 140153 (2016).

Author information

Affiliations

  1. Jory Lerback is a graduate student at the University of Utah, Salt Lake City, Utah, USA, and was previously a data analyst at the American Geophysical Union.

  2. Brooks Hanson is director of publications at the American Geophysical Union, Washington DC, USA.

Author details

Supplementary information

For the best commenting experience, please login or register as a user and agree to our Community Guidelines. You will be re-directed back to this page where you will see comments updating in real-time and have the ability to recommend comments to other users.

Comments

Commenting is currently unavailable.

sign up to Nature briefing

What matters in science — and why — free in your inbox every weekday.

Sign up

Listen

new-pod-red

Nature Podcast

Our award-winning show features highlights from the week's edition of Nature, interviews with the people behind the science, and in-depth commentary and analysis from journalists around the world.