Citation count has become one of the most important methods to evaluate a scientist’s contributions. In an extensive analysis of citations from a number of leading neuroscience journals, Dworkin and colleagues find evidence of gender bias in citation practices that can have an adverse impact on women’s careers.
We would hazard a guess that most, if not all, working scientists have felt disappointment and/or anger when seeing that a recently published paper failed to cite one of their own. Most of us view this as an inescapable consequence of the way science is published; however, a paper by Dworkin et al.1 in this issue of Nature Neuroscience shows that current citation practices in neuroscience particularly disadvantage female scientists, with potentially unfortunate career repercussions.
One hundred years ago, assessment of scholars was done by the communal wisdom, or lack thereof, of an old boys’ network, who among themselves decided the allocation of positions, fellowships, promotions and prizes. That system worked when scholarly fields were small but became less tenable as the academic community grew. Its failures were revealed as we collectively started to understand how both implicit and explicit biases influence who among the many talented scientists and scholars is chosen for a finite pool of opportunities and recognition.
The past 20 years have seen various attempts to create objective indices that capture the influence of an individual’s work on their field. For a variety of historical reasons, many institutions around the world started to use the Impact Factor of the journal in which a paper was published as a proxy for the importance of the paper itself2. This was a solution to the problems faced by departments and institutions without expert knowledge in all areas of science as they attempted to balance the value of the number of published papers against the quality of those papers. Nonetheless, the law of unintended consequences led to the subsequent promotion of the Impact Factor, with scientists chasing journals with higher Impact Factors and young scientists concluding that the name of the journal was more important to their career than the excellence of their papers.
There have been numerous attempts to replace Impact Factor for the evaluation of individual scientists with other citation metrics, most notably the h-index, which was introduced by Hirsch in 2005 (ref. 3), and the i10-index, which was introduced by Google Scholar4 and indicates the number of publications with at least 10 citations. However, all these proposed quantifiers depend on one key metric: the number of citations. The prominence of the h-index and the i10-index puts citation metrics front and center for a quick evaluation of a scholar’s ‘worth’. It is with this context in mind that the study by Dworkin et al.1 in this issue is of great interest.
Focusing on five leading neuroscience journals (Brain, Journal of Neuroscience, Nature Neuroscience, NeuroImage and Neuron), this study set out to evaluate whether neuroscience authors exhibit gender bias in citation practices. To do so, Dworkin et al.1 divided papers into four categories according to the inferred genders of the first and last authors, as estimated by first or given name (a method which as the authors note is sometimes ambiguous, incorrect and/or may not reflect the full spectrum of gender identities). They then compared the numbers of citations in each category relative to the estimated base rates of these authorship combinations in the literature. They found that man first author and man last author (M/M) papers tend to be overcited relative to their base rate, while woman/woman (W/W) papers are undercited. Papers with either a woman first or last author (W/M, M/W) are also undercited, but to a lesser extent.
Is this a gendered behavior? It would appear so. M/M authors tend to overcite M/M papers by about 8% over their base rate, but more dramatically undercite W/W papers by a substantial 20%. M/M authors also undercite M/W and W/M papers, by a more modest 10%. Author combinations involving a woman show very little citation bias on the whole; however, a deeper analysis reveals that a tendency of W/M authors to undercite W/W papers is partially balanced by a similar tendency (~10%) of W/W authors to overcite W/W.
Science is social; one might expect that one tends to cite the people one knows, directly or indirectly, through one’s social network. Further, the effect of ‘homophily’, the tendency to work with collaborators of one’s own gender, could influence individuals’ perceptions of the gender makeup of the field as a whole. The authors looked for the influence of this effect using network modeling of co-authorship groups. They found that these groups tend to have an overrepresentation of men and that this is more pronounced in the M/M networks. Looking specifically at overcitation of men, this is indeed partially accounted for by social network structure, but is more pronounced than can be explained in this way.
One might hope that citation biases have diminished over time as women started to make up a larger fraction of the field. Unfortunately, the opposite is the case. While the fraction of woman-authored papers is indeed increasing—by around 40% over the past ten years in the examined dataset—citation practices have remained relatively stable, meaning that citations have become increasingly unrepresentative of the field rates over time.
In the course of writing review articles, we have occasionally paid attention to patterns in individual authors’ citation behavior, and indeed we have seen egregious cases of gendered citation. We would say, “you know who you are!” But you probably don’t. As with many biased behaviors, it is likely that this behavior is largely unconscious. The first steps toward amelioration of bias are quantification and awareness. The National Science Foundation’s requirement for representative distributions of speakers at conferences it funds and the explicit quantification and publication of speaker lists by gender published on BiasWatchNeuro (https://www.biaswatchneuro.com) have had an influence in shifting those numbers. Similarly, awareness of citation effects and self-reflection can help to move this needle and ensure fairer practices for all.
How is this bias potentially deleterious to our female scientists? One of us (E.M.) is old enough to remember days of meetings with no female speakers and journals that forced female scientists to use their full first names in publication so that they couldn’t hide behind their initials (although men were able to publish with their initials). Judged by the standards of 50 years ago, these practices might not seem so pernicious, but the accrued damage done by small slights is not to be underestimated. Modest differences in numbers of ‘times cited’ per paper can accrue to large differences over years in personal citation counts. These can amount to the difference between coming first and second in a job search or in publishing a paper in a prestigious journal.
External factors can also play an important role in the practice of gender-biased reference lists, such as the restriction on the number of references imposed by many journals. Not only does this discourage the practice of appropriate scholarship, but it creates exactly the environment that encourages authors to include citations to the work of investigators of perceived influence and power, while underciting lesser-known, younger or more diverse individuals.
Along with encouraging authors to make sincere efforts to reflect on and correct one’s own practices, we also strongly suggest that journals change the restrictions on the number of citations that are allowed. By so doing, one can hope to get a fairer and more accurate representation of the contributions of the many neuroscientists who were important in setting the stage for the new studies! And by so doing we will live up to our individual and collective responsibilities to train the next generation of scientists to respect and value the work upon which we depend for future progress.
Dworkin, J. D. et al. Nat. Neurosci. https://doi.org/10.1038/s41593-020-0658-y (2020).
Marder, E., Kettenmann, H. & Grillner, S. Proc. Natl Acad. Sci. USA 107, 21233 (2010).
Hirsch, J. E. Proc. Natl Acad. Sci. USA 102, 16569–16572 (2005).
Connor, J. Google Scholar citations open to all. Google Scholar Blog https://scholar.googleblog.com/2011/11/google-scholar-citations-open-to-all.html (16 November 2011).
The authors declare no competing interests.