NASA has changed the way that reviewers evaluate requests for viewing time on the agency’s space telescopes in an effort to reduce gender and other biases. It now uses a ‘double-blind’ system, in which neither the proposer nor the reviewer knows who the other is.
The space agency made the announcement, effective immediately, in an e-mail to astronomers last month. The policy applies to all upcoming proposals to use telescopes including the Chandra X-ray Observatory, the Transiting Exoplanet Survey Satellite, the NuSTAR (Nuclear Spectroscopic Telescope Array) and NICER (Neutron Star Interior Composition Explorer) X-ray telescopes, the Swift Observatory and the Fermi γ-ray telescope.
The switch to double-blind review, which NASA calls dual-anonymous review, is designed to eliminate a host of systemic biases that have plagued the agency’s review of research proposals. In addition to discrimination on the basis of gender, they include ethnic bias and discrimination against scientists who are at small research institutions, who don’t hold prestigious positions or who haven’t received NASA grants before.
“You can never completely eliminate unconscious biases, but you can greatly reduce them,” says Michael New, deputy associate administrator for research in NASA’s science division in Washington DC. “Dual-anonymous reviewing swats a lot of flies with one hammer.”
The new policy will affect roughly 650 researchers working on 2,300 proposals for NASA telescope time. The agency distributes a total of US$55 million to each batch of observation projects that reviewers approve.
“It’s a great idea and a move in the right direction,” says Feryal Özel, an astronomer at the University of Arizona in Tucson who chairs an advisory committee for NASA’s astrophysics division. She was not involved in developing the agency’s double-blind policy, but has applied to use NASA telescopes in the past. “A lot of us serve on panel reviews and we do our absolute best to put aside conscious and unconscious bias,” she says. “But study after study has shown that it exists.”
Some journals, including Nature, offer authors a double-blind review option, but using such a system to allocate scientific resources is rare. (Nature’s news team is editorially independent of its journal team.)
NASA’s decision to use double-blind review for telescope time requests follows research1 showing that it dramatically reduced gender bias in the approval of Hubble Space Telescope proposals. A 2014 study2 of the projects granted time to use Hubble found that between 2001 and 2012, projects led by men had an average success rate of 24%. The average success rate for proposals led by women was 18%.
In 2017 the Space Telescope Science Institute in Baltimore, Maryland — which manages Hubble — invited two researchers who study bias in science to observe the peer-review process for telescope time, where they found gender bias during discussions of the proposals.
The institute adopted a double-blind review system in 2018 in response to those findings. As part of the new process, neutral observers joined the reviewers who evaluated proposals to keep the panelists focused on discussing the science, rather than the potential identity of the scientists, behind each project. In the first round of reviews that used this system, the success rate for proposals led by women was 8.7%, while the success rate for men was 8%.
The key to reducing bias is to keep reviewers focused on the written criteria for evaluating a proposal, not the personalities involved, says New.
An illuminating experience
“For me, the experience was eye-opening,” says Priyamvada Natarajan, an astronomer at Yale University in New Haven, Connecticut, who chaired the committee that allocated Hubble time last year. “It really steered us back to what a review is supposed to be — not a review of the PI, but a review of the science. It was a complete reshaping of the discourse, which was fantastic.”
NASA is considering expanding dual-anonymous review to other research programmes that it funds, although it expects some pushback from researchers.
“Not everybody will love this,” says Thomas Zurbuchen, who heads the agency’s science division in Washington DC. “But it’s the right thing to do.”
Nature 571, 156 (2019)