Grant reviewers for the US National Institutes of Health (NIH) report shorter attention spans and lower engagement during video grant-review meetings than in those held face-to-face, finds a survey of 3,288 reviewers.
The survey by the NIH’s Center for Scientific Review (CSR) in Bethesda, Maryland, gathered information from reviewers who had participated in Zoom meetings between August and October 2020. Compared with in-person grant-review meetings, 46% of respondents said that they paid less attention during the video meetings, and 51% said that their engagement was worse. “I get tired of looking at all the faces in Zoom meetings, so I’ll look at other things, too,” says survey participant Alexander Dent, an immunologist at Indiana University School of Medicine in Indianapolis. “But I’m certainly listening, and that happens at a normal meeting anyway.”
Other researchers worry that reviewers in video meetings might not discuss or consider grants in the same way as at face-to-face meetings. Jason Moore, a bioinformatician at the University of Pennsylvania in Philadelphia, says that he writes many grants and is concerned about the quality of reviews conducted on Zoom calls. “Is my grant getting a fair discussion and are all voices being heard?” he asks. “Because it’s often the case that a single person who really likes a grant that nobody else likes can be a loud voice in the room and can turn the table around and convince everybody else that the grant really does have merit.”
Some 43% of survey participants said that they preferred face-to-face meetings over those conducted online, whereas almost one-third preferred online platforms. Just 10–15% of reviewers rated Zoom meetings as the better option across all criteria of review quality and reviewer participation, according to the CSR report.
“We are a bit Zoom fatigued,” says Sandra Bendiscioli, senior science-policy officer at the European life-sciences organization EMBO in Heidelberg, Germany.
A spokesperson for the CSR, which reviews more than three-quarters of all NIH grant proposals, declined an interview with Nature on three separate occasions, saying the publicly available report speaks for itself.
Dent, for one, is not worried that Zoom is making it harder for reviewers to fully focus on grant applications. “I don’t think that’s a big issue,” he says. “The overwhelming sentiment was that the review process was still as stringent and as rigorous as normal.”
Sixty per cent of survey participants said that overall, reviews conducted during Zoom meetings were of the same quality as those done in person. Half of participants said discussions were of the same quality. Other studies1,2 have suggested that remote peer review can work well, and that scores decided in online meetings are likely to be similar to those from in-person discussions.
Still, Moore says, it is important for the NIH and other funders to determine whether virtual meetings change how reviewers rate grant applications. “If the scores are fundamentally changing in some way, that would be good to know,” he says.
Some say that virtual meetings are a positive outcome of the pandemic. Online platforms can help to boost the diversity of review panels and widen participation, says Susan Guthrie, associate research-group director at RAND Europe, a non-profit policy consultancy in Cambridge, UK. A study on researcher mobility that she co-authored in 2018 suggests that some academic scientists are excluded from international collaborations, far-flung conferences, or manuscript or grant peer review because they cannot afford to travel or pay for childcare.
Guthrie says that the pandemic has prompted many changes to the scientific enterprise. “In terms of peer review in particular,” she says, “we have seen how funding can be allocated rapidly and repurposed flexibly to address emerging challenges.”
Moore calls for improved technology that will allow researchers to attend conferences and meetings in virtual reality. “It’s closer to a real experience where you’re in a conference room and with other people,” he says. “The technology needs to catch up, and when it does, it will be a better replacement sometimes for face-to-face meetings.”
Nature 590, 172 (2021)