Can giant surveys of scientists fight misinformation on COVID, climate change and more?

In the next few weeks, some 20,000 researchers will receive an e-mail asking them if COVID-19 is caused by a virus. It might seem a futile exercise, given that the answer is obvious. But that’s not the point.

The e-mails are a test for a project by Peter Vickers, a philosopher of science at the University of Durham, UK, to assess scientific consensus. He wants to set up a tool that widely probes degrees of scientific agreement on issues in public health, climate change and other topics in the public eye to help support evidence-based policymaking.

He is not alone. Rocked by the hugely visible impact of misinformation during the COVID-19 pandemic, several researchers are launching efforts to survey scientists’ thinking on issues from vaccine safety to climate change.

Whereas Vickers favours giant, general surveys, other projects want to focus on smaller groups of specialists, and even to have them talk to each other to establish why they agree or disagree. But all of these projects aim to make scientific debate, arguments and degrees of consensus both more visible and transparent — which, their creators argue, will benefit public conversation and policymaking. Observers, however, caution that the projects might merely further politicize public debate.

“I think the world just needs a dedicated institute that actually measures scientific consensus across the world in a really serious way,” Vickers says.

The power of big numbers

Vickers became interested in consensus after writing a book on what science can confidently say is true — a state of knowledge he labels as future-proof (Identifying Future-Proof Science, 2022).

Frustrated by the gap between what scientists know about anthropogenic climate change and public surveys that suggest many people remain unconvinced, he wants to create a university institute to establish where scientists stand on this and other issues.

Vickers’s idea is to use surveys to assess agreement among tens of thousands of researchers in many scientific disciplines: physics, biology, chemistry, Earth sciences and health sciences. This vision differs from other kinds of consensus report, such as those from the Intergovernmental Panel on Climate Change, which don’t use surveys to query whether scientists agree with specific statements.

Although other studies have used surveys to judge consensus, Vickers says those efforts lacked global scale. Asking a select number of experts — a few hundred, say — leaves the conclusion open to criticisms of cherry-picking opinions, he argues. And sceptics might find it easier to claim that their minority positions counterbalance consensus views if, he says, the full extent of scientific agreement hasn’t been made visible.

“There’ll be some people who say they don’t care if a physicist believes that COVID is caused by a virus,” he says. “But that’s how you get big numbers of scientists, and you’ll be able to have this big statistic which may have influence over some people.”

Vickers has set up his system by signing up coordinators at 31 institutions across 12 countries, who will distribute e-mailed questions to scientists in many disciplines. Each will be presented with statements and asked simply whether they strongly agree, agree, disagree, strongly disagree or are neutral. It should take researchers only a minute to read and respond to the e-mail, Vickers says.

The results will be anonymous, but tagged to show institution and scientific area. Exactly how they will be tallied and reported is yet to be decided. The project was set up with £20,000 (US$25,200) from his university, and Vickers hopes to use the pilot results to win a much larger grant to establish a full-time research unit.

Top climate scientists are sceptical that nations will rein in global warming

Consensus, of course, says nothing about truth — just that there is broad agreement on something. And there is no consensus about how broad that needs to be. “Probably above 75%, or above 80% I would call consensus. That’s a wide-open question. But I think when you get into the nineties, it’s all consensus,” says Peter Doran, a geophysicist at Louisiana State University in Baton Rouge who has published studies demonstrating the levels of scientific consensus among climate-change experts (K. F. Myers et al. Environ. Res. Lett. 16, 10430; 2021).

Vickers has set his own bar for declaring consensus at 95%, something he argues counts as beyond reasonable doubt. “You want to set a bar that’s so high that there isn’t a single counter-example from history. You’ve never reached that bar before and then been wrong,” he says. It’s a provocative statement, but Vickers argues his book research shows that documented near-unanimous levels of scientific agreement — at least relating to issues based on data from experiments — are relatively rare throughout scientific history, and, when reached, have not yet been proved wrong.

‘Stick to the science’: Nature’s podcast series on science and politics

“The trajectory I’m talking about is where somebody has a hypothesis and it turns into a theory and then, eventually, people start calling it a fact,” he says.

However, the 20,000 scientists he has signed up so far are predominantly based at Western institutions — a bias he wants to address if the idea catches on. Groupthink can be a powerful corrupting force, Vickers acknowledges, as can prejudice. He brings up the example of continental drift. In the mid-twentieth century, many US and UK geologists insisted that drift was impossible, but many others, including those in South Africa and India, disagreed, Vickers notes. “There was a prejudice there.”

Helping or hurting?

Vickers argues that showing how scientists agree should help to fight disinformation and support evidence-based policymaking. But James Evans, a sociologist at the University of Chicago in Illinois, warns that efforts to deliberately find and promote scientific consensus in this way could be counterproductive. “It feels manipulative. It doesn’t feel scientific,” he says. “If we push too hard the narrative that you don’t believe this, but all scientists do, it could have the reverse effect.”

Scientists who are not experts in the field under discussion are likely to respond on the basis of their personal values, he says, which could alienate rather than engage many members of the public. “I feel like it’s a weaponization of scientific voices,” Evans says.

In response, Vickers says Evans’s sociological critique doesn’t concern him. From his point of view, he’s just working to correct misconceptions. He argues that scientists have expertise in judging scientific statements, including those outside their field. “They often have to use second-order evidence as a proxy for first-order evidence,” he says.

And he points to a study on vaccine take-up in the Czech Republic as an example of how consensus can help (V. Bartoš et al. Nature 606, 542–549; 2022). In early 2021, reports suggested that people thought that only half the country’s doctors believed that available COVID-19 vaccines were safe. But a team of economists led by Vojtěch Bartoš at the University of Munich, Germany, surveyed 9,650 doctors in the Czech Republic and found that was a massive underestimate. In fact, 90% of the doctors trusted the vaccines.

Are repeat COVID infections dangerous? What the science says

When the researchers asked a sample of around 2,000 people in the Czech Republic whether they intended to get vaccinated, they found that mentioning the consensus among doctors about vaccine safety significantly raised the number of people who said they would. Intended uptake rose by about four percentage points compared with a control group that was not told about the doctors’ views. And those intentions carried through into a similar difference for actual vaccination between the two groups, as measured some nine months after vaccines became widely available.

“It really seems that these people took this information seriously, even though we just provided it once,” Bartoš says.

The four-point difference is more important than it might sound. By the end of the study, around 25% of the control group had not been vaccinated. “If we think about it as a reduction of the group of people who do not get vaccinated, then we have actually reduced this group by about a fifth,” Bartoš adds.

Haixin Dang, a philosopher at the University of Nebraska Omaha who studies how researchers work together to generate scientific knowledge, says Vickers’s project could make a positive contribution.

One issue with asking scientists to establish levels of consensus, she cautions, is that questions that the most people are able to agree on will be the least interesting. “The consensus view has to be bland. Because the more specific it is, the more idiosyncratic a statement becomes, then the fewer and fewer people will be able to agree,” she says. “Often, consensus hides the diversity of views underneath the consensus.”

Countering misinformation

Two other projects that kicked off this year have related ambitions to Vickers’s, but with contrasting methods and goals. One, a non-profit organization called the Clarity Foundation, is based in British Columbia, Canada. The other, a website called comCensus, is the work of Daniele Fanelli, a metascientist at Heriot-Watt University in Edinburgh, UK. Both projects are interested in establishing degrees of consensus, but will survey the views of much smaller groups of experts, at least at first.

David Sweet, a critical-care doctor at Vancouver General Hospital in Canada, co-founded the Clarity Foundation after witnessing the impact of misinformation on people’s attitudes to treatments and vaccine safety at first hand during the pandemic. “I’d have patients in the ICU dying and then learning that their family members had convinced them not to get vaccinated,” he says. “And the reason they had done that is because of the misinformation that was being spread on social media and the Internet. And I saw that repeatedly and it was heartbreaking.”

His experiences suggested to him that much of the public has lost trust in institutions. The Clarity Foundation intends to work on ways to fix the situation.

“We need to blast society with the good, the same way that the misinformation is being blasted from the other side,” he says. The group’s co-founders include entrepreneur Thomas Lamb (chief executive of the mining firm Myriad Uranium in Vancouver), infectious-diseases specialist Jolanta Piszczek at the University of British Columbia in Vancouver and neuroscientist Jay Van Bavel at New York University.

Working on the principle that people still trust members of professional organizations, in April, Sweet’s group asked physician members of the British Columbia infectious-diseases and critical-care societies about the safety of COVID-19 vaccines.

They asked the 199 experts to agree or disagree with 7 statements, such as “I would recommend COVID-19 vaccinations to my family and friends.” Some 135 (68%) responded, with the results showing that, for example, 99% of the doctors thought that the vaccines were safe and effective in adults.

Follow-up questions to other specialist professional groups could feature topics including climate change, microplastics and even conspiracy theories, Sweet says. His group is less interested in proving (or disproving) consensus, he says, and more in gauging levels of agreement and disagreement, then later probing the reasons why.

After gathering responses to questions, the project plans to use the results to generate videos and other social-media-friendly content, with the idea of combating misinformation. “It then has the ability to go viral, just like all the misinformation out there,” Sweet says.

To promote its COVID-19 vaccine poll results, the foundation has produced a video featuring Canadian actor Jewel Staite. “So much of what you see on social media and the Internet conflicts [with] what public institutions are telling us,” she says in the video. “Many people don’t know what to believe, but there are new solutions coming.”

Website debate

Fanelli also plans to invite only experts to weigh in on questions at the comCensus website. His project is specifically not seeking the average scientist’s view, because researchers without direct experience in a field won’t have first-hand expertise of it.

Those invited (perhaps a few dozen to a few hundred) get a password to access a question on the site, so that no one can comment without being invited. Fanelli’s system, which could be closed or open for the public to view, depending on the issue, aims to encourage participants to articulate their evidence and to vote on others’ contributions, such as which comment they found most insightful or got them to reconsider. It also allows participants to change their minds on the question over time.

The result, Fanelli hopes, is a website that will empower people to find out how experts think and to reflect on the rationales driving different answers, with the help of data visualizations that show, for instance, which answers were voted most helpful or most convincing.

“By the end, one has a better, worked-out picture of what the true consensus is, because everybody who could have changed their mind — or made someone else change their mind with good arguments — has had a chance to do so,” he says.

Fanelli had the idea for his site after seeing how claims of scientific consensus were used to stifle debate during the pandemic. Specifically, he says, he did not like the way in which questions on the origins of the coronavirus SARS-CoV-2, and whether it could have escaped from a laboratory, were dismissed by many high-profile voices as unscientific and off-limits.

With no dedicated funding, Fanelli launched a basic version of the website in April to show how it could work (www.comcensus.org). In pilot studies, he has invited virologists to comment on whether the database GISAID is the best venue for sharing information on pathogens in a public-health emergency, and asked researchers in artificial intelligence (AI) about the merits of a moratorium on the development of advanced AI systems. In these pilots, the platform creates pseudonyms for commenters, so that it is not possible to see who holds each view.

Rather than suggest questions himself, Fanelli has created a model in which individuals or organizations would pay a nominal fee for administration costs (perhaps £100 or so) and use his site to ask questions of their own devising. Those sponsors would decide whether the question, answers and discussions are available for anyone to read, or restricted to a limited group.

Fanelli has built it, but will people come? “People wanting to use it, that would definitely be a marker of success,” he says. The ultimate mission, he says, is for the platform to play a part in public conversations.

Many social scientists have already established robust methodologies to assess degrees of scientific and medical consensus — for instance, to help policymakers understand the evidence for the cost-effectiveness of a drug or a social policy. And if Vickers, Sweet or Fanelli envisage their work being used to guide authoritative advice on high-stakes policy issues, such as in public health, more-rigorous survey methods might be needed, says Sean Grant, who studies the generation and use of evidence at the University of Oregon in Eugene. “How they think [their findings] will be used is really important for me,” he says.

At this stage, it’s unclear how the three projects will pan out. But they could prove useful to inform and influence public discussion, says Dang. Scientific consensus has an important social role: besides driving policymaking, it can help to communicate science and connect with people outside science, she says. “It depends on what the goal is, what you think consensus does for you.”