The Netherlands has launched what researchers say is the world’s first national fund dedicated to replication studies: a pot of €3 million (US$3.3 million) over the next 3 years for Dutch scientists to test whether they can reproduce important research results in social and medical sciences.
The pilot programme was announced on 19 July by the Netherlands Organisation for Scientific Research (NWO), the country’s largest research-funding agency. It marks a tiny fraction of the agency’s €700-million annual budget, but is an important step, says Brian Nosek, executive director of the Center for Open Science in Charlottesville, Virginia. “If my calculations are correct, this is an increase of infinity per cent of federal funding dedicated to replication studies,” he says.
Nosek has led an effort to replicate work from 100 psychology publications, which relied on what he estimates was $4 million in donations of time and resources from participating researchers. Even limited funds for replication can make innovative research more efficient, because it helps researchers flag up blind alleys, he says. “Confirming a result helps justify further investment in that line of inquiry. Failing to confirm a result identifies challenges that should be investigated further before going all in.”
“This clearly signals that NWO feels there is imbalance in how much scientists perform replication research, and how much scientists perform novel research,” says Daniel Lakens, a cognitive psychologist at Eindhoven University of Technology in the Netherlands.
The NWO pilot will focus on repeating “cornerstone” research — studies that have had a large impact on science, government policy or public debate. Jos Engelen, chair of the NWO’s governing board, says that the agency expects to be able to fund 8–10 projects each year. A study that collects new data can be funded with up to €150,000; one that re-analyses existing data can receive up to €75,000. Scientists are not allowed to use the money to replicate their own work, and the official call for proposals should come in September.
The NWO programme is making a good investment by encouraging replication, says Sam Schwarzkopf, a cognitive neuroscientist at University College London. But he argues that replication should simply be built into general research — for instance, by requiring that researchers funded for innovative work must also perform direct replications of previous work.
Daniel Gilbert, a psychologist at Harvard University in Cambridge, Massachusetts, who has previously criticized Nosek's work, says the benefits of dedicating funds for replication studies remain to be seen. "If the Dutch government wants to spend its money on research whose sole qualification is its unoriginality, then that’s their prerogative. Will we learn something valuable from such research? Probably. Will it be more valuable than what we would have learned if the same amount of money had been spent exploring important new ideas? That’s a difficult question to answer. But it is the critical question, and it is the question no one asks," he says.
The NWO hopes to use insights from its pilot programme to incorporate replication into research more broadly, but exactly how that will work is uncertain, says Engelen. “It is too early to have expectations about this type of funding becoming a regular fixture,” he says.
- Journal name: