Scientists have long suspected that proposals that bridge diverse disciplines of science have lower odds of being funded. A study of Australian funding decisions now suggests that this is true.
Nature's Shamini Bundell asks Lindell Bromham why, when it comes to interdisciplinarity, funding agencies may not put their money where their mouths are.
The findings seem to confirm the anecdotal evidence, says Rick Rylance, a chair of Research Councils UK, a major funding agency in the United Kingdom. “I would be surprised if a similar study undertaken here did not come to the same conclusions.”
The study is itself a multidisciplinary project, combining methods of evolutionary biology and social sciences1. Lindell Bromham, a biologist at the Australian National University in Canberra, and her collaborators analysed data on more than 18,000 proposals — including successful and unsuccessful ones — submitted to the Australian Research Council (ARC) Discovery Programme between 2010 and 2014.
To establish each proposal’s level of interdisciplinarity — something that is notoriously difficult to evaluate — the team developed its own metric, which it calls ‘interdisciplinary distance’. The measure assigns a value of between 0 and 1 on the basis of the information included in the submissions. Every proposal to the ARC has to select one or more fields from the ARC’s official list, which includes 1,238 hierarchically structured field-of-research codes. Candidates must also specify the relative weight given to the various fields.
The team gave proposals that indicated only one code an interdisciplinary distance of 0; the rest were assigned a value of up to 1 depending on the relative weights of the indicated fields and on how distant the fields were from each other in the hierarchical tree. The method is similar to a biodiversity metric that takes into account the distance between species on an evolutionary tree and their relative abundance in an ecosystem.
The researchers found that the higher a proposal scores on their interdisciplinary scale, the lower the probability that the ARC would fund it.
The authors say that they were unable to uncover the reason. In particular, they found no correlation between a university’s success rates and the number of interdisciplinary proposals it submits, Bromham says. “But we cannot rule out that there are other indirect factors we cannot test — for example, whether more experienced or successful chief investigators tend to submit fewer interdisciplinary proposals.”
Still, she and her co-authors suggest some possible explanations for the pattern: perhaps, the way that the grant-review process works could be intrinsically skewed against interdisciplinary projects; or independent reviewers may have a hard time understanding the value of proposals that involve fields outside their area of expertise; or perhaps such proposals just tend be of lower quality, on average, or not as convincing.
Whatever the cause of the pattern, the authors think that their study should prompt funding agencies to conduct further research to work out the possible reasons. They also suggest that the metric they developed could give a way to identify highly interdisciplinary proposals that might need a special approach to evaluate.
“This work is a significant outcome and one that the continuing discussion of interdisciplinary research needs,” Rylance says.
ARC chief executive Aidan Byrne says that his agency is taking a serious look at the results. “Their work gives us an independent way of determining the degree of interdisciplinarity of a research proposal”, and that could help the agency to track the progress of any changes it implements in its selection process, he says.
The paper’s conclusions are similar to those drawn by the US National Institutes of Health (NIH) before it launched a roadmap programme in 2004 to stimulate interdisciplinary research, says Betsy Wilder, a director of the NIH Office of Strategic Coordination.“It would be interesting to use the metric devised by Bromham et al. to assess interdisciplinarity of applications versus funded projects in the NIH portfolio and to determine whether the ratio has changed over the past ten years.”
Vincent Larivière, an information scientist at the University of Montreal in Canada, warns that assessing the level of interdisciplinarity through a principal investigator’s choice of field codes could be misleading. “These self-declared disciplines might serve more as signalling devices rather than actual markers of the content of the proposal,” he says. Using the cited references in grant proposals would give a more accurate picture, he says, in part because “the act of citing” is something that is better understood.
The authors say that it would be interesting to apply their techniques to data from other funding agencies. A 2015 review of research commissioned by the Higher Education Funding Council for England showed that research publications that straddled disciplines tended to have a lower citation impact.
- Journal name: