Less prestigious institutions deliver better value for grant money

Study finds "wasteful" skew in NIH funding towards those at the top.

18 October 2018

TaniaLuz/Getty

An analysis of United States National Institutes of Health (NIH) grant allocations finds that prestigious institutions deliver much lower output per dollar spent than less prestigious institutions, yet get a disproportionate share of funding.

Wayne Wahls

The study of NIH grants from 2006 to 2015 found that less prestigious institutions produced 65% more publications and had a 35% higher citation impact per dollar of funding than prestigious institutions.

Even so, prestigious institutions had on average 65% higher grant application success rates and 50% larger award sizes from the NIH, according to the paper by biomedical researcher, Wayne Wahls, of the University of Arkansas for Medical Sciences. It is published on the preprint server bioRxiv.

The institutions were divided into prestigious and less prestigious groups based on their positions in the 2016 US News & World Report Best Medical Schools: Research rankings. The prestigious institutions were Harvard Medical School, Stanford University, Johns Hopkins University, University of California, San Francisco, and the University of Pennsylvania. The less prestigious institutions included the University of Nebraska Medical Center (ranked 63) and the University of South Dakota (88).

Previous studies have shown troubling disparities in success rates for NIH applicants based on colour, age, gender and geography. Institutional prestige can now be added to that list, says Wahls.

Even if the funding disparities weren’t driven by bias, “giving the lion’s share of grant dollars to a small minority of institutions seems counterproductive and wasteful,’’ he asserts in his paper.

The NIH is the largest source of funding for biomedical research in the world, distributing nearly 80% of its US$37.3 billion budget for 2018 in competitive grants.

Value per dollar

Wahls analysed funding and success rates of a cross-section of 15 institutions receiving NIH funding of between US$3 million and US$400 million a year. The analysis took in 41,000 research project grant awards allocated to more than 6,000 principal investigators.

Over the decade from 2006 to 2015, “each of the prestigious institutions outperformed, by every metric, each of the less prestigious institutions in securing NIH research project grant funding,” the study found.

For example, as a group, investigators at the prestigious institutions were on average 1.7 times more likely to get each grant application funded than those at the less prestigious institutions. They were awarded on average 2.4 times more funding than those at less prestigious institutions — US$3.5 million versus US$1.5 million.

The disparities might be justified if the more prestigious institutions added greater value to the scientific research enterprise, Wahls reasoned. But based on 95,000 publications associated with the research grants, he found that the less prestigious institutions each produced more scientific publications per dollar of grant funding than each of the prestigious institutions (8.7 versus 5.3 publications per million dollars of funding).

Inequality rising

A 2017 study by Yarden Katz, a departmental fellow in systems biology at Harvard Medical School, and Ulrich Matter at the University of St. Gallen found that funding inequality has been rising since 1985. A small number of researchers and institutes receive a growing share of NIH funds, with a static pool of investigators at the top ranks of funding.

Wahls’s analysis also found that in 2015, half of all NIH grant dollars went to just 2% of funded institutions, 19% of funded investigators, and 10% of US states.

Wahls does not suggest grant reviewers and NIH officials at large are overtly biased, but rather that they are subject to subconscious bias, which favours “non-meritocratic factors such as the wealth, reputation and selectivity of institutions.”

Diminishing marginal returns for large grants are well documented from other studies, attributed to the fact that individual investigators have finite capacity, and their productivity declines when the size of their grants exceeds their capacity to manage them.

Wahls proposes that the NIH adjust success rates and award sizes to establish parity or near parity between institutions in a ratio of success rate to productivity within a target range. “A more egalitarian distribution of funding among institutions would yield greater collective gains for the research enterprise and the taxpayers who support it,” he asserts.

Critically important

A spokesperson for the NIH Office of Extramural Research said in an email that the Wahls study dealt with “a critically important issue” but had not yet undergone peer review.

“We encourage the research community to provide feedback to the author so that the results and analysis can be as strong as possible before it is submitted for publication,” the NIH spokesperson wrote.

Katz at Harvard took issue with the “return on investment” approach to evaluating science.

“We pour dollars in and apparently all we care about is how many citations and publications come out. This is a toxic and silly goal. It ends up masking the many important structural problems within academic science, and giving the illusion that with more data, the ‘objective’ metrics can be crafted,” wrote Katz in an emailed response, without commenting on Wahls’s paper specifically.

“The hard questions about what kind of academic scientific sphere we’d like to have — what kind of science, and what types of scientists, should be given opportunities — are mostly erased in this framing,” wrote Katz.

Cap plan abandoned

The NIH in 2017 moved to address inequity concerns by capping the number of grants provided to a single principal investigator, a reform that was calculated to affect only 3% of investigators while freeing up funding for potentially 900 more, but abandoned the plan after an outcry.

Wahls has been an independent principal investigator since 1995. His research on chromosome dynamics in reproductive cell division (meiosis) is supported by a current NIH grant.