Unpopular policy limiting applicants to two submissions has sped up approvals.
When does no mean no? The answer for researchers applying for funding from the US National Institutes of Health (NIH) is after the second 'no'. Last week, officials at the biomedical agency stood by that policy despite a 20 February petition from 2,335 scientists urging them to restore a system that allowed three tries per project. Some 400 more researchers have since asked to join the campaign.
The NIH's two-strikes-and-you're-out approach to funding grants was implemented in 2009 in an effort to speed up a dilatory review process. Last week, in a letter to petitioners, Sally Rockey, the agency's chief of extramural research, and principal deputy director Larry Tabak wrote that the policy is working as planned. Not only has the average time to approve a grant been reduced by almost half a year, they argue, but the proportion of applicants winning funding on their first attempt has "risen dramatically" (see 'Shorter, faster, better?').
Tabak told Nature that, under the former system, peer reviewers tended to reject meritorious grants that were being submitted for the first time because they knew that the applicants would have two more attempts at funding — in effect setting up a queuing system like that of planes lining up for take-off. The two-strikes policy, he says, "is accomplishing what it set out to do, namely to ensure that we support the strongest science as early as possible".
Robert Benezra, the cancer biologist at the Memorial Sloan-Kettering Cancer Center in New York who instigated the petition, is not persuaded. After digesting comments from the first 200 petitioners who wrote to him after he circulated the NIH response, he says, "the overwhelming sentiment is frustration with the fact that the NIH just ignored the central premise of our argument".
That premise is that peer reviewers cannot reliably distinguish between an application scoring in the fifth percentile — near the top — and one in the twentieth percentile. Therefore, in a time when limited resources are squeezing the proportion of fundable projects to as low as 5%, reviewers are effectively choosing at random from too many worthy proposals.
"Removing [second resubmissions] has the effect of eliminating outstanding grants that would otherwise be funded in time. In effect, queuing is the only fair way to go when dollars are so scarce," Benezra wrote to the petitioners on Friday. He recommends that the agency define a threshold — he suggests the 25th percentile — and allow grants that score at this level or better a second resubmission.
One of the petition's signatories, Martin Chalfie, a Nobel-prizewinning neurogeneticist at Columbia University in New York City, says that his deepest concern is the effect of the two-strikes policy on young scientists. "There's a learning curve" to grant writing, he says, and "for junior people, we are making that learning curve exceptionally steep".
In their response, Rockey and Tabak counter that the new policy "seems not to have had an effect on new or early-career investigators". In fact, they argue, returning to a three-strikes policy might in fact work against young scientists trying to launch independent labs by lengthening the time it takes to get winning grants through the system.
Although "there is little doubt that some great science is not being funded", Rockey and Tabak wrote, that is because budget pressures have forced the agency to fund an ever-diminishing proportion of applications. "Restoring [second resubmissions] will not change that picture."
Despite the groundswell calling for a return to a three-strikes system, some researchers say that even the two-strikes rule does not go far enough. Keith Yamamoto, executive vice-dean of the School of Medicine at the University of California, San Francisco, chaired an advisory committee that recommended that the NIH abolish resubmissions altogether — sparing applicants the time and stress of responding to one set of reviewer criticisms only to be rejected by a new group of reviewers with different opinions. Although the new system is working, says Yamamoto, eliminating resubmissions "would have worked even better".
Tabaka and Rockey note that the agency plans to solicit community opinion on the two-strikes policy in its next survey of extramural grantees late this year.