متوفر باللغة العربية

The US National Institutes of Health (NIH) receives many more grant applications proposing outstanding scientific projects than its budget can support. The overall success rate for grants in fiscal year 2011 was 18% — a historic low.

Last month, the agency announced a policy that will apply to the next round of R01 grants. Applications from investigators who receive more than US$1 million a year in funding from the NIH (not including 'indirect costs' that go to the investigator's university) will be subjected to extra review. The agency's advisory councils will examine how distinct the proposed project is from the investigator's other funded work. If it is a grant renewal, they will consider whether the project has been productive, and look at its value to the investigator's research programme and collaborations. This analysis will be provided to NIH programme staff, who will make funding recommendations.

The policy has been criticized for adding to the NIH's administrative burden, and for potentially penalizing the most productive scientists. However, I believe that it will complement peer review and help to determine the best possible portfolio of NIH-funded research, short and long term.

The National Institute of General Medical Sciences (NIGMS) in Bethesda, Maryland — where I was director between 2003 and 2011 — has, for around two decades, given extra scrutiny to applications from investigators receiving more than $750,000 per year in costs from all funding sources, including the pending NIGMS application. This policy is not a cap; rather, it is a useful tool for managing taxpayer resources to achieve the goals of the NIGMS and the NIH. The scrutiny has influenced the distribution of resources, either by denying or reducing funding for well-off applicants, or providing grants on the condition that others will not be renewed. This frees up money for other investigators and allows those who are well-funded to change the course of their research. Assuming that the NIH policy can be implemented effectively, it is a step in the right direction.

Critics of the NIH policy argue that although peer review is not perfect, it is the best system for evaluating scientific projects. Many reviewers try to ensure that a new project does not overlap significantly with one that is already funded. But they do not have full access to NIH portfolio data, nor the time to analyse the information.

Peer review has several limitations. To compare and rank grant applications across hundreds of NIH sections, administrators assign each peer-review score a percentile value. The first percentile (the top 1%) corresponds to the most highly rated applications. These percentile values have uncertainties that stem from the potential biases of primary reviewers, the characteristics of the process used to calculate percentiles, and the fact that an application is judged by whether it will be an important future scientific advance — which is inherently difficult to predict.

Bang for buck

During my time as director of the NIGMS, I analysed the power of peer-review scores to predict scientific productivity. This was judged by various measures, including numbers of publications, citations and highly cited publications four years after the grants were funded. The distribution of percentile scores for applications did correlate with future differences in productivity to some extent. But applications that were separated by five or even ten percentile points did not differ significantly in subsequent productivity (see 'Score draw'). Therefore, funding grants in strict percentile order does not necessarily mean that ensuing investigations will be of higher quality. So percentile score should not be the only factor taken into consideration when making funding decisions.

Credit: SOURCE: J. M. BERG/NIGMS

There are good reasons to consider how much other funding an investigator is receiving. Projects in one lab almost always overlap, even if they are addressing different questions, so some of the running costs will already be covered. Giving another grant to the same investigator might have less impact than giving it to someone with nearly the same percentile score who has little or no other funding. In another NIGMS analysis, I showed that research productivity (measured by numbers of publications or citations and averaged over groups of investigators with similar levels of support) did not increase consistently as the level of support increased, but reached a plateau near $700,000 in annual direct costs per investigator.

Some investigators are better than others at managing large amounts of resources. If an investigator is not performing as expected given the level of support, using the funds to support another researcher might be a better investment. The NIH needs to invest in a broad range of research; if scientists are well funded, their area could already be receiving enough support from other sources.

The layer of extra scrutiny is not ideal. I would prefer the limit to be less than $1 million and to include funding outside the NIH. Exceptions to the policy also mean that it might be easy to dodge. For example, grants with multiple principal investigators are scrutinized only if all receive more than $1 million in NIH funding, so I am concerned that some applicants may avoid extra review by adding a co-investigator who receives less.

I believe that special consideration should be given to investigators with strong proposals who have few or no other sources of funding, such as those at the beginning of their careers or established, productive investigators. Funding these applicants would probably have a bigger impact — by helping to develop a new lab or keeping an effective one functioning — rather than providing incremental support to an investigator who already has substantial other support.