Sir

Further to your News story 'Revamp for NIH grants' (Nature 451, 1035; 2008), the final draft of the US National Institutes of Health (NIH) 2007–08 peer-review self-study report tackles a critical set of issues for us grantees and reviewers. We all want an efficient and stable system; we all particularly care about junior investigators. But the solutions need to be grounded in evidence.

As you point out, a back-of-the-envelope calculation shows that eliminating more than two grants per investigator would allow a 5% increase in the number of grants. But is this better? Another casual calculation shows that requiring all grants to be shared by two investigators would more than double the number of grantees. So?

How do we know that eliminating amended applications will help? Amended applications are what keeps everyone honest. Without the back-and-forth of peer review, efforts by the NIH will degenerate into the dismal and ad hoc review process that characterizes many other agencies. And I would be reluctant to review the expanded load of always-new proposals.

Would a shorter application really enhance the quality of research or the efficiency of the review process? I don't want to spend extra time looking for lost key information. Is this a good idea? Why don't we study it?

Likewise, it is unclear whether using more than four reviewers would enhance the system or bring it to a screeching halt owing to the huge burden of review.

Also, what is the evidence that giving more R01 grants (individual grants that are the mainstay of extramural funding through the NIH) for early-career investigators will work in attracting the best and brightest to the life sciences and biomedical research? The issue is complex. I am not sure whether attaining that first R01 is the solution: it is the ability to stay there in a healthy and productive career.

Some suggestions are 'pilots', which should be applauded, but again there is little discussion of what is being measured and how.

It is high time for the NIH to conduct research on the review process itself. With hundreds of study sections meeting and thousands of applications reviewed every year, the material is there for retrospective analysis and for prospective studies that test specific hypotheses and interventions.

Perhaps the peer-review self-study should have submitted an R01 application on the proposed changes.