Published online 12 August 2011 | Nature | doi:10.1038/news.2011.475

News: Q&A

Toni Scarpa: reviewing peer review

The National Institutes of Health's departing head of scientific review reflects on his tenure.

Toni Scarpa.NIH

Toni Scarpa, the director of the Center for Scientific Review at the US National Institutes of Health (NIH), is leaving the job next month after six years. Scarpa came to the agency from Case Western Reserve University in Cleveland, where he had built a nationally ranked department of physiology and biophysics. Then-NIH director Elias Zerhouni charged him with overseeing a major overhaul of the agency's peer-review system. This included a shift to evaluating applications for impact and significance, and halving the length of grant applications. Scarpa spoke to Nature earlier this week.

What accomplishment do you feel proudest of during your tenure?

Making the biggest change in peer review in 65 years. Elias [Zerhouni] was passionate about peer review. And he brought me in as the agent of change in an institution where changes are resisted.

What were the most important of those changes?

We started by trying to get the best reviewers, and by making it so that a lot of clinicians could review without coming here. We abolished deadlines, making the process one or two months faster. And we brought in affirmative action for new investigators: now, each institute has to find a sufficient number of new investigators to fund, about 30% of its grants.

NIH reduced the length of grant applications from 25 to 12 pages. Has that been a good thing?

The goal was to ask people to focus more on impact and significance. Peer review is simple — I think it should ask only two questions. First: Is it worth doing? That is impact and significance. If the answer is yes, then you ask the second question: Can they do it? In the past we were asking those questions in reverse.

There were several ways to implement this shift: one was to score impact and significance independently. The other was to make the application shorter, so that it can't be filled with a lot of details.

Has that change been successful?

I think so. I have the privilege of interacting with 600 chairs of study sections, in small groups. They seemed to be quite pleased. People can see that focusing on impact and significance is probably the way to go.

The peer-review process is subject to continuous evaluation. What metrics have you used to assess whether the changes are working?

Sally Rockey, NIH's director of extramural research, has done a major review of those changes.

In May, Francis Collins, the current NIH director, told the Senate subcommittee that funds the NIH that the success rate for grant applications in 2011 would be historically low, at 17-18%, and possibly lower in 2012. How has this affected morale?

This situation is not unique. I remember that when I was on the other side in 1988, you had to have [extremely high grant scores] to sleep well at night.

People should not forget that we still have a US$30-billion agency. We are privileged. In flat budget times — and even flat may be optimistic — somebody has to set priorities. We should not just cut everything by inflation or by 3% every year. You expect the NIH to make some strategic decisions.

What's the most important thing you've learned on the job?

Changing an organization that resists change is quite difficult. And you need support from the top. Dr Zerhouni not only hired me but wanted those changes as much as I did.

Why are you leaving?

ADVERTISEMENT

I have been here six years, which I believe is the right amount of time for a centre director. Francis can select somebody with new energy and new ideas to carry on a different way.

You've mentioned Zerhouni's support. Have you not felt the same support from Collins?

Enhancing peer review was a major part of Dr. Zerhouni's agenda, and it is a major part of his legacy. Historic changes to NIH peer review are mostly done. Francis is in a different boat, and his priorities, as they should be, are different now.

What's next for you?

I have had to postpone some private matters, and I need to attend to those for a few months. And then I will look at the horizon and see what appears. 

Commenting is now closed.