Access
This article is part of Nature's premium content.
Published online 19 March 2009 | Nature | doi:10.1038/news.2009.177
News
UK researchers lament grant ban
A string of unsuccessful proposals means being barred from making further applications.
A British research funding body has provoked outrage among scientists by imposing a year-long ban on serially unsuccessful applicants. The rule was announced by the Engineering and Physical Sciences Research Council (EPSRC) on 12 March.
To read this story in full you will need to login or make a payment (see right).
Comments
Reader comments are usually moderated after posting. If you find something offensive or inappropriate, you can speed this process by clicking 'Report this comment' (or, if that doesn't work for you, email webadmin@nature.com). For more controversial topics, we reserve the right to moderate before comments are published.
Call me cynical, but it seems to me that this EPSRC policy isn't so much aimed at relieving the burden on peer reviewers (who, after all, are also usually grant applicants themselves), so much as it is aimed at further concentrating limited funds in the hands of ever fewer principal investigators. Already, EPSRC has had platform grants, program grants and the like, all draining funds from the ordinary responsive mode which is open to all. I think this move is designed to deter the mid-ranking but research-active scientist, probably already burdened by more than their fair share of teaching and administration, from even thinking of applying to EPSRC to support their research. As I have no intention of being party to causing a colleague in another institution to have to undergo 'mentoring' and being banned from EPSRC for a year, I am going to refuse to do peer review for as long as EPSRC persist with this mistaken and patronising idea. If they really want to cut the burden on peer review, why don't they limit the number and/or value of grants people can apply for in any one year, as I believe is done in the US system?
The statements made here by EPSRC and Wakeham seem to fail to address two critical issues which must surely like at the centre of a discussion of this proposal. Firstly is blacklisting morally and ethically acceptable? It is clear that many scientists feel strongly it is not. But far more importantly is the proposed criteria. Blacklisting will essentially destroy a researcher's career. Therefore it is critical that any measure used to do this be robust, objective, and widely trusted. Peer review as a method of numerically ranking proposals and papers is known to be unreliable. Studies on this in peer review of grants (Cole et al, Science, 214, 881, 1981; Hodgson, J Clin Epedem, 50, 1189, 1998) and of papers (Rothwell and Martin, Brain, 123, 1954, 2000; Bornmann and Lutz, Learned Publishing, 22, 117, 2009) have consistently shown that peer review performs poorly at ranking, that it is subject to serious random effects, and specifically that it is poor at distinguishing between the majority of proposals and papers that fall between the very top and very bottom. Correlation between the results when a peer review process is repeated are marginally better than that expected by chance. Thus whether a given proposal falls in the top or bottom 50% of the panel ranking is to a very large extent determined by chance, and has little or no relation to any objective measure of the proposal's quality. This is the key measure that EPSRC propose to use. The current proposal therefore amounts to destroying people's careers on the basis of a measure which is only slightly better than flipping a coin. Disclaimer: This comment represents my personal opinion and does not represent the policy, opinion or position of either of my employers (I am employed by one of the UK research councils, STFC, and by the University of Southampton)
On Cameron's rather important point above, it's worth noting that David Delpy admitted at an open EPSRC meeting in Nottingham last year that, "We all know the peer review process is a lottery". In addition, the suggestion that the changes to peer review were introduced as a response to the RCUK consultation on peer review is laughable, given that EPSRC chose to ignore extremely negative feedback via that "consultation" from practically every UK university on the subject of the imposition of economic impact criteria in peer review. This negative feedback is discussed at the blog of Steven Hill, the Head of RCUK's Strategy Unit. (See "http://hypotheses.wordpress.com/2009/02/13/a-nobel-effort/#comment-3" - unfortunately we can't include HTML links in comments here). Where in the consultation did RCUK ask the acadcemic community for their comments on the retrospective introducion of a ban on researchers applying for grants? Moreover, to suggest that EPSRC introduced this policy because it was listening to advice from the academic community begs the question as to why they didn't follow the advice from UK universities on the imposition of economic impact criteria in peer review. Philip
Many times a grant is rejected and the applicant gets a Nobel. The system is also not free of personal biases. Hence exclusion right in the beginning is not in the interest of science. It will also discourage students and young scientists to go persue a real research career.
Instead of improving quality, this policy may stop very active researchers submitting risky ideas. Peer review is subjective and a person's success rate is not a direct measure of the -quality- of his or her ideas. Some proposals for extremely high quality work have been rejected in the past. And even if it were a measure, 25% is now actually above the success rate of the council! The criterion EPSRC seems to want to introduce is, therefore, not related to quality, but to the 'amount of proposals submitted'... a strange criterion for a council aiming to fund high quality, creative, projects. How can EPSRC know how many is too many? Einstein published a lot of papers around 1905, and most people agree that it would have been silly to slow him down at the time. The real question is why success rates are dropping so rapidly and what else EPSRC is doing about it. Simply reducing the number of proposals to make success rates look better again seems cheating. Has the number of proposals this year really gone up so much? Is it really the -cause- of the drop in success rates? If not, reducing numbers seems a strange solution. Next year, with much less proposals, I bet EPSRC will publish the 'positive' effect of their policies on success rates...but really, what would that mean?
See also a News story in the 25 March issue of Nature here: http://www.nature.com/news/2009/090323/full/458391a.html And an associated Editorial here: http://www.nature.com/nature/journal/v458/n7237/full/458385b.html