Sir

In your News story 'UK scientists get funding ban reprieve' (Nature 459, 20; 2009), you report the response of the UK Engineering and Physical Sciences Research Council (EPSRC) to criticism of proposed changes to grant-submission eligibility. In my view, the EPSRC response still fails to address the central issue.

Even with the softened proposal, blacklisting will severely damage researchers' careers; moreover, it will selectively damage the productivity of our most innovative and daring researchers. It is critical that any measure used to do this should be robust, objective, transparent and widely trusted. Peer review as a means of numerically ranking grant proposals satisfies none of these criteria. Studies on peer review of grants (for example, S. Cole et al. Science 214, 881–886; 1981) and of papers (for example, L. Bornmann and H.-D. Daniel Learn. Publ. 22, 117–125; 2009) have consistently shown that peer review performs poorly at numerical ranking, that it is subject to serious random effects, and that it is not good at distinguishing between the majority of proposals and papers that fall between the very top and the very bottom. Correlation between results when a peer-review process is repeated is only marginally better than would be expected by chance.

Therefore, whether a proposal falls within the top or bottom 50% of the ranking is largely determined by chance, and has little relation to any objective measure of quality. The chief executive of the EPSRC has been reported as accepting that peer review is “basically a lottery” — a widely held view among researchers. However, the 50% mark is the key measure that the EPSRC proposes to use. The revised proposals are therefore still capable of potentially destroying people's careers on the basis of a measure that is only slightly better than flipping a coin.