Nature 457, 785 (12 February 2009) | doi:10.1038/457785a; Published online 11 February 2009

Multiple metrics required to measure research performance

Stevan Harnad1

  1. Institut des Sciences Cognitives, Université du Québec à Montréal, Montréal, Québec H3C 3P8, Canada; and School of Electronics and Computer Science, University of Southampton, Southampton SO17 1BJ, UK


Your Editorial 'Experts still needed' (Nature 457, 7–8; 2009) is correct in that no metric alone can substitute for expert evaluation, because no single metric (including citation counts) is correlated strongly enough with expert judgements for it to take their place. But some individual metrics, such as citation counts, are nevertheless significantly correlated with expert judgements. It is likely that a battery of multiple metrics, when considered jointly, will be even more strongly correlated.

The UK Research Assessment Exercise (RAE) provides such an opportunity, alongside the wealth of potential performance indicators that are increasingly available online. Both enable a candidate battery of metrics — such as citations, co-citations, downloads, tags and growth/decay metrics — to be systematically validated against expert judgements, field by field. The 2008 RAE has also provided data that make it possible to do this validation exercise now, across all disciplines, on an important nationwide scale.