Achievement index climbs the ranks

Hirsch measure can predict future success of researchers, says Hirsch.

A tool that ranks researchers by the quality of their work is also a good predictor of their future performance1, according to its designer, physicist Jorge Hirsch at the University of California, San Diego.

At 110, Ed Witten (left) has a much higher h-index than Stephen Hawking's 62. Credit: TECHNION; J. RAOUX/AP PHOTO

The h-index was first proposed in 2005 by Hirsch2, and attracted immense interest from scientists because of its claimed objectivity in ranking scientific achievement. Although other bibliometric measures of performance, such as counting up the number of papers published in Nature and Science, have widely acknowledged shortcomings, they are still used to assess an individual's or department's research.

Scientists generally prefer the alternative of assessment by peer review, but this process can be affected by subjectivity and bias. A fair and transparent method for gauging performance is greatly desired, especially by young scientists seeking tenure or academic placement. To this end, the British government has announced that after 2008, it will base funding assessments for universities purely on such a metric.

But which to use? Counting the number of papers might give an undue advantage to scientists who publish lots of little consequence, or in small 'salami slices'. The number of citations is a better measure of a work's impact, but it can be distorted by a few highly cited papers among many indifferent ones.

The ratio of these two measures — the mean number of citations per paper — is also a good indicator of performance3. But the h-index offers an alternative: it is the number n of a researcher's papers that have all received at least n citations. Thus, Frank Wilczek's h-index of 68 (see 'From the top down'), means that 68 of his papers have been cited at least 68 times each.

From a selection of well-cited papers in the journal Physical Review B, Hirsch selected 50 authors who had started publishing papers in the 1980s, and assessed how their careers had developed. He calculated how well various metrics — the h-index, the number of publications (productivity), the number of citations and the mean number of citations — during the first 12 years of their publishing career predicted the values over the next 12. But rather than assessing the cumulative impact of previous and future work, he investigated how well the indices predicted the quality of subsequent work.

Hirsch found that the h-index was slightly more accurate in this respect than the number of citations, and substantially better than the other two indices. It even predicted future productivity better than did past productivity. “Perhaps prolific authors with small citation counts feel less incentive to continue being prolific, as they perceive that their work is not having an impact,” he suggests. In other words, they decide that it's not worth trying so hard.

Hirsch also shows how the h-index can discriminate in papers with multiple authors. “I'm concerned about the blind way in which a co-author of a paper is allocated a full citation count for that paper, irrespective of his or her individual contribution,” he says. The h-index automatically allocates a smaller proportion of the credit to the researchers who were likely to have contributed less, so “it does a good job of unravelling that”, he explains.

One of the h-index's benefits, Hirsch claims, is that it is hard to manipulate by self-citation — researchers would have to cite many of their own papers many times to boost their h-index rating. But physicist Michael Schreiber, of the Technical University of Chemnitz in Germany, argues that self-citations can distort the value appreciably4, and he proposes that they be excluded to 'sharpen' the index. James Fowler, also of the University of California, San Diego, says that even when self-citations are removed from citation counts, those who cite themselves more seem to be more successful. His group found that more than half of the citations a scientist receives can be attributed directly or indirectly to self-citation5.

Nonetheless, the h-index does seem to be able to identify good scientists, and it is becoming widely used informally, for example to rank applicants for research posts. “People in various disciplines are using or considering using the h-index as one of the criteria both for appointments and for grants allocation”, says Hirsch. “I have seen several job applicants send us their h-index in their CV.”

Schreiber agrees that “the use of the h-index is increasing, at least unofficially.” Whether you like it or not, he says, “the h-index is here to stay”. However, Schreiber cautions, “it is always a problem to reduce research work to one number. We should be careful using it and should be aware of its limitations.”

References

  1. 1

    Hirsch, J. E. Preprint at http://www.arxiv.org/abs/0708.0646 (2007).

  2. 2

    Hirsch, J. E. Proc. Natl Acad. Sci. USA 102, 16569–16572 (2005).

  3. 3

    Lehmann, S., Jackson, A. D. & Lautrup, B. E. Nature 444, 1003–1004 (2006).

  4. 4

    Schreiber, M. Preprint at http://www.arxiv.org/abs/0708.0120 (2007).

  5. 5

    Fowler, J. H. & Aksnes, D. W. Scientometrics 72, 427–437 (2007). | Abstract

Download references

Related links

Related links

Related links in Nature Research

Top five in physics

Brown's budget briefing

Prestige is factored into journal ratings

Index aims for fair ranking of scientists

Related external links

Hirsch paper

Schreiber paper

H-index

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ball, P. Achievement index climbs the ranks. Nature 448, 737 (2007). https://doi.org/10.1038/448737a

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.