Publishing: Criteria for Nature Index questioned

Journal name:
Date published:
Published online

The Nature Index gauges the performance of countries and research institutions by tracking the number of papers they publish in reputable journals ( We suggest that using absolute numbers and journal reputation can yield misleading results (see R. Haunschild and L. Bornmann Scientometrics (in the press) and at; 2014).

For example, the index ranks the Chinese Academy of Sciences above Harvard University in Cambridge, Massachusetts. Using the complete publication output from each of these in 2013 (31,428 and 17,836 articles, respectively, according to Thomson Reuters InCites), we calculate that only 8% (2,661) of papers from the Chinese Academy of Sciences contributed to the Nature Index, whereas 14% (2,555 papers) of Harvard papers contributed. This relative perspective is important, given that an institution with a high publication output would be expected to publish more papers in reputable journals.

In our view, it would also be better to measure the performance of countries and institutions on the basis of individual papers, rather than on the journals in which they are published (see This is because the quality of a journal (as measured by peers or citations) is not a reliable proxy for the quality of each paper it publishes.

To illustrate this point, we counted citations over 5 years for papers published in 2008 in Applied Physics Letters — the journal contributing most articles to the Nature Index — using an in-house bibliometrics database maintained by the Max Planck Digital Library in Munich, Germany. We found that some 40% of the papers in Applied Physics Letters accounted for about 80% of the citations to this journal, suggesting an uneven distribution of quality.


Editorial note: For details of Nature Index methodology, see Nature 515, S52–S53 (2014). A response from Nature Publishing Group to R. Haunschild and L. Bornmann's forthcoming Scientometrics article is available at (2014).

Author information


  1. Max Planck Institute for Solid State Research, Stuttgart, Germany.

    • Robin Haunschild
  2. Max Planck Society, Munich, Germany.

    • Lutz Bornmann

Corresponding author

Correspondence to:

Author details

Additional data