Impact factors: Is the Nature Index at odds with DORA?

Journal name:
Nature
Volume:
545,
Page:
412
Date published:
DOI:
doi:10.1038/545412a
Published online

We find Nature Research's critical attitude towards journal impact factors, embodied in its signing of the San Francisco Declaration on Research Assessment (DORA; Nature 544, 394; 2017), to be inconsistent with the aims of its Nature Index.

The Nature Index provides statistics on the publication output of institutions and countries. These statistics are collated from “high-quality research” published in an independently selected set of 68 “high-quality science journals” (see www.natureindex.com/faq). The data are presented as metrics that can be used to assess “research excellence and institutional performance”.

This seems to us to be in violation of DORA's principles, which state that research should be assessed “on its own merits rather than on the basis of the journal in which the research is published” (www.ascb.org/dora).

Even though we disagree with some of the ideas underlying DORA (see L. Waltman and V. A. Traag, preprint at arxiv.org/abs/1703.02334; 2017), we believe that Nature Research should practice what it preaches and abandon the Nature Index.

On behalf of the Nature Index, David Swinbanks replies — The Nature Index provides an indicator of an institution's contributions to high-quality research, on the basis of papers from a suite of journals that exhibit high editorial standards and have been independently chosen by a committee of active researchers — without regard to impact factor (see go.nature.com/2qo53pj). The Nature Index enables the tracking of institutional research output and collaboration at the city, state and country levels.

It is therefore not in violation of DORA, which champions appropriate ways of appraising individual researchers and opposes the abuse of the journal impact factor as a metric for research assessment.

Author information

Affiliations

  1. Leiden University, the Netherlands.

    • Ludo Waltman &
    • Vincent Traag

Corresponding author

Correspondence to:

Author details

Comments

  1. Report this comment #70197

    Bernd Pulverer said:

    In my view there is an issue.
    David notes the select group of journals assembled for the Nature Index was chosen without any recourse to Journal Impact Factor (JIF), and the list indeed includes journals with a range of JIFs. However, DORA went further to also state 'assess research on its own merits rather than on the basis of the journal in which the research is published' (http://www.ascb.org/dora/). This statement recognizes that while it is certainly meaningful to publish in a broadly read journal with high quality editorial processes and standards, journals ultimately select for editorial criteria and that these are not necessarily identical to the criteria applied by research assessment exercises.
    Publishing in a journal on the Nature Index list can be seen as one sign of quality, it cannot be seen as the only indicator. In other words, not publishing in a journal on the list should not be seen as being a sign of poor quality.
    In general, the same argument applies for research assessment at the level of an individual researcher, a research institution and even a country. To be fair, the bigger the group of papers analyzed, the more the data publication venues will even out and the more informative the data becomes.

    (for transparency: I am chief editor of The EMBO Journal, which is listed on the Nature Index)

Subscribe to comments

Additional data