Abstract
One disadvantage of the JCR impact factor, the most commonly used assessment tool for ranking and evaluating scientific journals, is its inability in distinguishing among different shapes of citation distribution curves, leading to unfair evaluation of journals in some cases. This paper aims to put forward an alternative impact factor (IF′) that can properly reflect citation distributions. The two impact factors are linearly and positively correlated, and have roughly the same order of magnitude. Because of the ability of IF′ in distinguishing among different shapes of citation distribution curves, IF′ may properly reflect the academic performance of a scientific journal in a way that is different from the JCR impact factor with some unique features that reward journals with highly cited papers. Therefore, it is suggested that IF′ could be used to complement the JCR impact factor.
Introduction
The concept of the impact factor was proposed by Garfield^{1} about 60 years ago. Since then, the impact factor has become the most commonly used assessment tool for ranking and evaluating scientific journals. However, the impact factor suffers from a number of shortcomings and limitations^{2,3,4,5}. These shortcomings include: (i) Because the JCR impact factor is only an average citation received, it does not possess the ability to discriminate the shapes of citation distribution curves, leading to cases in which the same impact factor may correspond to different journals with different citation distributions. (ii) For the same reason, the JCR impact factor as an index of citations per publication usually rewards low productivity, and penalizes high productivity^{6}. (iii) The JCR impact factor is relatively easily manipulated by increasing selfcitations. On the other hand, Hirsch launched a new study direction of scientometrics by proposing a novel index, now known as the hindex^{6}. The hindex was quickly adopted in various research areas. For example, the hindex or htype indices were used for evaluating physicists^{7}, evaluating the 100 most prolific economists^{8}, and for evaluating chemical research groups correlated with peer judgment^{9}. Recent studies of the hindex were reviewed^{10,11,12,13}.
Meanwhile, a number of researchers explored the possibility of using the hindex to complement or correct the traditional impact factor. Indeed, the hindex or htype indices were used to complement journal impact factors or to rank scientific journals. For example, Braun, Glänzel, & Schubert suggested that the htype index may be a useful complement to journal impact factors^{14}. Schubert and Glänzel performed a systematic analysis of htype indices for journals^{15}. A theoretical model of the dependence of htype indices on the number of publications and the average citation rate was tested successfully on empirical samples of journal hindices^{15}. Vanclay ranked forestry journals using the hindex^{16}. Relationship between the hindex and impact factor in the powerlaw model was studied by Egghe, Liang, & Rousseau^{17}.
Despite these beneficial exploring researches, the hindex suffers from a number of inherent shortcomings. Probably, the most notable shortcoming of the hindex is that only hsquared citations can be inferred from the hindex, which completely ignores excess citations^{18} and htail citations^{19,20}. The hindex by itself does not carry information for excess and htail citations, which can play an even more dominant role than the hindex in determining the shape of citation distribution curve. Ignoring the contributions from the excess and htail citations usually either underestimates or overestimates the academic performance of the scientist or the journal under study. To solve this problem, recently a new htype index was proposed, called the h′index^{21}. One of the main merits of the h′index is that it takes hsquared, excess and tail citations all into account. The h′index possesses the ability to distinguish between different shapes of the citation distribution curves. The proposal of the h′index inspired us to explore the possibility of using the h′index to complement and correct the traditional impact factor, and this is the aim of the present paper.
Results
Derivation of an alternative impact factor, IF′
To complement the hindex^{6}, the eindex^{18} and tindex^{20} were proposed. The eindex is the square root of the excess citations over h^{2} in the hcore. The tindex is the square root of the htail citations^{19}. Therefore, the area under the citation distribution curve is divided by the hindex into three parts, representing the h^{2}, the excess citations (e^{2}) and htail citations (t^{2}), respectively. To capture the main shape of the citation distribution curve, the headtail ratio, denoted by r, was defined^{21} The three cases of r > 1, r = 1 and r < 1 correspond to three types of the citation distribution functions. The shapes of citation distribution functions for r > 1 are peaked, and for r < 1 the shapes of the citation functions are flat with a long tail, whereas for r = 1 the citation functions are roughly symmetrical with respect to the diagonal line of the coordinate system. A number of authors attempted to apply the hindex to complement or correct the JCR impact factor^{14,15,16}. It should be pointed out that we cannot simply use the hindex to complement or correct the JCR impact factor. This is due to the fact that the hindex by itself does not carry information for the excess and htail citations, which can play an even more dominant role than the hindex in determining the shape of citation distribution curve. As pointed out previously^{21}, when r > 1, especially r ≫ 1, the hindex underestimates the academic performance of a scientific journal,whereas when r < 1, especially r ≪ 1, the hindex overestimates the academic performance of a scientific journal. When r = 1, the hindex properly reflects the academic performance of a scientific journal.
To provide an alternative evaluation of the academic performance of a scientific journal, we propose a novel impact factor, denoted by IF′, to complement the JCR impact factor, defined by where h′ is the h′index^{21}.
The citations received by all papers in the hcore, denoted by C_{hcore}, are where cit_{j} are the citations received by the j^{th} paper. Letting e^{2} denote the excess citations within the hcore, we find^{18} where R is the Rindex^{22}. So, Meanwhile, the tindex was defined by^{20} where C_{total} is the number of total citations received by all papers published. Finally, we have and Based on the WoS database, all the published items for the 227 journals of biochemistry and molecular biology in 2009 and 2010 were retrieved. The retrieval resulted in 111,002 records, which were downloaded and saved as Excel files. Finally, we calculated the hindex, eindex and tindex of the 227 journals according to the citation data derived from the 111,002 records. Of note, all the calculations of the hindex, eindex and tindex were within a time window from the publication year of items to 2011. The parameter C_{total} is the total citations received by all items published in 2009 and 2010 during the time window from the publication year of items to 2011. The parameter C_{hcore} is the citations received by all items in the hcore of the citation distribution curve. Then IF′ can be calculated using eqs. (2) or (8). The JCR impact factor for the year 2011 is the ratio of the total citations received in 2011 by all items published in 2009 and 2010 over the total numbers of citable items published in 2009 and 2010^{23}.
Correlation between IF′ and the JCR impact factor
Based on the data derived from the JCR and WoS, the related parameters including the total citations C_{total}, the hindex, eindex, tindex, the headtail ration r and IF′ are calculated for each of the 227 journals in the category of biochemistry and molecular biology. To save printing space, only the front ten journals in the alphabetic order are listed in Table 1. The related data for all the 227 journals are listed in the Appendix1. As we can see from Table 1 that the headtail ratio r for nine of the ten journals listed is all less than 1. Indeed, for almost all of the 227 journals listed in the Appendix1, the values of r are less than 1. To study the relation between the two impact factors, the correlation between IF′ and the JCR impact factor is shown in Fig. 1. It is seen that IF′ is highly linearly and positively correlated with the JCR impact factor, as reflected by the fact that the correlation coefficient is as high as 0.89. Of note, IF′ and the JCR impact factor have roughly the same order of magnitude. The above two features make IF′ relatively easy to be accepted by the academic community.
Applications of IF′
As an application, we show here the rankings of the 227 journals in the category of biochemistry and molecular biology based on IF′. For comparison, rankings are also provided based on traditional JCR impact factor. To save printing space, only the top ten journals are listed in Table 3. The rankings for all the 227 journals are listed in the Appendix2. As we can see from Table 3 that the two rankings overlapped in 8 of the top 10 journals (80%), except that orders of some journals were different. This fact indicates that IF′ is basically consistent with the JCR impact factor, confirming the observation in Fig. 1. The famous journal Cell is clearly ranked No. 1 in both rankings. Of note, the journal Nucleic Acids Research is ranked No. 15 in the rankings based on the JCR impact factor, whereas it is ranked No. 3 in that based on IF′. On the contrary, the EMBO Journal is ranked No.10 in the JCR ranking, whereas it is ranked No. 24 in that based on IF′. Furthermore, the journal Current Biology is ranked No. 8 in the JCRrankings, whereas it is ranked No. 40 in the rankings based on IF′. The variations in both ranks are determined mainly by their shapes of citation distribution curves. Of note, according to the data listed in the Appendeix1, the headtail ratios for the journals Nucleic Acids Research, EMBO Journal and Current Biology are 0.597, 0.245 and 0.231, respectively. It is obvious that the shape of the citation distribution curve for the journal Nucleic Acids Research is relatively sharp, whereas those for the journals EMBO Journal and Current Biology are relatively flat with a longer tail.
In the analysis above, we show that with these journals, IF′ has a onetoone correspondence, whereas the JCR impact factor has a onetomultiple correspondence. This is due to the fact that the JCR impact factor does not possess the ability to discriminate the shapes of the citation distribution functions, whereas the shapes of the citation distribution curves are properly taken into account by IF′. As a consequence, the JCR impact factor as an index of citations per publication usually rewards low productivity, and penalizes high productivity^{6}.
Discussion
The JCR impact factor is the averaged number of citations received per publication. As pointed out by Hirsch^{6}, measuring the average citations per publication, as does the JCR impact factor, usually rewards low productivity, and penalizes high productivity. This is due to the fact that such indices do not possess the ability to distinguish between different shapes of citation distribution curves. To illustrate this point, let us consider two journals among the 227 journals. The first one is the Journal of Physiology and Biochemistry (JPB) and the second one is the Journal of Liposome Research (JLR). The JCR impact factor and the number of publications of both journals are roughly equal. The related data of the two journals are listed in Table 2. Their citation distribution curves are shown in Fig. 2. Note that the shapes of the two curves are different, as clearly shown in Fig. 2. The shape of the citation distribution curve for JPB is relatively sharp, whereas that for JLR is relatively flat. Unfortunately, the JCR impact factor cannot distinguish between the two cases, resulting in almost identical impact factor values (1.711 vs. 1.707). On the contrary, IF′ is able to distinguish between the academic performance of the two journals,resulting in the values of IF′ (3.658 vs.1.867).This example shows that IF′ results in distinct values for different journal citation distributions, whereas different citation distribution curves may correspond to the same JCR impact factor value. In other words, IF′ and journals have a relation of onetoone correspondence, whereas the JCR impact factor and journals have a relation of onetomultiple correspondence. Table 2 shows that in addition to the numbers of publication, the total citations, the average citations per publication (i.e., the JCR impact factor) and the hindices are all roughly equal to each other for the two journals. Of note, the four indices mentioned above are all the singlenumber evaluation indices currently available and widely used nowadays. Unfortunately, none of them is capable of discriminating the academic performance of the two journals. On the contrary, IF′ (or the h′index) is a singlenumber evaluation index that is able to distinguish between the academic performance of journals with different citation distribution. In addition to the two journals mentioned above, other three similar pairs of journals are also listed in Table 2.
Of note, the example above is a typical case, rather than a rare event, because we can find a number of similar cases, as shown in Table 2. Let us further consider two or more journals with roughly equal JCR impact factors, but the numbers of publication may be different. Such cases occur more frequently than the example shown above. Although their JCR impact factors are roughly equal, their citation distribution curves may be different, even quite different. The traditional JCR impact factor lacks the ability to discriminate different shapes of the citation distribution curves, whereas IF′ proposed in this paper does not have this drawback. In this regard, IF′ is an alternative to the JCR impact factor with unique features that reward journals with highlycited papers, but disregard lowlycited papers. We need to point out that whether journals with such peaked citation distribution are preferable to those having uniform distribution is arguable. However, current studies on the citation distributions show that the citation distributions obey the socalled power law^{24,25,26}, stretched exponential^{24,27,28}, lognormal^{24,29,30,31} and modified Bessel function^{24,32}. According to the parameters involved in these functions, the citation distributions are usually skewed, rather than uniform.
In summary, IF′ is proposed as an alternative to the JCR impact factor with some unique features. (i) IF′ is designed to possess the ability to distinguish between different shapes of the citation distribution curves. It gives larger values of IF′ to those with relatively sharp citation distribution curves, whereas lower values to those with flat ones. This is in agreement with the common point of view of academic evaluation^{33,34}. (ii) Citation information of three years is considered by IF′, rather than 2 years for JCR impact factor, and therefore IF′ carries more citation information. (iii) IF′ is basically consistent with the JCR impact factor, and the two impact factors have roughly the same order of magnitude. These features make IF′ relatively easy to be accepted by the academic community. IF′ may properly reflect the academic performance of a scientific journal in a way that is different from the JCR impact factor with some unique features that reward journals with highly cited papers, and therefore it could be used to complement the JCR impact factor.
Methods
As an example, the journals in the category of biochemistry and molecular biology were studied. According to the Thomson Reuters Journal Citation Reports (JCR) Science Edition 2011, there are 290 journals listed in the subject category BIOCHEMISTRY & MOLECULAR BIOLOGY. Among these 290 journals, a few of them changed their titles during the period of 2006 to 2010. Some journals published fewer than 100 articles, whereas some were not continuously indexed by the Web of Science database (WoS) during the period of 2006–2010. All the journals mentioned above were excluded from the current study. Consequently, 227 journals were remained for the present study. We downloaded the Journal Summary List of the subject category BIOCHEMISTRY & MOLECULAR BIOLOGY in the JCR Science Edition 2011 and extracted the Impact Factor values of the 227 journals. The detailed titles and related data for the 227 journals are listed in the Appendix1.
References
 1.
Garfield, E. Citation indexes to science: a new dimension in documentation through association of ideas. Science. 122, 108–111 (1955).
 2.
Hecht, F., Hecht, B. & Sandberg, A. The journal impact factor: A misnamed, misleading, misused measure. Cancer Genet. Cytogenet. 104, 77–81 (1998).
 3.
Dong, P., Loh, M. & Mondry, A. The “impact factor” revisited. Biomed Digit Libr. 2, 7. 10.1186/1742–558127. (2005).
 4.
Curtis, W. & Hunter, J. What the impact factor means for surgery journals. World J. Surg. 30, 1368–1370 (2006).
 5.
Dellavalle, R., Schilling, L., Rodriguez, M., Van De Sompel, H. & Bollen, J. Refining dermatology journal impact factors using pagerank. J. Am. Acad. Dermatol. 57, 116–119 (2007).
 6.
Hirsch, J. E. An index to quantify an individual's scientific research output. Proc. Natl. Acad. Sci. U. S. A. 102, 16569–16572 (2005).
 7.
Schreiber, M. A case study of the Hirsch index for 26 nonprominent physicists. Ann. Phys.Berlin. 16, 640–652 (2007).
 8.
Tol, R. S. J. The hindex and its alternatives: An application to the 100 most prolific economists. Scientometrics. 80, 317–324 (2009).
 9.
Van Raan, A. F. J. Comparison of the Hirschindex with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics. 67, 491–502 (2006).
 10.
Bornmann, L. & Daniel, H.D. The state of h index research: Is the h index the ideal way to measure research performance? EMBO Rep. 10, 2–6 (2009).
 11.
Egghe, L. The Hirsch index and related impact measures. Annu. Rev. Inform. Sci. Technol. 44, 65–114 (2010).
 12.
Zhang, L., Thijs, B. & Glänzel, W. The diffusion of Hrelated literature. J. Informetr. 5, 583–593 (2011).
 13.
Rousseau, R., GarciaZorita, C. & SanzCasado, E. The hbubble. J. Informetr. 7, 294–300 (2013).
 14.
Braun, T., Glänzel, W. & Schubert, A. A Hirschtype index for journals. Scientometrics. 69, 169–173 (2006).
 15.
Schubert, A. & Glänzel, W. A systematic analysis of Hirschtype indices for journals. J. Informetr. 1, 179–184 (2007).
 16.
Vanclay, J. Ranking forestry journals using the hindex. J. Informetr. 2, 326–334 (2008).
 17.
Egghe, L., Liang, L. & Rousseau, R. A relation between hindex and impact factor in the powerlaw model. J. Am. Soc. Inf. Sci. Technol. 60, 2362–2365 (2009).
 18.
Zhang, C.T. The eindex, complementing the hindex for excess citations. PLoS One. 4, e5429 (2009).
 19.
Ye, F. Y. & Rousseau, R. Probing the hcore: an investigation of the tailcore ratio for rank distributions. Scientometrics. 84, 431–439 (2010).
 20.
Zhang, C.T. A novel triangle mapping technique to study the hindex based citation distribution. Sci Rep. 3, 1023 (2013).
 21.
Zhang, C.T. The h′index, effectively improving the hindex based on the citation distribution. PLoS One. 8, e59912 (2013).
 22.
Jin, B.H., Liang, L.M., Rousseau, R. & Egghe, L. The R and ARindices: Complementing the hindex. Chin. Sci. Bull. 5, 855–863 (2007).
 23.
Thomson Reuters. Journal citation reports help. http://adminapps.webofknowledge.com/JCR/help/h_impfact.htm#impact_factor (20120522).
 24.
Radicchi, F. & Castellano, C. A Reverse Engineering Approach to the Suppression of Citation Biases Reveals Universal Properties of Citation Distributions. PLoS One. 7, e33833. 10.1371/journal.pone.0033833 (2012).
 25.
Redner, S. How popular is your paper? An empirical study of the citation distribution. Eur. Phys. J. B. 4, 131–134 (1998).
 26.
Seglen Per, O. The Skewness of Science. J. Am. Soc. Inf. Sci. 43, 628–638 (1992).
 27.
Laherrére, J. & Sornette, D. Stretched exponential distributions in nature and economy: “fat tails” with characteristic scales. Eur. Phys. J. B. 2, 525–539 (1998).
 28.
Wallace, M. L., Larivière, V. & Gingras, Y. Modeling a century of citation distributions. J. Informetr. 3, 296–303 (2009).
 29.
Stringer, M. J., SalesPardo, M. & Amaral, L. A. N. Effectiveness of journal ranking schemes as a tool for locating information. PLoS One. 3, e1683. 10.1371/journal.pone.0001683 (2008).
 30.
Radicchi, F., Fortunato, S. & Castellano, C. Universality of citation distributions: Toward an objective measure of scientific impact. Proc. Natl. Acad. Sci. U. S. A. 105, 17268–17272 (2008).
 31.
Stringer, M. J., SalesPardo, M. & Amaral, L. A. N. Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal. J. Am. Soc. Inf. Sci. Technol. 61, 1377–1385 (2010).
 32.
van Raan, A. F. J. Twostep competition process leads to quasi powerlaw income distributions. Application to scientific publication and citation distributions. Physica A. 298, 530–536 (2001).
 33.
Cole, S. & Cole, J. R. Scientific Output and Recognition  Study in Operation of Reward System in Science. Am. Sociol. Rev. 32, 377–390 (1967).
 34.
Bornmann, L., Mutz, R. & Daniel, H.D. The h index research output measurement: Two approaches to enhance its accuracy. J. Informetr. 4, 407–414 (2010).
Acknowledgements
We thank Dr. F Gao for the help in preparing Fig. 1 and Fig. 2. The present study was supported in part by the Research Foundation of the ISTICThomson Reuters Joint Lab for Scientometrics Research (No. IT2010014) and National Social Science Foundation of China (No. 11BTQ022), both to ZGY.
Author information
Affiliations
Library, Tianjin University, Tianjin 300072, China
 ZuGuo Yang
Department of Physics, Tianjin University, Tianjin 300072, China
 ChunTing Zhang
Authors
Search for ZuGuo Yang in:
Search for ChunTing Zhang in:
Contributions
Conceived and designed the experiments: C.T.Z. Performed the experiments: Z.G.Y. Analyzed the data: C.T.Z. and Z.G.Y. Wrote the paper: C.T.Z.
Competing interests
The authors declare no competing financial interests.
Corresponding author
Correspondence to ChunTing Zhang.
Supplementary information
PDF files
 1.
Supplementary Information
Journal title, JCR impact factor and IF'
Rights and permissions
This work is licensed under a Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/
About this article
Further reading

Analytical mapping of research on disaster management, types and role of ICT during 2011–2018
Environmental Hazards (2019)

Navigating manuscript assessment: The new practitioner’s guide to primary literature peer review
Journal of Oncology Pharmacy Practice (2019)

The international journal of production research in the past, the present and the future: a bibliometric analysis
International Journal of Production Research (2019)

Exploring the HjifIndex, an Analogue to the HLike Index for Journal Impact Factors
Publications (2018)

Bibliometric Indices of Scientific Journals: Time to overcome the obsession and think beyond the Impact Factor
Medical Journal Armed Forces India (2017)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.