Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Utilizing a composite citation index for evaluating clinical ophthalmology research: insights into gender, nationality, and self-citation among top ophthalmology researchers

Eye (2024)Cite this article

Subjects

Abstract

Objectives

To compare the performance of a composite citation score (c-score) and its six constituent citation indices, including H-index, in predicting winners of the Weisenfeld Award in ophthalmologic research. Secondary objectives were to explore career and demographic characteristics of the most highly cited researchers in ophthalmology.

Methods

A publicly available database was accessed to compile a set of top researchers in the field of clinical ophthalmology and optometry based on Scopus data from 1996 to 2021. Each citation index was used to construct a multivariable model adjusted for author demographic characteristics. Using area under the receiver operating curve (AUC) analysis, each index’s model was evaluated for its ability to predict winners of the Weisenfeld Award in Ophthalmology, a research distinction presented by the Association for Research in Vision and Ophthalmology (ARVO). Secondary analyses investigated authors’ self-citation rates, career length, gender, and country affiliation over time.

Results

Approximately one thousand unique authors publishing primarily in clinical ophthalmology/optometry were analyzed. The c-score outperformed all other citation indices at predicting Weisenfeld Awardees, with an AUC of 0.99 (95% CI: 0.97–1.0). The H-index had an AUC of 0.89 (95% CI: 0.83–0.96). Authors with higher c-scores tended to have longer career lengths and similar self-citation rates compared to other authors. Sixteen percent of authors in the database were identified as female, and 64% were affiliated with the United States of America.

Conclusion

The c-score is an effective metric for assessing research impact in ophthalmology, as seen through its ability to predict Weisenfeld Awardees.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Correlogram of Pearson correlation coefficients between various citation indices.
Fig. 2: Number of Weisenfeld awardees captured by rankings based on different citation indices.
Fig. 3: Receiver operating characteristic (ROC) curves for citation indices used to predict Weisenfeld Awardees.

Similar content being viewed by others

Data availability

All data used in this project are publicly available in the “Updated science-wide author databases of standardized citation indicators” found at the following website: https://elsevier.digitalcommonsdata.com/datasets/btchxktzyw/6.

References

  1. Abbot A, Cyranoski D, Jones N. Metrics: do metrics matter? Nature. 2010;465:860–2. https://doi.org/10.1038/465860a.

    Article  CAS  Google Scholar 

  2. Holden G, Rosenberg G, Barker K. Bibliometrics: a potential decision making aid in hiring, reappointment, tenure and promotion decisions. Soc Work Health Care. 2005;41:67–92. https://doi.org/10.1300/J010v41n03_03.

    Article  PubMed  Google Scholar 

  3. Haustein S, Larivière V. The use of bibliometrics for assessing research: possibilities, limitations and adverse effects. In: Welpe, I, Wollersheim, J, Ringelhan, S, Osterloh, M, eds. Incentives and performance.Cham: Springer; 2015. https://doi.org/10.1007/978-3-319-09785-5_8.

  4. Tanya SM, He B, Tang J, He P, Zhang A, Sharma E, et al. Research productivity and impact of Canadian academic ophthalmologists: trends in H-index, gender, subspecialty, and faculty appointment. Can J Ophthalmol. 2022;57:188–94. https://doi.org/10.1016/j.jcjo.2021.03.011.

    Article  PubMed  Google Scholar 

  5. Lopez SA, Svider PF, Misra P, Bhagat N, Langer PD, Eloy JA. Gender differences in promotion and scholarly impact: an analysis of 1460 academic ophthalmologists. J Surg Educ. 2014;71:851–9. https://doi.org/10.1016/j.jsurg.2014.03.015.

    Article  PubMed  Google Scholar 

  6. Huang G, Fang CH, Lopez SA, Bhagat N, Langer PD, Eloy JA. Impact of fellowship training on research productivity in academic ophthalmology. J Surg Educ. 2015;72:410–7. https://doi.org/10.1016/j.jsurg.2014.10.010.

    Article  PubMed  Google Scholar 

  7. Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA. 2005;102:16569–72. https://doi.org/10.1073/pnas.0507655102.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Koltun V, Hafner D. The h-index is no longer an effective correlate of scientific reputation. PLOS ONE. 2021;16:e0253397. https://doi.org/10.1371/journal.pone.0253397.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Schreiber M. A modification of the h-index: the hm-index accounts for multi-authored manuscripts. J Informetr. 2008;2:211–6. https://doi.org/10.1016/j.joi.2008.05.001.

    Article  Google Scholar 

  10. Ioannidis JPA, Klavans R, Boyack KW. Multiple citation indicators and their composite across scientific disciplines. PLOS Biol. 2016;14:e1002501. https://doi.org/10.1371/journal.pbio.1002501.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. ARVO Achievement Awards. https://www.arvo.org/awards-grants-and-fellowships/arvo-achievement-awards/.

  12. Ioannidis JPA, Boyack KW, Baas J. Updated science-wide author databases of standardized citation indicators. PLOS Biol. 2020;18:e3000918. https://doi.org/10.1371/journal.pbio.3000918.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinforma. 2011;12:77. https://doi.org/10.1186/1471-2105-12-77.

    Article  Google Scholar 

  14. Nielsen MW, Andersen JP, Schiebinger L, Schneider JW. One and a half million medical papers reveal a link between author gender and attention to gender and sex analysis. Nat Hum Behav. 2017;1:791–6. https://doi.org/10.1038/s41562-017-0235-x.

    Article  PubMed  Google Scholar 

  15. Nguyen AX, Ratan S, Biyani A, Trinh XV, Saleh S, Sun Y, et al. Gender of award recipients in major ophthalmology societies. Am J Ophthalmol. 2021;231:120–33. https://doi.org/10.1016/j.ajo.2021.05.021.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Sinatra R, Wang D, Deville P, Song C, Barabási AL. Quantifying the evolution of individual scientific impact. Science. 2016;354:aaf5239. https://doi.org/10.1126/science.aaf5239.

    Article  CAS  PubMed  Google Scholar 

  17. Szomszor M, Pendlebury DA, Adams J. How much is too much? The difference between research influence and self-citation excess. Scientometrics. 2020;123:1119–47. https://doi.org/10.1007/s11192-020-03417-5.

    Article  Google Scholar 

  18. Reddy AK, Bounds GW, Bakri SJ, Gordon LK, Smith JR, Haller JA, et al. Representation of women with industry ties in ophthalmology. JAMA Ophthalmol. 2016;134:636. https://doi.org/10.1001/jamaophthalmol.2016.0552.

    Article  PubMed  Google Scholar 

  19. Mimouni M, Zayit-Soudry S, Segal O, Barak Y, Nemet AY, Shulman S, et al. Trends in authorship of articles in major ophthalmology journals by gender, 2002–2014. Ophthalmology. 2016;123:1824–8. https://doi.org/10.1016/j.ophtha.2016.04.034.

    Article  PubMed  Google Scholar 

  20. Santamaría L, Mihaljević H. Comparison and benchmark of name-to-gender inference services. PeerJ Comput Sci. 2018;4:e156. https://doi.org/10.7717/peerj-cs.156.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Yang K, Meho LI. Citation analysis: a comparison of Google Scholar, Scopus, and Web of Science. Proc Am Soc Inf Sci Technol. 2006;43:1–15. https://doi.org/10.1002/meet.14504301185.

    Article  Google Scholar 

  22. Allen P. World-Wide News Coverage in PlumX. 2017. https://plumanalytics.com/world-wide-news-coverage-plumx/.

  23. Meschede C, Siebenlist T. Cross-metric compatability and inconsistencies of altmetrics. Scientometrics. 2018;115:283–97. https://doi.org/10.1007/s11192-018-2674-1.

    Article  Google Scholar 

  24. How Outputs Are Tracked and Measured. Altmetric https://help.altmetric.com/support/solutions/articles/6000234171-how-outputs-are-tracked-and-measured.

Download references

Author information

Authors and Affiliations

  1. Johns Hopkins University School of Medicine, Baltimore, MD, USA

    Annabelle A. Pan & Alex T. Pham

  2. Department of Ophthalmology, Wilford Hall Eye Center, San Antonio, TX, USA

    Ben Appelo

  3. Department of Surgery, Uniformed Services University of the Health Sciences, Bethesda, MD, USA

    Gary L. Legault & Grant A. Justin

  4. Wilmer Eye Institute, Baltimore, MD, USA

    Fasika A. Woreta

  5. Department of Ophthalmology, Walter Reed National Military Medical Center, Bethesda, MD, USA

    Grant A. Justin

Authors
  1. Annabelle A. Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alex T. Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ben Appelo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gary L. Legault
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fasika A. Woreta
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Grant A. Justin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

GJ, AAP, and FW contributed to the conception and design of the work. AAP and ATP contributed to extracting and analyzing the data and drafting initial sections of the paper. BA, GJ, GL, and FW provided feedback during manuscript revision and approved of the submitted version.

Corresponding author

Correspondence to Annabelle A. Pan.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pan, A.A., Pham, A.T., Appelo, B. et al. Utilizing a composite citation index for evaluating clinical ophthalmology research: insights into gender, nationality, and self-citation among top ophthalmology researchers. Eye (2024). https://doi.org/10.1038/s41433-023-02912-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41433-023-02912-2

Search

Advanced search

Quick links