Social media services, especially Twitter, are used as a commonly sharing tool in the scientific world. This widespread use of Twitter would be an effective method in spreading academic publications. So, we aimed to investigate the relationship between Twitter mentions and traditional citations of articles in sexual medicine journals in this study. We reviewed the articles published in seven journals of sexual medicine (2 years after the publication of the articles) between January 2018 and June 2018. In the first half of 2018, 410 articles were extracted. Of these, 352 (85.9%) were original articles, while 58 (14.1%) were review articles. The median number of citations of the articles mentioned at least once on Twitter was 7 (interquartile range: 0–111) for Google Scholar, whereas it was 0 (interquartile range: 0–63) for Scopus, respectively. It was 4 (interquartile range: 0–25) for Google Scholar and 0 (interquartile range: 0–7) for Scopus. The publications mentioned on Twitter were cited more than the non-mentioned publications in the traditional-based citation system (p < 0.001). A significant relationship between the citation numbers and tweet numbers was also observed (p < 0.001). Also, in the linear regression model, the tweet numbers (p < 0.001) and article types (p < 0.001) were found to be related to the Google Scholar citation numbers. In conclusion, using Twitter as a professional tool in academic life would allow information to be propagated and responded quickly, especially for sexual medicine journals.
The ways people interact and communicate worldwide have changed drastically with social media (SoMe) . This change was not only limited to social life but also appeared in the scientific world . The scientific role of SoMe in healthcare represents a significant area in medicine [3, 4].
Scientific data sharing has accelerated with the increasing use of SoMe in the academic community. In this way, it has facilitated to reach of wider audiences . The use of SoMe as a resource in evaluating the impact levels of scientific research increases due to its contribution to knowledge sharing . In order to measure the scientific impact of SoMe, new metric evaluation systems have been recently introduced such as Altmetric and PlumX metric to determine the online attention from SoMe services such as Facebook, Twitter, blogs, citations on Wikipedia and in public policy documents, discussions on research blogs, mainstream media coverage, bookmarks on reference managers like Mendeley, etc[7, 8].. Some publishers such as Elsevier, Nature Publishing Group, Frontiers have begun to provide readers with alternative measurement information using these different sources . These metrics systems provide qualitative data that are considered complementary to traditional, citation-based metrics .
Twitter is a microblogging service where text-based messages are shared . It has around 230 million users who post almost 500 million tweets per day, and its usage increases day by day. This increase is observed not only in social content sharing but also in scientific content sharing. Many academicians share scientific articles here and enable them to reach more people. Many urologists have already adapted to this effect . It is shown that approximately one-third of the American Urological Association (AUA) members have a Twitter account [10, 11]. This widespread use of Twitter would be an effective method in spreading academic publications, especially for sexual medicine journals.
Therefore, we aimed to investigate the relationship between Twitter mentions and traditional citations of articles in sexual medicine journals.
We retrospectively reviewed articles published in ten sexual medicine journals based on Clarivate Analytics and Scimago Journal & Country Rank (cited in 2019). The impact factor (IF) of these Journals according to Journal Citation Reports in 2019 was ≥1, and the Q index was Q1–Q3. Journals with IF < 1 IF and/or Q4 were excluded from the present study. Journal of Sexual Medicine (Q1, IF: 3.293), Andrology (Q1, IF: 2.860), Basic and Clinical Andrology (Q2, IF: 2.600), The World Journal of Men’s Health (Q3; IF:2.547), Asian Journal of Andrology (Q2, IF:2.448), Translational Andrology and Urology (Q2, IF: 2.445), Andrologia (Q2, IF:1.951), Sexual Medicine (Q3, IF: 1.923), Sexual Medicine Reviews (Q1, IF: 1.807), International Journal of Impotence Research (IJIR) (Q3, IF: 1.388), were selected.
Of these, journals providing Altmetric or PlumX metric measurements for their articles were included in the study. Altmetric and PlumX metrics are the Digital Science companies utilizing diverse resources such as academic platforms, citations, forums, web-based platforms, social media, Wikipedia articles, etc., to ensure an alternative metric score to the traditional citation system . The sharing number on SoMe platforms, number of references, and frequency of clicks contribute to this metric score algorithm. The higher quality sources provide greater weight, while exposure, less respectable or less known sources take up lower weight . Thus, an alternative metric attention score to the traditional citation is obtained. Based on obtained findings, these metric systems aim to evaluate the popularity of the research output . Moreover, metric scores obtained may be considered as an indicator of the scientific effect of published articles. Various companies use both two platforms for a similar purpose. Altmetric score is used by some publishers such Springer Nature, John Wiley & Sons, Taylor and Francis on their publisher platforms [12, 14], while the PlumX metric is used by Elsevier products (Mendeley, Scopus, etc.) .
The number of tweets that articles took in some sexual medicine journals was achieved by Altmetric, while that of in other journals was attained via PlumX metric. The articles in the Journal of Sexual Medicine, Sexual Medicine Reviews, and Sexual Medicine were evaluated by the PlumX metric. Besides, the Altmetric evaluation system was used for articles in Andrology, Andrologia, IJIR, and Basic and Clinical Andrology journals. The Asian Journal of Andrology was excluded from the study because it does not provide any metric measure for Twitter mentions. Translational Andrology and Urology and The World Journal of Men’s Health were excluded from the study because they have publications on general urology in addition to sexual medicine. Original articles and reviews were included for article selection criteria, while editorial comments, case reports, expert opinions, conference summaries, and letters to editors were excluded. Besides, articles outside of the specified time frame and non-English articles were also excluded.
A total of seven sexual medicine journals were included in the study before analysis. The past issues, from January 2018 to June 2018, were checked through the journals’ web pages, and the article titles were searched in Scopus and Google Scholar. The date range was considered based on the publication dates (not online or early view) of the articles in the journal. The data were evaluated, on average, 2 years after the articles were published. It was thought that this process would allow the impact of published articles fully observed. Two authors performed the search, and the Altmetric and PlumX metric data of the articles, the number of Twitter mentions, and the traditional citation numbers were recorded. The search bars “https://www.altmetric.com/products/free-tools/bookmarklet/” and “https://plumanalytics.com/learn/about-metrics/” were used to obtain these data.
Statistical Package for Social Sciences volume 22.0 (IBM Corp., IL, Chicago, USA) program was used to analyze the data. The independent t-test was used to compare parametric data, and Mann–Whitney U test was used to compare non-parametric data. Spearman and Pearson correlation analyzes were used for correlation assessment. A linear regression analysis was conducted to examine the effect of the Twitter mentions’ numbers on the articles’ citation numbers. A confidence interval of 95% and a p of less than 0.05 were considered significant.
In the first half of 2018, 410 articles, original research articles, and review articles were published in the sexual medicine journals included in the present study. Out of these articles, 352 (85.9%) were original researches and 58 (14.1%) review articles (Table 1). The descriptive analysis of these articles based on the included journals is shown in Table 2.
Three hundred ninety-five (96.3%) of the articles were cited at least once in Google Scholar. The median number of citations for articles published in this period was six citations (interquartile range (IR): 0–111 cite). In the Scopus query, 32.2% of all these articles (n = 132) were cited at least once (median 0; IR: 0–63). We observed that 263 (64.1%) of all articles were tweeted at least once. The median number of mentions on Twitter was one Tweet (IR: 0–2348 Tweet) (Fig. 1).
The median citation number of articles on Google Scholar and Scopus but not mentioned on Twitter was 4 (IR: 0–25) and 0 (IR: 0–7), respectively. The median citation numbers of the articles mentioned at least once on Twitter was 7 (IR: 0–111) for Google Scholar, whereas it was 0 (IR: 0–63) for Scopus, respectively.
Subgroup analysis of the articles showed that three journals were evaluated with the PlumX metric. The median citation numbers of these three journals were 8 for Google Scholar (IR: 0–111) and 5 for Scopus (IR: 0–63). The median number of Twitter mentions was 10 (IR: 0–2348), while the median numeric value in the PlumX metric was 41 (IR: 0–378). In this analysis, a significant positive correlation was found between PlumX metric value, the number of Twitter mentions, and traditional citations (p < 0.001).
The median citation numbers of the other four journals that evaluated with Altmetric was 5 (IR: 0–46) for Google Scholar and 0 for Scopus (IR: 0–20). Again, the median number of mentions of these articles on Twitter was 0 (IR: 0–46), and the median numerical metric value in Altmetric was 1 (IR: 0–251). In this other subgroup analysis, we showed a positive correlation between the Altmetric value, the number of Twitter mentions, and the number of traditional citations (p < 0.001).
Articles that had been tweeted at least once received more citations than those without (Mann–Whitney U: 12749.5, p < 0.001). Besides, we found a significant relationship between the number of citations and tweets in evaluated articles (p < 0.001). In the linear regression model, we observed that the number of tweets (p < 0.001) and the article type (p < 0.001) are the factors related to the number of Google Scholar and Scopus citations (p < 0.001, both) (Fig. 2). However, the journal type did not affect the citation rates (p: 0.282).
The first ten articles with the most citations (GS and Scopus) and tweets are shown in Table 3. Besides, the review articles had more citations on both platforms (GS and Scopus) and more tweets than the original articles (all of the p values < 0.001) (Table 4).
Citation is accepted as an acknowledgment dedicated to the cited article in the scientific community . Furthermore, the citation is assumed to be an indicator for exploring different aspects, such as detecting the latest research trends, finding the study topics, ranking corporations, and ranking the journals . Until recently, an article’s scientific effect primarily has been measured by its citations . The new metric systems such as Altmetric and PlumX metric have been increasingly recognized to meter the real-time reach and influence of an article . Alternative metrics, Altmetric and PlumX metric included quantifying the digital attention that an article receives in diverse online sources [18, 19]. The articles produced from these metrics showed a significant correlation with the number of traditional citations . For this reason, Altmetric scores have been used frequently to evaluate article impact in recent years [21, 22].
This trend raises the question of how SoMe usage affects traditional metrics. Recently, some brands of SoMe, such as Twitter and Facebook, have become a part of our daily life, and their popularity has increased steadily over the last years [23,24,25]. The majority of SoMe platforms are frequently used in the scientific area for different purposes, including to provide information and to spread scientific work [26, 27]. Moreover, they have found a place in medical education as well . Besides, social network programs are frequently used to create a collaborative environment and to increase cooperation between employees in the common area. Furthermore, it is also an effective vehicle in finding solutions to important social issues .
Like in the general population, SoMe popularity manifested itself among healthcare professionals as well. A survey among Canadian urologists has revealed that 26% used SoMe routinely in their personal lives . Another survey among AUA members revealed that 74% had a SoMe account but only 28% used these in a professional setting . Another study has shown that the 99% of young urologist in Europe uses social media both personally and professionally . Moreover, SoMe is not only limited to personal or professional use but also in scientific meetings. It is frequently used, especially during the urology conferences, for the increased interaction among participants . Twitter metrics related to the meetings of some American societies reveal data on the number of users, tweets, and interactions at their annual meetings [32, 33]. Similarly, the AUA and the European Association of Urology have formally adopted and promoted the use of year-specific meeting hashtags [23, 34].
The emergence of SoMe and its adoption by health professionals have opened new sharing venues for scientific literature. With the common usage of SoMe, it enables that information can be rapidly shared and disseminated. Hence, many journals have established Twitter-based Journal clubs to make interdisciplinary interaction possible, inform patients and meet patients’ expectations as well as make published articles more visible . Thus, SoMe may spread information to a wider and diverse public in a shorter time . Manuscripts tweeted after publication were shown to be 11 times more likely to be cited compared to those cited as usual .
The scientific effect of SoMe in various medical fields has been reported in some studies [21, 37, 38]. Similarly, its impact on urology and pediatric urology has been previously evaluated. In the study of O’Kelly et al., SoMe use within pediatric urology was associated with a higher impact factor . Hayon et al. stated that articles mentioned on Twitter have more citations in Scopus and Google Scholar than those not mentioned. (For Scopus 2-fold; for Google Scholar 2.3-fold). Also, they thought that Twitter activity after journal publication might be an early indicator of the impact factor of a urologic academic paper . Despite the existence of studies related to SoMe and articles in general urology, pediatric urology, and uro-oncology, to our best knowledge, there has been no study evaluating the scientific impact of SoMe in sexual medicine and reproductive health [39,40,41]. In fact, the internet and SoMe platforms serve as a source of information and solution for patients with sexual problems [26, 42,43,44]. Therefore, in this study, we researched the mention numbers of publications on Twitter, their evaluation with different metric scales, and traditional citation systems due to the increasing demand for information on virtual social networks of sexual medicine. As a result, Twitter is determined to be an important tool in disseminating information about sexual medicine . Increasing the use of Twitter as a knowledge-sharing tool will make articles published more visible and thus result in an increased number of citations.
This study has some limitations. First, we reviewed only the top seven considerable sexual medicine journals. We could not evaluate all of the sexual journals. Second, we assessed the data on average only two years after the publication, but we could not examine the longer-term effects of Twitter using. Third, we only evaluated the 6 months-frame of the publications, but we believe this time frame is enough to generalize the available data. Fourth, we could not determine the narrator of the Tweets analyzed. Therefore, we were unable to evaluate whether the narrator of the Tweets contributed to the citation rates. Also, our study does not explain the reason-result relationship. It only evaluates the correlation between Twitter mentions and Google Scholar citations.
In conclusion, SoMe are integrated with modern communication in our personal and professional lives. Today, many academics in the health system share abstracts or entire articles that attract significant attention on Twitter. Thanks to this sharing, faster access to information are provided in sexual and reproductive health both in the general population and in the academic community. Moreover, according to these results, we observed that the publications evaluated with SoMe and different metric scoring systems were correlated with the citation numbers in the traditional citation system. We believe that this study would provide a guide for future research.
Boyd DM, Ellison NB. Social network sites: definition, history, and scholarship. J Comput Mediat Commun. 2007;13:210–30.
Ucar T, Culpan M, Caskurlu T, Karaman Mİ, Silay MS. The activity and discussion points of# Circumcision through Twitter; a microblogging platform. Int J Impot Res. 2018;30:249–52.
Matta R, Doiron C, Leveridge MJ. The dramatic increase in social media in urology. J Urol. 2014;192:494–8.
Loeb S, Catto J, Kutikov A. Social media offers unprecedented opportunities for vibrant exchange of professional ideas across continents. Eur Urol. 2014;66:118–9.
Dol J, Tutelman PR, Chambers CT, Barwick M, Drake EK, Parker JA, et al. Health researchers’ use of social media: scoping review. J Med Internet Res. 2019;21:e13687.
Davidson PM, Newton PJ, Ferguson C, Daly J, Elliott D, Homer C, et al. Rating and ranking the role of bibliometrics and webometrics in nursing and midwifery. Sci. World J. 2014;2014:135812.
Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J Med Internet Res. 2011;13:e123.
Zahedi Z, Costas R, Wouters P. How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics. 2014;101:1491–513.
Kwok R. Research impact: altmetrics make their mark. Nature. 2013;500:491–3.
Loeb S, Bayne CE, Frey C, Davies BJ, Averch TD, Woo HH, et al. Updated survey of social media use by members of the American Urological Association. Urol Pract. 2015;2:138–43.
Borgmann H, DeWitt S, Tsaur I, Haferkamp A, Loeb S. Novel survey disseminated through Twitter supports its utility for networking, disseminating research, advocacy, clinical practice and other professional goals. Can Urol Assoc J. 2015;9:E713–7.
Hicks D, Wouters P, Waltman L, De Rijcke S, Rafols I. Bibliometrics: the Leiden Manifesto for research metrics. Nature. 2015;520:429–31.
Elmore SA. The Altmetric attention score: what does it mean and why should I care? Toxicol Pathol. 2018;46:252–5.
Ortega JL. Reliability and accuracy of altmetric providers: a comparison among Altmetric. com, PlumX and Crossref Event Data. Scientometrics. 2018;116:2123–38.
Cooper C, Booth A, Britten N, Garside R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: a methodological review. Syst Rev. 2017;6:234.
Bornmann L, Daniel HD. What do citation counts measure? A review of studies on citing behavior. J Doc. 2008;64:45–80.
Zhu X, Turney P, Lemire D, Vellino A. Measuring academic influence: not all citations are equal. J Assoc Inf Sci Technol. 2015;66:408–27.
Costas R, Zahedi Z, Wouters P. Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. J Assoc Inf Sci Technol. 2015;66:2003–19.
Warren HR, Raison N, Dasgupta P. The rise of altmetrics. J Am Med Assoc. 2017;317:131–2.
Chang J, Desai N, Gosain A. Correlation between altmetric score and citations in pediatric surgery core journals. J Surg Res. 2019;243:52–8.
Azer SA, Azer S. Top-cited articles in medical professionalism: a bibliometric analysis versus altmetric scores. BMJ Open. 2019;9:e029433.
Loeb S, Bayne CE, Frey C, Davies BJ, Averch TD, Woo HH, et al. Use of social media in urology: data from the A merican U rological A ssociation (AUA). BJU Int. 2014;113:993–8.
Fuoco M, Leveridge MJ. Early adopters or laggards? Attitudes toward and use of social media among urologists. BJU Int. 2015;115:491–7.
Ku S, Balasubramanian A, Yu J, Srivatsav A, Gondokusumo J, Tatem AJ, et al. A systematic evaluation of YouTube as an information source for male infertility. Int J Impot Res. 2020. https://doi.org/10.1038/s41443-020-0322-9 [Epub ahead of print].
R Russo GI, di Mauro M, Cocci A, Cacciamani G, Cimino S, Serefoglu EC, et al. Consulting “Dr Google” for sexual dysfunction: a contemporary worldwide trend analysis. Int J Impot Res. 2020;32:455–61.
Altintas E, Gul M. The dark side of the internet regarding sexual education. Int J Impot Res. 2021. https://doi.org/10.1038/s41443-020-00398-0 [Epub ahead of print].
George DR, Dellasega C. Use of social media in graduate-level medical humanities education: two pilot studies from Penn State College of Medicine. Med Teach. 2011;33:e429–34.
Greil-Soyka M, Quayle E, Bitzer J. European Sexual Medicine Network: a unique platform for collaboration. Int J Impot Res. 2020. https://doi.org/10.1038/s41443-020-0299-4 [Epub ahead of print].
Rivas JG, Socarras MR, Patruno G, Uvin P, Esperto F, Dinis PJ, et al. Perceived role of social media in urologic knowledge acquisition among young urologists: a European survey. Eur Urol Focus. 2018;4:768–73.
Loeb S, Carrick T, Frey C, Titus T. Increasing social media use in urology: 2017 American Urological Association Survey. Eur Urol Focus. 2020;6:605–8.
Chaudhry A, Glodé LM, Gillman M, Miller RS. Trends in Twitter use by physicians at the American Society of Clinical Oncology annual meeting, 2010 and 2011. J Oncol Pract. 2012;8:173–8.
Cochran A, Kao LS, Gusani NJ, Suliburk JW, Nwomeh BC. Use of twitter to document the 2013 academic surgical congress. J Surg Res. 2014;190:36–40.
Wilkinson SE, Basto MY, Perovic G, Lawrentschuk N, Murphy DG. The social media revolution is changing the conference experience: analytics and trends from eight international meetings. BJU Int. 2015;115:839–46.
Thangasamy IA, Leveridge M, Davies BJ, Finelli A, Stork B, Woo HH. International urology journal club via Twitter: 12-month experience. Eur Urol. 2014;66:112–7.
Darling ES, Shiffman D, Côté IM, Drew JA. The role of Twitter in the life cycle of a scientific publication. PeerJ PrePrints. 2013;1:e16v1. https://doi.org/10.7287/peerj.preprints.16v1.
Chandrasekar T, Goldberg H, Klaassen Z, Wallis CJ, Leong JY, Liem S, et al. Twitter and academic urology in the United States and Canada: a comprehensive assessment of the Twitterverse in 2019. BJU Int. 2020;125:173–81.
Kelly BS, Redmond CE, Nason GJ, Healy GM, Horgan NA, Heffernan EJ. The use of Twitter by radiology journals: an analysis of Twitter activity and impact factor. J Am Coll Radiol. 2016;13:1391–6.
Smith ZL, Chiang AL, Bowman D, Wallace MB. Longitudinal relationship between social media activity and article citations in the journal Gastrointestinal Endoscopy. Gastrointest Endosc. 2019;90:77–83.
Gast KM, Kuzon WM Jr, Waljee JF. Bibliometric indices and academic promotion within plastic surgery. Plast Reconstr Surg. 2014;134:838e–44e.
O’Kelly F, Nason G, Manecksha R, Cascio S, Quinn F, Leonard M, et al. The effect of social media (# SoMe) on journal impact factor and parental awareness in paediatric urology. J Pediatr Urol. 2017;13:513.
Hayon S, Tripathi H, Stormont IM, Dunne MM, Naslund MJ, Siddiqui MM. Twitter mentions and academic citations in the urologic literature. Urology. 2019;123:28–33.
Alsyouf M, Stokes P, Hur D, Amasyali A, Ruckle H, Hu B. ‘Fake news’ in urology: evaluating the accuracy of articles shared on social media in genitourinary malignancies. BJU Int. 2019;124:701–6.
Gul M, Diri MA. YouTube as a source of information about premature ejaculation treatment. J Sex Med. 2019;16:1734–40.
Baunacke M, Groeben C, Borgmann H, Salem J, Kliesch S, Huber J. Andrology on the Internet: most wanted, controversial and often primary source of information for patients. Andrologia. 2018;50:e12877.
Zattoni F, Gül M, Soligo M, Morlacco A, Motterle G, Collavino J, et al. The impact of COVID-19 pandemic on pornography habits: a global analysis of Google Trends. Int J Impot Res. 2020;28:1–8.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Ozkent, M.S., Böcü, K., Altintas, E. et al. Correlation between Twitter mentions and academic citations in sexual medicine journals. Int J Impot Res (2021). https://doi.org/10.1038/s41443-021-00457-0