Abstract

Aim

To analyse the Altmetric data of all dental literature and to find the most important and effective Altmetric factors.

Method

PubMed was searched to find all types of dental articles. To discover the most important and effective Altmetric factors in contemporary dental literature, multi-layered perceptron artificial neural network was used.

Results

A total of 582,227 PubMed records were found. The mean Altmetric scores were 3.7 ± 18, 4.1 ± 16 and 3.2 ± 14 for clinical trials, reviews and free full-text articles, respectively. Sensitivity analysis showed that news outlets, tweeters and scientific bloggers were the most important and influential Altmetric data resources. Among 150 analysed dental journals the British Dental Journal acquired the first rank. Twitter was the most popular Altmetric resource among dental journals.

Conclusion

Dental researchers, journal editors, publishers and research funders should pay more attention to Altmetrics as a newly-emerging scholarly tool measuring the social impact of research findings.

Introduction

New internet technologies such as social media have opened up new ways of measuring the impact of scientific research outputs. The term Altmetrics was introduced in 2010 as a supplement to traditional citation-based bibliometrics. Altmetrics can catalogue links to a journal article from social media channels (such as Twitter, Facebook, Google+, Reddit, Weibo and Pinterest), scholarly blog posts, discussions on peer review sites (such as Faculty of 1000 Prime, Publons or Pubpeer), coverage in news outlets, public policy, Wikipedia, YouTube, sites running Stack Exchange (Q&A), and others.1,2,3,4,5

It has been reported that Altmetrics are significantly correlated with citation and download.6 The findings of a large-scale investigation showed robust evidence that six of the Altmetric data resources (tweets, Facebook wall posts, research highlights, blog mentions, mainstream media mentions and forum posts) were associated with the number of citations, particularly in medical and biological sciences.7

Of more interest, medical research funders and charities (for example, the Wellcome Trust and John Templeton Foundation) were drawn to Altmetric data.8,9,10 A Google trend analysis shows that the popularity of Altmetrics has increased dramatically since 2012 in comparison with bibliometrics (Fig. 1).

Figure 1: Google trends analysis for the search terms 'Altmetrics' and 'bibliometrics'.
figure 1

A value of 100 is the peak popularity for the term. A value of 50 means that the term is half as popular. Likewise, a score of zero means the term is less than 1% as popular as the peak. Linear trend-line analyses are also shown. Box and whisker plot of data are shown on the left. Data are from http://trends.google.com (accessed 7 July 2017)

Full size image

The Altmetric data for 2014 and 2015 dental literature have been analysed previously, with the mean Altmetric score of 50 top dental articles in 2014 reported as 69.5 ± 73.3 The British Dental Journal (48%) and the Journal of Dental Research (16%) had the maximum number of top articles among dental journals. In 2015, all published dental literature was analysed, and the British Dental Journal gained the first rank again, followed by the Journal of Dental Research.4 In this study, we aimed to describe and analyse the Altmetric data of all dental literature to find the most important and effective Altmetric factors.

Methods

On 16 June 2017, PubMed was searched using the following queries to find all types of dental articles, dental clinical trials, review articles, and free full-text dental articles:

“1800/1/1”[PDAT]: “2017/12/31”[PDAT] AND jsubsetd[text]

“1800/1/1”[PDAT]: “2017/12/31”[PDAT] AND jsubsetd[text] AND Clinical Trial[ptyp]

“1800/1/1”[PDAT]: “2017/12/31”[PDAT] AND jsubsetd[text] AND Review[ptyp]

“1800/1/1”[PDAT]: “2017/12/31”[PDAT] AND jsubsetd[text] AND “loattrfree full text”[sb]

For each query, only the most recent 25,000 results were considered in this study. The source of Altmetric data was the Altmetric database (Altmetric LLP, London, UK).

To discover the most important and effective Altmetric factors in contemporary dental literature, a multi-layered perceptron artificial neural network was used, owing to the huge amount and complexity of data.11,12 The architecture of the proposed feed-forward artificial neural network involved two hidden layers: 70% of the data were chosen randomly for supervised batch network training and the remaining 30% of the data were used for network testing. The hidden layer activation function is hyperbolic tangent and the output layer activation function is identity. Scaled conjugate gradient was used to estimate the synaptic weights. Artificial neural network analyses were carried out by SPSS 23 (IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp.). Graphs were drawn by Microsoft Office Excel 2016.

Results

A total of 582,227 PubMed records were found. The most recent 25,000 results were considered in this investigation, of which 9,038 (36%) had Altmetric data (Fig. 2). The mean Altmetric score for all types of dental articles was 3.5 ± 21. The mean Altmetric scores for clinical trials, reviews, and free full-text articles were 3.7 ± 18, 4.1 ± 16 and 3.2 ± 14, respectively. Twitter and Facebook were the most popular social media (Fig. 3). The United States and the United Kingdom had the highest number of tweets and unique tweeters (Table 1). To decline the convolution of the artificial neural network, only the top eight potentially significant Altmetric resources were included in sensitivity analyses, which showed that news outlets, tweeters and scientific bloggers were the most important and influential Altmetric data resources in contemporary dental literature (Fig. 4).

Figure 2
figure 2

Number of different types of dental articles and Altmetric data related to the most recent 25,000 dental articles

Full size image
Figure 3: Sum of scores of different Altmetric data resources for the most recent 25,000 dental articles.
figure 3

Please note the logarithmic scale of the vertical axis

Full size image
Table 1 Ten top countries and geographical breakdown of 24,490 tweets related to the most recent 25,000 all type dental articles
Full size table
Figure 4
figure 4

Normalised importance of common Altmetric data resources related to the most recent 25,000 dental articles

Full size image

At a journal level, 150 dental journals were analysed. The mean total number of mentions was 203.5 ± 607 and the mean number of mentioned outputs was 59.5 ± 109. The British Dental Journal acquired the first rank (Fig. 5). Twitter was the most popular Altmetric resource among dental journals (Fig. 6). Readers should note that the Altmetric score is a dynamic measure and may alter over time.

Figure 5
figure 5

Top ten dental journals, showing total mentions and number of mentioned outputs for the most recent 25,000 dental articles

Full size image
Figure 6
figure 6

Altmetric data resources of 150 dental journals

Full size image

Discussion

In this cross-sectional survey, the Altmetric data of contemporary dental literature were analysed. The mean Altmetric score was 3.5; review articles had a higher mean Altmetric score. Twitter was the most popular Altmetric resource, and tweets were mainly from U.S and U.K. The number of tweets related to dental articles has apparently increased since 2015.13 The British Dental Journal had the first rank among the dental journals, followed by the Journal of Dental Research (replicating the findings of 2014 and 2015).3,4 An interesting point is that these two journals, which have Impact Factors that are far apart (the Journal of Dental Research has an Impact Factor of 4.755 while the British Dental Journal has an Impact Factor of 1.009), are close together on Altmetrics. According to the results of previous reports13,14 and those of the present study, Twitter and Facebook were popular social media among dental scientists. The British Dental Journal has its own Twitter account, @The_BDJ, with 20.2k followers and 22.7k tweets, while the Journal of Dental Research has no Twitter account and refers to the account of The International Association for Dental Research, @IADR, with 2.9k followers and 391 tweets. The situation with Facebook is similar: the British Dental Journal has its own Facebook account, @britishdentaljournal, with 5.3k followers, while the Journal of Dental Research has no Facebook account and refers to the account of The International Association for Dental Research, @DentalResearch, with 4.4k followers.

We can examine the size of the journals' online social influence via their Klout scores. A total of 400 signals from eight different social networks are used to calculate the Klout Score every day, which ranges from 1 to 100.15,16 On 22 December 2017, the Klout Score for @The_BDJ was 52, while for @IADR it was 44. It seems logical to postulate that the British Dental Journal is more active in social media than the Journal of Dental Research; hence, it achieved better Altmetric results.

As in previous reports,3,4 the present study showed that Twitter was the most common social media among dental researchers. In society as a whole, Facebook (2061 million users) and YouTube (1500 million users) were more popular networks than Twitter (328 million users) worldwide in 2017.17

One of the principles of evidence-based policymaking is use of the best available research findings at all stages of the policymaking process in government.18,19 The U.K. government policy document on the cost-effectiveness of interventions to improve the oral health of children is a good example.20 However, the results of the present study confirmed the previous report21 that contemporary dental research output has rarely been used for health policymaking, signifying the necessity of more collaborations between dental research community and policymakers to bridge the current gap between dental research and policy.

Altmetrics is growing very quickly.22,23 Recent analysis of the top 100 dental articles (according to Altmetric score) has shown no correlation between Altmetric score and citations.24 Similar results have been reported regarding orthodontic articles.25 A meta-analysis showed a weak correlation (ranging from 0.08 to 0.5) between Altmetrics and citation counts.26 In contrast, the results of a large-scale study showed an association between Altmetric data and citation amounts.7 A moderate correlation was found between citation counts and Altmetric scores for the top papers in emergency medicine and other biomedical journals.27 The results of a recent systematic review pointed to the positive correlation between Altmetrics and traditional citations.28 Of more interest, Twitter can be used to predict the citation rate of a scientific article.29,30 Also, the number of Mendeley readers was associated with the number of citations.31,32 Alongside these controversies, we must keep in mind that Altmetrics does truly measure a different kind of research impact, performing as a complement to, rather than as a substitute for, traditional citation-based metrics33 while Altmetric resources such as Twitter and post-publication peer review are growing quickly and becoming more popular among scientists.

Twitter has recently allowed expanded 280-character tweets for all its users,34 and can also be a useful tool for scientists. It allows scientists to communicate with a global network of peers, generating ideas and fostering interdisciplinary research and to both educate and learn from a Twitter community of advisors and collaborators. Scientists can also ensure they are on the cutting edge of science by following leading research institutions and scientists; they can communicate scientific and research findings directly to a public audience, or to companies and organisations that may be interested in their findings and their applications, and can benefit from conferences that they are not able to attend in person. (To examine the power of Twitter, try searching for the hashtag #aaasmtg, intended for the American Association for the Advancement of Science (AAAS) Annual Meeting.)

Post-publication peer review is a relatively novel scholarly concept, principally found in the field of dental research.36 PubMed Commons and PubPeer are two well-known resources. PubPeer allows academics to engage anonymously in post-publication peer review, which can highlight scientific shortcomings and misconduct.37. For example, in 2014, two high-profile papers on 'Stimulus-Triggered Acquisition of Pluripotency' (STAP) published in Nature claimed that putting differentiated cells under stress can re-programme them and make them pluripotent, meaning that they can mature into any kind of tissue. Before long, 134 critical post-publication peer reviews appeared on PubPeer,38 errors were found, and ultimately the papers were retracted.39

It is forecast that there will be 2.9 billion social media users around the world in 2020.40 We cannot turn a blind eye to these newly emerging technologies and rely only on traditional citation-based metrics. Dental researchers, journal editors, publishers and research funders must go beyond the boundaries of traditional bibliometrics and pay more attention to Altmetrics as a newly emerging scholarly tool measuring the real-time social impact of research findings.