Introduction

Open access (OA) publishing has gained significant traction in recent years within the scientific community. Open science practices are reported to improve the rigor, replicability, availability, and pace of scientific research [1,2,3]. The push for open scientific publishing has been accelerated by the evolution of the Internet, which has afforded greater opportunities for scientific work to be accessed worldwide by scientists as well as for the general public. Since the Budapest Open Access Initiative was first introduced in 2002, scientific journals have increasingly adopted open access policies by: 1) becoming fully OA, or 2) revising their existing policies to allow authors to opt in to OA [1, 4]. Additionally, free digital repositories, such as PubMed or ResearchGate, have allowed for more rapid, open dissemination of scientific research.

Since a goal of OA publishing is to improve the availability of scientific research, OA may also have a positive impact on the audience and reach of a particular scientific article. Traditionally, article visibility was measured using citation counts, defined as the number of times an article is mentioned in scientific literature [5]. Citation counts were also used to calculate the impact factor of a particular journal, which served as a standard metric to evaluate the quality and impact of a journal [6, 7]. However, using this strategy alone to evaluate the visibility of an article may overlook key elements of modern readership for scientific publications [5, 8]. As citations are collected from purely academic publications, citation counts may not be an accurate measure of the overall number of mentions an article receives or reflect the entire audience of an article, particularly in the age of the Internet and open science. Moreover, with the increased pace of online social media and media coverage, citation counts are slow to reflect real-time attention that is paid to an article.

In response to these pitfalls, the Altmetric method of measuring article visibility has emerged as an additional means to determine the impact of any given scientific article. Altmetric scores aim to quantify non-traditional sources of attention for scientific articles. The Altmetric score is a proprietary weighted algorithm that tracks mentions of an article from many different sources, such as news media or Twitter (X) mentions, to represent a single weighted count that reflects online article access and is therefore a measure of rapid dissemination of article content [6, 9, 10]. When combined with citation counts, Altmetrics provide a more complete, nuanced, method of tracking the performance and impact of scientific articles, particularly as the Internet is increasingly used by a broader non-academic population of readers to access scientific research.

Previous studies suggest that OA publishing may increase both citation counts and Altmetric scores of scientific articles – an effect that has been termed “the open access advantage” [11,12,13,14,15,16]. These studies, however, are specific to their respective datasets, and results may vary depending on scientific field and/or journal. For instance, while one study suggested that OA increases both citation rates and Altmetrics of orthopedic studies, a different publication found that OA increases Altmetrics, but not citation rates for articles investigating total knee arthroplasty, a subfield of orthopedic research [17, 18]. These discrepancies demonstrate the need to consider the complex, nuanced effects of OA on article impact, rather than drawing broad conclusions for scientific publishing as a whole.

In Neuropsychopharmcology (NPP), there are several tiers of access that determine the article processing charges (APCs) paid by the authors and the final availability of the article on the publisher’s website [19, 20]. The first tier, called the Green tier, is a standard access (SA) subscription model. In Green tier, authors pay a reduced APC, which in some instances is free, and the article content on the publisher’s website is restricted to readers who pay a one-time or yearly subscription for access. Six months after publication, these articles may be self-archived by authors on a freely available online repository [19]. The second tier, called the Bronze tier, is determined by NPP’s publisher, Springer Nature. Articles in this tier are published by authors under the same APCs as Green tier articles, but their contents are made fully available free of charge for any reader on the publisher’s website at Springer Nature’s discretion. As with Green tier articles, Bronze articles may be self-archived by authors on a public online repository six months after publication [19]. Many articles published prior to 2021 in the journal’s archives belong within this category. The final group is Hybrid tier, in which authors pay higher APCs to allow free access to article content on the NPP website. Additionally, this article type is deposited by Springer Nature to PubMed Central and Europe PubMed Central, and authors may post the final published version to a free online repository immediately upon publication [19]. A more detailed description of these categories is available in the Methods section. It is still unclear whether these differences in article availability or publication costs may affect the impact of research published in NPP. While a previous article indicated that OA publishing may increase citation count in psychiatric journals, the dataset was not specific to NPP articles and did not consider Altmetrics [21].

Accordingly, we aimed to quantify the effect of OA publishing on article visibility and impact in NPP. A selection of 6000 articles published between 2001–2021 were analyzed for the effect of article availability and publishing tier on citation counts and Altmetric scores. Articles were separated by article type (e.g., primary research article, review article), and by publication year. Overall, Bronze articles received significantly more citations than Green and Hybrid articles. However, this effect was mainly driven by the performance of older Bronze articles, as Bronze and Hybrid articles were cited at comparable rates within the last decade. Altmetric scores were comparable between all three tiers, although there was a trend of increased Altmetric score in Bronze + Hybrid articles published within the last decade. Together, these results indicate that articles that are freely available on the NPP website have increased citation counts and may also have slightly increased Altmetric scores, although the higher APCs associated with Hybrid articles do not guarantee improved article metrics.

Materials and Methods

Article selection and categorization

Six thousand articles published between 2001–2021 were selected for analysis. Articles were first sorted for highest cited NPP articles, according to the Web of Science database (https://www.webofscience.com/wos/), then the top 6000 most cited articles were selected. As citations trended towards 0 below this limit, this method was utilized to reduce data skew, which could potentially confound our analyses. Subsequently, article titles were cross-referenced with the Altmetrics database (https://www.altmetric.com/explorer/) to obtain the Altmetric Attention Score of each article. As article metrics are constantly changing over time, citation counts and Altmetric scores were collected within a 24-hour period. Items in NPP that fell within any category (e.g., Article, Letter, Editorial Material) were included within the “All Articles” category. Items in the primary research article group were restricted to “Articles,” while review articles were restricted to “Reviews.”

Articles in the standard subscription model are considered Green tier articles. These articles have a lower APC, which in many cases is free, and may be self-archived by authors within a funder or institutional repository (e.g., PubMed) after an embargo period of 6 months after publication. While other publishers may automatically deposit standard access articles to an online repository, in NPP, authors are required to self-archive Green tier articles to comply with institutional and/or funding mandates [19]. Once archived by the authors, Green tier articles are publicly and freely available online via a third-party repository. To access the article prior to this embargo or at any time on the publisher website, a one-time or longer-term subscription fee must be paid by the reader to access the articles. Following the embargo period, readers may fully access the contents of the article on the repository, although access on the publisher’s website is still restricted to readers with a subscription. Authors may also choose to publish their articles OA, classified in this dataset as Hybrid tier. Here, for a higher APC, articles are immediately available on the publisher website under a Creative Commons Attribution 4.0 International (CC BY) License [19, 22]. Additionally, if the Hybrid tier option is selected, NPP’s publisher, Springer Nature, will automatically deposit the article in PubMed Central and European PubMed Central upon publication. Authors of Hybrid tier articles are permitted to immediately post their article to a free online repository.

In addition to Green and Hybrid tiers, NPP articles may also fall within a third category: Bronze tier. Bronze tier articles are freely available on the publisher website (e.g., freely accessible journal archives) but lack a CC BY License, thereby prohibiting reuse or adaptation of article content [20]. Articles in Bronze tier are submitted under the standard access option but their content is provided free of charge on the publisher’s website. Additionally, Bronze tier articles are self-archived on online repositories six months after publication, comparable to Green tier articles. To our knowledge, there is no standardized method for authors to request or select that their articles be published as Bronze. Articles may be added to or removed from the Bronze tier category at any time after publication at the publisher’s discretion.

Statistical analysis

Data were analyzed using Prism 10.0 (Graphpad Software, La Jolla, CA), with α = 0.05 for all analyses. All data were graphed with a frequency histogram and tested for normal distribution using a D’Agostino & Pearson Test [23]. Data that were significantly skewed were transformed using a natural logarithm function and re-tested for normality. Nonparametric data were compared using a Kruskal–Wallis Test and follow-up post-hoc Dunn’s Multiple Comparisons Tests. When comparing by year, a Two-Way ANOVA was used to determine the effect of Tier and Year, with post-hoc Tukey’s Multiple Comparisons Tests where applicable [24]. When only two groups were compared, a Welch’s t-test with False Discovery Rate correction was used to determine significance. All statistical tests and p-values are listed in Table S01.

Results

Bronze tier articles significantly outnumber Hybrid and Green tier articles

We first quantified the percentage of articles that were published in each tier (Fig. 1). We predicted that the smallest number of articles would be in the Hybrid tier, while the majority of articles would be classified within the Green tier. As predicted, the smallest category was articles that were published as Hybrid tier, comprising ~7% of the overall group of articles included in the dataset. Surprisingly, a large majority of articles (~70.5%) were in the Bronze tier, while a much smaller percentage of articles (~22.5%) were classified as Green tier. These data indicate that while many articles are available to the general public, the majority of these pieces are published without paying the higher APC associated with Hybrid tier publishing. A similar trend was found in the Articles and Reviews datasets, with the largest percentage of these article types classified as Bronze tier, and the smallest percentage classified as Green tier (Fig. S1).

Fig. 1: There are three publishing tier options in NPP.
figure 1

A Table depicting the cost, payer, location, time, and availability of articles depending on publishing tier. B The majority of articles were published under the Bronze tier. Approximately 22% of articles were published under the Green tier, while the lowest amount was published under the Hybrid tier. (N = 6000 articles). *, individual costs may vary.

Full size image

Bronze tier articles receive the highest number of citations

Previous data suggest that OA may enhance research impact by increasing citations across various research fields [11,12,13,14,15,16]. Here, we investigated whether citation counts were altered by publishing tier. In all articles (Fig. S2A), primary research articles (Fig. S2C), and reviews (Fig. S2E), pieces published in Bronze tiers received significantly more citations than those published as Green or Hybrid. There were no statistical differences between Green and Hybrid tier articles. However, a frequency histogram revealed that all datasets had a positive skew, as there were many datapoints with 0 citations (Fig. S2B, D, F). We therefore performed a natural logarithmic transformation of the data, which produced a frequency histogram with a better fit to a normal Gaussian distribution (Fig. 2A, B, D, E, G, H). Analyses of these transformed data revealed that in all article types (Fig. 2C), primary research articles (Fig. 2F), and review articles (Fig. 2I), citations per article was significantly increased in Bronze articles when compared with Green and Hybrid tiers.

Fig. 2: Bronze tier articles receive the highest number of citations.
figure 2

A Normal QQ plot of transformed data for all articles. B A natural log transformation allowed for a normal Gaussian distribution of citation count data. C Ln transform of citation count per article for all articles tiers. Bronze tier articles received the highest citation counts when compared with Green and Hybrid articles. D, E A natural log transformation was used to fit primary research article citation data to a normal distribution. F Bronze tier primary research articles received the highest citation counts in comparison to Green and Hybrid tiers. G, H The citation count data for review articles followed a Gaussian distribution following a natural log data transformation. I As with previous categories, Bronze tier reviews received significantly higher citation counts than Green and Hybrid tier reviews. J When analyzed per year, Bronze tier articles received the most citations in 2001, 2004, 2005, and 2008. After 2012, Green tier articles received the lowest number of citations, although this difference was not statistically significant. (N = 6000 articles, 4084 primary research articles, 363 review articles). Error bars indicate ± SEM. *p < 0.05, ****p < 0.0001.

Full size image

As the number of citations per article is a metric that generally increases over time, we aimed to determine the effect by year in all article types. A frequency distribution and QQ plot revealed that these data passed tests for normality, indicating that parametric strategies were appropriate for statistical analysis (Fig. S2G, H). In 2001, 2004, 2005, and 2008, there were 0 Hybrid tier articles published, limiting options for data analysis. Therefore, we excluded these years from our dataset and ran a 2-way ANOVA, which revealed that there was a significant effect of year, but not article tier on citation counts. We than ran t-tests to compare citations between Green and Bronze tier articles within the aforementioned years, which revealed that Bronze articles received significantly more citations than Green publications (Fig. 2J). From 2012 onwards, Hybrid tier articles received either comparable or slightly higher citation counts when compared with Bronze tier articles. Conversely, the citation counts of Green tier articles within the last decade continuously declined; however, these results were not statistically significant.

Overall, these data suggest that public availability of articles on the NPP website may have a positive effect in increasing citations and thereby enhancing the impact of articles published within NPP. Nonetheless, these results may be largely driven by Bronze articles, with Green and Hybrid tiers receiving similar overall citations. However, when broken down by year, recent trends show that Hybrid tier articles receive citation counts that are more similar to Bronze than to Green tier articles. Together, these results suggest that as the OA publishing model becomes more established, the scholarly impact of Hybrid tier articles may increase over time.

Altmetric scores are comparable between all article tiers

Subsequently, we examined whether Altmetric score was significantly different between Green, Bronze, and Hybrid tiers. Although Altmetric scores were established in 2011, articles published prior to this year may still be assigned an Altmetric Attention Score and were therefore included in the analysis. Previous studies have reported that OA may enhance Altmetric scores, suggesting that broader access increases attention for research articles [11, 12, 14, 17, 18]. However, it is currently unclear whether OA publishing may affect the Altmetric scores of articles within NPP.

Prior to data transformation, in all article types, Bronze articles received significantly higher Altmetric scores when compared with Green and Hybrid tiers (Fig. S3A). Hybrid tier articles also received significantly higher Altmetric scores than Green tier articles. In primary research articles, Hybrid tier articles received significantly lower Altmetric scores when compared with both Green and Bronze tier articles (Fig. S3C). Bronze tier review articles received the highest Altmetric scores, but there was no significant difference between Green and Hybrid tier review articles (Fig. S3E). Because all three datasets were significantly positively skewed (Fig. S3B, D, F), we applied a natural logarithmic transformation to better fit the data to a natural Gaussian distribution (Fig. 3A, B, D, E, G, H). Following this transformation, we found that Altmetric score was comparable between Green, Bronze, and Hybrid tiers for all article categories (Fig. 3C), including primary research articles (Fig. 3F) and reviews (Fig. 3I).

Fig. 3: Altmetric scores are comparable between all article tiers.
figure 3

A, B A natural log transformation was used to fit the data to a normal Gaussian distribution. C There was no statistical difference in Altmetric score between Green, Bronze, and Hybrid tiers. D, E For primary research articles, data were transformed to fit a normal distribution using a natural log function. F Once again, Altmetric scores were comparable between all 3 publishing tiers. G, H Consistent with previous categories, a natural log function was used to transform the data for review articles in order to fit a Gaussian distribution. I Similarly, review articles received comparable Altmetric scores in Green, Bronze, and Hybrid tiers. J When analyzing the data by year, Bronze tier articles received the highest Altmetric scores in 2002, 20004-2006, and 2021. In 2021, Hybrid articles also had significantly Altmetric scores when compared with Green tier articles. n = 6000 articles, 4084 primary research articles, 363 review articles). Error bars indicate ± SEM. ***p < 0.001, ****p < 0.0001.

Full size image

We then examined the effect of article availability on Altmetric score by individual year for all articles. Because these data passed statistical tests for normality (Fig. S3G, H), we used parametric methods for data analysis. First, we excluded 2001, 2004, 2005, and 2008, as there no were Hybrid articles published in these years. A 2-way ANOVA revealed a significant Tier x Year interaction. Subsequent posthoc tests indicated Bronze tier articles received the highest Altmetric scores in 2002, 2006, and 2021. In 2021, Hybrid tier articles also received significantly higher Altmetric scores when compared to Green tier articles. Finally, t-tests were performed to compare Altmetric scores between Green and Bronze tiers within the previously excluded years, which revealed that Bronze received significantly higher Altmetric scores than Green (Fig. 3J). Overall, these data indicate that publishing tier does not significantly or consistently impact Altmetric score. However, for articles published within the past 9 years, there were several years in which many Green articles had an Altmetric score of 0. As a natural logarithm transformation removes data points with a value of 0, it is possible that transforming our dataset to fit a normal distribution may result in missing data trends. Nonetheless, our findings suggest that while public access to article content on the NPP website may have a trending positive impact on enhancing online attention in recent years, its effects are not consistent.

Discussion

In this analysis, we characterized the effect of OA publishing on article metrics published in NPP. We found that the majority of articles are Bronze tier articles that are made freely available online without paying higher APCs typically associated with OA publishing. Overall, Bronze articles received the highest citation counts; however, in the past decade, there was a trend in which Bronze and Hybrid articles received comparable citations. Conversely, article tier did not significantly impact Altmetric scores. In some years, Bronze articles received significantly higher Altmetric scores than Green articles, but this effect was not consistent.

Our results expose a complex relationship between impact and public access. Overall, articles that are freely available on the Springer Nature website received significantly more citations than standard access articles. While it is difficult to establish causal relationships in our analyses, this increase may be attributed to improved global access for researchers, particularly in countries with lower access to scientific funding. Moreover, increasing the online accessibility of articles at the primary publication source may improve ease of access for all scientists, making them more likely to cite these sources within their own manuscripts. In essence, these researchers may be citing articles that they can access, such as Hybrid, compared to other publishing tiers. Although Hybrid articles received comparable overall citation counts to Green articles, separating the dataset by publication year revealed that this finding may be due to low numbers of Hybrid articles prior to 2011. Following this year, when a greater number of Hybrid articles began to be published in NPP, there was a trend in which Hybrid articles received citation counts that were more comparable to Bronze than to Green tiers.

Although the effects of publishing tier on Altmetric score were not statistically significant overall, data from recent years suggest that newly published Bronze and Hybrid material may receive more online attention than Green tier publications. This trend may reflect a shift in scientific publishing away from traditional, issue-based printed publications towards a greater presence of online material that is often available prior to the publication of the monthly NPP print issue. We predict that as published online material continues to become more prevalent, freely available NPP articles will continue to receive higher article metrics in comparison to those restricted by a paywall.

While OA publishing may have positive effects on article impact, the increased cost of publishing OA articles is a potential hurdle for expanding OA science. As Springer Nature retains sole discretion to add or remove articles within the Bronze category, Hybrid tier publishing is the only currently available method for authors to ensure that articles remain available to all readers free of charge in perpetuity. Nonetheless, our results indicated only a moderate potential benefit to Hybrid tier on enhancing citation counts. As OA publishing has numerous positive benefits to improving the accessibility of scientific content and the rigor and transparency of science as a whole, charging higher APCs while demonstrating a moderate benefit to authors remains a challenge. In NPP, APCs for Hybrid articles are almost twice the APCs for Green articles. This cost difference likely contributes to the small percentage of Hybrid tier articles in the dataset and may be prohibitive for early career researchers as well as scientists located internationally or at non-research-focused universities. The cost difference also presents a potential confound, as labs with the funding to pay Hybrid APCs are also likely to be directed by established scientists who are already influential within their respective fields and are under institutional or funding mandates to publish OA. Therefore, it is possible that articles from these well-funded groups would receive a large number of citations regardless of publishing tier, making it difficult to isolate the effects of article availability on enhancing citation count. Offering alternative methods of reducing Hybrid APC costs for authors, such as fee waivers, may contribute to making Hybrid tier a more accessible, equitable, and widespread method of publishing.

It is also important to acknowledge the role of public repositories (e.g., PubMed) and preprints in affecting article impact. As many authors, particularly those in the United States, have funding or institutional obligations to archive accessible versions of their publications on a free public server, Green tier NPP articles are still accessible online to the general public. This accessibility makes it difficult to reconcile the statistically different citation counts between the 3 publishing tiers. For the lay public, reduced awareness of publicly available scientific content, such as PubMed, may contribute to differences in article accessibility. Additionally, some authors may not have funding or institutional mandates to archive Green tier articles after embargo. In these instances, the publisher’s NPP website would be the only legal source to access article content and would be dependent on a reader’s ability to pay for a subscription. Finally, as previously acknowledged, an author’s reputation and funding may play a stronger role than article availability in determining the likelihood of citation for a given article. As preprints are a relatively new phenomenon, it is difficult to determine their effect on article impact. Given that preprints may be cited prior to an article’s publication, we hypothesize that it is unlikely that they will have a strong impact on citation counts. Rather, we predict that by generating awareness of a given study prior to publication, it is more likely that they would alter Altmetric scores, driving online traffic and increasing awareness through social media and other online sources. Another possibility is that authors may opt to publish their articles via Green tier and post their articles on a preprint server, thus making a manuscript effectively open access without incurring associated costs.

Interestingly, our analyses demonstrated an inconsistency between the effects of OA publishing on citation counts and Altmetric scores. As a previous study demonstrated a correlation between the two types of article metrics in NPP, we expected that OA would have similar effects on these measures of article access [25]. The inconsistency in our results may be the result of how these two measures are calculated over time. For instance, citation counts continually increase over time as articles are published and gradually accrue citations in the literature, making them a cumulative, enduring measure of research impact. Conversely, Altmetric scores fluctuate more readily, and may even decrease as online posts are deleted, posts are flagged as spam, or when the database restructures the scoring algorithm [10]. Finally, the online activity of an article’s authors may also contribute to differences we observed. Following publication, NPP authors receive a link to publicly share or distribute their article online regardless of article tier, and this distribution may contribute to the differences observed in citation count and Altmetric score. Additionally, as authors who self-promote their work by posting on social media can often generate subsequent online traffic, perhaps through replies or reposts, that does not affect the later likelihood of article citations. Considering these differences in data accumulation and article promotion, it seems plausible that OA status may have a dissociable impact on enhancing these measures of article performance.

Although our data highlight the benefits of OA publishing, our analyses have several limitations. As citations and Altmetrics continuously change, the effect of publishing tier on these values may also shift, particularly as more readers access articles online. The results of future analyses may differ from our current findings, in part due to the evolving role of the Internet in scientific research. Additionally, our data were strongly positively skewed, with many articles exhibiting a citation count and/or Altmetric score of 0. We attempted to reduce this skew by limiting the articles in our analysis and logarithmic transformations. Despite our efforts, interpreting the results of the analyses remains challenging. Future analyses may have more success in limiting data skew by further restricting the articles included to obtain a normally distributed dataset, thus making analysis and interpretation more straightforward. Despite these limitations, our results suggest that increasing public access to scientific research is correlated with higher article impact and increased online attention.

In conclusion, we report that OA publishing may affect the visibility and impact of research published in NPP. Having full access to rigorous, peer-reviewed scientific articles is imperative to advancing knowledge and improving the transparency of scientific research. This quantitative analysis of the effects of OA publishing in NPP provides an understanding of the available options as well as the benefits and limitations of choosing Green or Hybrid publishing.