Introduction

The peer review process is a pivotal component in the publication of scientific articles. In peer review, a submitted manuscript is examined by one or more peers in order to determine the quality and suitability of the manuscript for publication.1 The approach for peer review blinding may present in several forms, such as single-blind, double-blind and a totally open approach in which no blind review actually takes place.2 These different strategies aim to produce a reliable and fair peer review process. This process should be seen in the context of ethical issues in research. As in any aspect of human relationships, a conflict of interest (COI) may influence the peer review process with various negative consequences. For example, reviewers could hinder the development of a specific manuscript because they are also involved in similar research and want to publish their material first.3 Also, reviewers could push for the acceptance of a manuscript of lower quality because of strong personal or professional ties with the authors.4

Impact factor (IF) is a measure of individual scientific journals that provides one way of indicating the relative importance of their published content within a scientific field. The IF calculation is based on two main items as reported by the developer of the measure:5 'the numerator, which is the number of citations in the current year to items published in the previous two years, and the denominator, which is the number of substantive articles and reviews published in the same two years'. Thus, the IF is essentially the average number of citations that a journal's articles receive in a larger set of indexed literature. The higher the IF of a journal, the higher its position in the IF ranking. Although there is criticism on the limitations of the IF6 and also discussion on the validity of the IF as a proxy for a journal's quality and true impact,7,8 some consider the IF one of the most important determinants in choosing a journal to which they will submit their manuscripts.9 Therefore, in the present article, the IF was used as a reference to rank dental journals.

Highly ranked dental journals use the peer review process to select articles that will be published. It is important to understand the strategies of these journals in managing the peer review process, especially regarding the process of blinding reviewers and authors. Therefore, the main objectives of this study were to investigate the types of peer review blinding applied by top dental journals as ranked by IF and to assess the association between types of review and IF. Another objective was to provide a discussion of potential biases associated with the different types of peer review blinding.

Methods

Between 20 and 21 September 2020, all 91 dental journals classified by IF10 had their authors' instructions sections scrutinised in order to answer the following question: what is the type of peer review blinding used to evaluate manuscripts submitted to the journal? If this information was not on the journal's website, an e-mail was sent directly to the journal. A reminder e-mail was sent to those journals that did not answer the original request. The type of journal was also assessed (general or specialised) and the association of the type of peer review (single-blind or double-blind), and the IF was investigated through a simple linear regression analysis. Furthermore, the journals were split by IF into two categories (IF <2 and IF >2), and a binary logistic regression was applied to measure the relationship between the type of peer review blinding (single-blind vs double-blind) and the category of IF (dependent variable). All statistical analyses were performed with IBM SPSS Statistics (Version 26).

Results

The median for IF for the 91 dental journals was 1.766 (interquartile range 1.339-2.379). In 52 (57.1%) journals, the type of peer review blinding was explicitly reported (the information from four journals was obtained from the publisher's website). E-mail messages were sent to the remaining 39 (42.9%) journals to obtain the missing information. Thirty-three journals answered this request. Thirty-six (39.6%) journals had single-blind peer review, 46 (50.5%) had double-blind peer review, two (2.2%) journals reported having 'transparent' peer review and one (1.1%) journal reported having 'open' peer review. Three (3%) journals offered the possibility of two types of peer review (single-blind and transparent), but they were classified as single-blind for regression analysis purposes. One (1.1%) journal reported single- or double-blind review, depending on type of study submitted. For regression analysis purposes, this journal also classified as single-blind review. Six (6.6%) journals had the peer review classified as 'unclear' in this study due to a lack of information. Twenty-five (27.5%) and 66 (72.5%) journals were classified as general dentistry and specialised journals, respectively. A linear regression reported an association between the type of peer review and the IF, with a lower IF associated with double-blind review (p = 0.001). A logistic regression reported lower odds of association between single-blind peer review and journals with an IF below a threshold of 2 (odds ratio 0.157, confidence interval 0.059 to 0.417, p <0.001).

Discussion

The peer review process is complex, involving different dimensions and attributes. For example, it can be presented as open questions or mandatory checklists.2 Alternatively, peer review can present different types of assessors, such as internal or external reviewers.2 The focus of this study was on the type of blinding or anonymity used in the peer review process to evaluate submitted manuscripts to highly ranked dental journals, as measured by IF. The findings report an association of the type of peer review blinding and level of IF. Dental journals with a higher IF used more single-blind peer reviews and dental journals with a lower IF used more double-blind peer reviews. As far as the author is aware, this is the first study to evaluate the type of peer review blinding in dentistry.

Component blinding is applied to reduce potential biases that can interfere with the peer review process. Different biases can generate negative consequences. For example, a reviewer might reject a submitted manuscript with important research findings based on his/her interest in the fate of the manuscript. In single-blind peer review, the reviewers know the identity of the authors, but the author does not know the identity of the reviewer. Therefore, one can consider that a single-blind peer review may be sensitive to bias, depending on the reviewer's intention. Although editors ask for reviewers to declare any potential COI, there are many potential non-declared COIs that are difficult to identify.11 Furthermore, for the sake of transparency in research, all parties involved with a specific publication (including readers) should be aware of the steps that lead to a publication or rejection decision. In single-blind peer review, this transparency is not possible. Other evidence suggests that single-blind reviewers accept more publications from renowned authors and institutions than a double-blind approach in which reviewers do not have knowledge of authors and institutions.12

A substantial part of this cohort of journals opted for double-blind peer review. In this form of peer review, both sides (reviewer and author) do not know the identities of the other. Some consider this less biased than the single-blind approach because reviewers judge the manuscript without any kind of influence of the knowledge of the background of the authors. However, some criticism also exists about potential bias affecting this alternative. Some data suggest that reviewers can identify authors and their institutions by checking the way authors write or how they report their own self-citations.13 This lack of a truly double-blind effect can influence the acceptance of the manuscript. Furthermore, author and institution familiarity has been found to be associated with higher acceptance rates in medical speciality journals.14,15 Hence, in this case, a double-blind peer review becomes a single-blind one. In addition, there are inherent limitations to realistically double-blinding the process when some types of manuscripts are submitted. Randomised studies and systematic reviews usually have their protocols registered in different public databases.16,17 Hence, reviewers would be able to quickly identify the authors' identities. Finally, many authors publish their manuscripts on public platforms even before the submission of the manuscript to a scientific journal. This is the case with preprint versions of a manuscript.18 Another variation of peer review anonymity is the triple-blind peer review,19 in which the authors and their affiliations are not only anonymous to reviewers but also to the editors handling the submitted manuscripts (of course, the reviewers remain anonymous to the authors). In this form, any potential COI of the involved editors could be minimised, although this form of peer review might present limitations similar to those of double-blind peer review.

A third option is the fully open peer review, which does not involve any form of blinding of reviewers and authors. In other words, both reviewers and authors have full knowledge of their identities. The review reports might also be published with the accepted manuscript. This form of peer review is currently used by only one IF dental journal. Clearly, this is the most transparent form of peer review because the process is made public. Furthermore, although some argue that a COI in open peer review might exist because all the peer review data is made public, it is possible to investigate a potential COI after the peer review was performed, as the review reports and the identities of all involved in the peer review process are publicly reported. Also, the declaration of reviewers' names will likely work as a barrier for any further potential unethical behaviour of reviewers regarding the judgement of the manuscript. One potential limitation of this system of peer review is the difficulty of finding reviewers for evaluating manuscripts. Some data suggest that journals that use open peer review have lower rates of reviewers accepting invitations than journals using double- and single-blind peer review methods.20 Declining an invitation to review might occur due to the fear of providing negative comments when reviewing senior and well-known authors.

Two of the journals in this sample reported 'transparent' peer review. Transparent peer review means that the reviewer reports are made publicly available, but generally, the identity of the reviewers is not revealed.21 Three journals in the present sample offer authors the possibility of single-blind or transparent peer review. Journals from other medical fields may offer authors a mixture of different types of peer review.

Other interesting forms of peer review include, for example, fully open peer review with post-publication and commenting.2,19 In this case, the manuscript is first published on the journal's platform after being checked according to the journal's policies and ethical guidelines, and it is then openly reviewed as to the quality and accuracy of its content. In this form, which extends the pool of potential reviewers, interested readers can review the published article and publish their comments on the journal's platform. This is a dynamic process in which authors have the possibility of taking these comments into consideration and updating versions of the article. Hence, one can see this strategy as an optimisation of the peer review process by having a more heterogeneous group of post-publication reviewers. However, further investigation is necessary to evaluate the impact of such an approach on the quality of the article as well as on the influence of potential biases that can distort the findings/conclusions presented. With this form of peer review, one can consider that feedback from so many 'reviewers' might put the article at a higher risk of having structural changes that can cause deviations from the original idea of the project.22

It is also important to emphasise that other measures, in addition to the concept of anonymity, must be considered if peer review is to be made a more effective process.23 For instance, the concept of reproducibility in a study could be examined during the review process to strengthen trust in whether the data are in fact accurate. Journals' editorial policies should consider the systematic use of checklists for reporting different study designs;24 establishing minimal standards of reporting would allow for the proper analysis of reproducibility. Furthermore, for peer review consistency and quality, clinicians as well as reviewers should be encouraged to use the same checklists when examining submitted manuscripts.25 Some evidence suggests that the quality of manuscripts might increase when reviews are based on reporting guidelines.26 Furthermore, reporting guidelines would likely generate more homogeneous reviews of submitted manuscripts, based on the critical information and criteria identified in the checklists endorsed by the journals. To increase the effectiveness of reviews, one measure would be the adequate selection of reviewers to examine specific areas of a manuscript. For example, research librarians could be invited to check the quality of the search strategies in systematic reviews or adequate references in other studies.27 Statisticians could be responsible for the examination of the statistical components of a manuscript, mainly in the case of more complex analyses.28 These measures may work side by side with the concept of anonymity (or lack of it, in the case of open peer review) in the peer review process in order to provide the most accurate data possible for the reader.

Conclusion

In conclusion, dental journals with higher IFs differed substantially from journals with lower IFs regarding the type of peer review blinding. Journals with higher IFs had more single-blind peer reviews and journals with lower IFs had more double-blind peer reviews. The most transparent form of peer review, by definition, seems to be open peer review, which makes both the full reviewer reports and reviewer identities publicly available. The present findings inform readers of the type of peer review blinding used in these journals as well as help editors to consider the type of peer review blinding appropriate for their journals.