The number of articles published yearly in dental science is immense. Scopus data indicate that 22,115 documents with “dentistry” as the subject area were published in 20211, representing increases of 28% and 175% in articles compared with 2011 and 2001, respectively. While the number of studies constantly increases, many challenges remain to be faced relative to the research methods that are used. Reports have highlighted that poor-quality research is a major issue in the biomedical field and in oral health research2,3. Up to October 2022, the Catalogue of Bias had identified 62 different types of bias in biomedical research4 making it challenging to plan new studies. New biases and data security issues may continue to arise as an increasing amount of research involves digital sources and artificial intelligence to acquire and use information obtained from patients. Once the studies have been completed, open sharing of research data could enable data verification and reuse. Although calls for open science have intensified in recent years, the majority of authors are still reluctant to share their data, even when they indicate in their article that their data is available on request5. These are only some of the issues plaguing the modern research enterprise.
It is of crucial importance for the dental community to understand how dental research is planned, funded, conducted, reported, and disseminated. Moreover, the consequences of research biases, use of open science tools, and impact of dental research on society should be assessed. In such a complex scenario, meta-research studies play a key role.
Meta-research, i.e., research on research or methodological studies, could be considered a new discipline devoted to studying research practices. Results of meta-research studies (MRS) make it possible to identify problems and plan initiatives to qualify and disseminate good scientific practices6,7. MRS may include mixed research methods and present a variety of frameworks, including different objectives (e.g., assessing bias, methods, reporting practices, test interventions to improve research practices, or summarize knowledge), designs (cross-sectional, longitudinal, prospective, retrospective, or studies of interventions), units of analysis (types of study, analysis, records, or humans), and sampling strategies8.
A recent example of the importance of MRS could be illustrated throughout the COVID-19 pandemic. While a considerable number of articles were published, MRS emphasized the presence of research problems and the need to interpret study findings cautiously. The problems included redundant, poorly reported, and irreplicable systematic reviews9,10,11 and biased publications with the presence of spin and ethical issues12,13,14. It was shown that many researchers did not share their data openly at the beginning of the pandemic although they hurried to post their findings on preprint servers15. Cases such as these demonstrated how meta-research could help to improve the applicability of scientific findings during a public health emergency.
MRS studies in dentistry are still scarce. Faggion et al. identified that the majority of the 155 dental MRS published in 5 years focused on general dentistry questions, studied research methods, and the primary studies included in most MRS were randomized trials16. To date, few MRS have assessed in vitro studies and we still do not have a proper tool for critically appraising dental laboratory studies. Ioannidis et al. pointed out that many themes could be covered by meta-research, including methods, reporting, reproducibility, evaluation, and incentives6. These areas have still not been sufficiently explored in the dental science. This link https://osf.io/72pqg presents examples of how meta-research themes and topics have been investigated in dentistry, in addition to opportunities that are open for exploration. Apparently, we have a long road ahead to attain advancement with MRS in dentistry.
The dental research community needs to pay attention to this matter because meta-research scientists may face several challenges throughout their processes of research. The first challenge to overcome is to acquire sponsorship for MRS, as funding agencies and grant assessors may not be very receptive to meta-research proposals6. One reason for this barrier could be that sponsors and assessors assume that MRS do not require specific apparatus or laboratory structure, and consequently, funding may not be necessary. However, computers, software programs, file hosting and other online services are essential for this work, and the team involved in the research should be paid. Furthermore, there is prejudice against MRS, which has been described as not being “real research” or “not even medicine”17. Therefore, authors, journals, and editors play an essential role in demonstrating the importance of MRS to sponsors, for the improvement of dental science.
Another challenge is the lack of proper guidance to help researchers to plan and conduct meta-research. One example is the usual arbitrary selection of a relevant period (e.g., number of years) within which studies will be retrieved and data synthesized when evaluating a given subject. In this situation, authors could either conduct searches without time restrictions, use a specific time period (e.g., last 5 years), or two periods (e.g., before and after guideline publication). In this case, the main recommendations are to justify the choice of the period selected and verify whether there is any relevant event (e.g., publication of a guideline or checklist) that could have impact on the selection and results18. The time is ripe for the development of guidance on conducting evidence-based meta-research with emphasis on the methodological expectations.
The third challenge is related to the peer-review process. Editors and reviewers may be tempted to classify all meta-research studies as being systematic reviews and encourage authors to use the PRISMA reporting guideline, for instance19. However, meta-research is not merely a systematic review, even when systematic searches are conducted, using structured article screening and data extraction processes. A more recent initiative has been to develop a specific reporting guideline for MRS8,20, but this is also particularly challenging due to the complex and varied nature and possibilities in meta-science.
This editorial emphasizes the contemporary relevance of MRS in dentistry and challenges faced by meta-researchers. Meta-research plays a vital role in the advancement of oral health research, implementation of good research practices, and reduction of research waste. By doing so, MRS may improve the benefits of dental research to society and the value of dental research to people in general. Good MRS practices may lead to better research, which will ultimately enable better health care. Meta-research has been suggested as being our “best chance to defend science and gain public support for research”, thereby helping to antagonize anti-science movements7. This is a call to authors, editors, journals, and sponsors: not only do we need more and better MRS in dentistry, but we also need to understand their importance for the future of dental science.
Scopus. Documents in subject area dentistry. 2021. www.scopus.com.
Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;2016:e1002028.
Bassani R, Pereira GKR, Page MJ, Tricco AC, Moher D, Sarkis-Onofre R. Systematic reviews in dentistry: current status, epidemiological and reporting characteristics. J Dent. 2019;82:71–84.
University of Oxford. Catalogue of Bias [Internet]. Oxford: Centre for Evidence-Based Medicine; 2017 [cited 2022 Sep 26]. Available from: https://catalogofbias.org.
Gabelica M, Bojčić R, Puljak L. Many researchers were not compliant with their published data sharing statement: a mixed-methods study. J Clin Epidemiol. 2022;150:33–41.
Ioannidis JPA, Fanelli D, Dunne DD, Goodman SN. Meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 2015;13:e1002264.
Ioannidis JPA. Meta-research: why research on research matters. PLoS Biol. 2018;16:e2005468.
Lawson DO, Puljak L, Pieper D, Schandelmaier S, Collins GS, Brignardello-Petersen R, et al. Reporting of methodological studies in health research: a protocol for the development of the MethodologIcal STudy reportIng Checklist (MISTIC). BMJ Open. 2020;10:e040478.
Li Y, Cao L, Zhang Z, Hou L, Qin Y, Hui X, et al. Reporting and methodological quality of COVID-19 systematic reviews needs to be improved: an evidence mapping. J Clin Epidemiol. 2021;135:17–28.
Dotto L, Kinalski MA, Machado PS, Pereira G, Sarkis-Onofre R, Dos Santos M. The mass production of systematic reviews about COVID-19: an analysis of PROSPERO records. J Evid Based Med. 2021;14:56–64.
McDonald S, Turner S, Page MJ, Turner T. Most published systematic reviews of remdesivir for COVID-19 were redundant and lacked currency. J Clin Epidemiol. 2022;146:22–31.
Izcovich A, Peiris S, Ragusa M, Tortosa F, Rada G, Aldighieri S, et al. Bias as a source of inconsistency in ivermectin trials for COVID-19: a systematic review. Ivermectin’s suggested benefits are mainly based on potentially biased results. J Clin Epidemiol. 2022;144:43–55.
Wang D, Chen L, Wang L, Hua F, Li J, Li Y, et al. Abstracts for reports of randomized trials of COVID-19 interventions had low quality and high spin. J Clin Epidemiol. 2021;139:107–20.
Yogendrakumar V, Dewar B, McGuinty M, Dowlatshahi D, Dyason C, Kwok ES, et al. Many trials of hydroxychloroquine for SARS-CoV-2 were redundant and potentially unethical: an analysis of the NIH clinical trials registry. J Clin Epidemiol. 2022;143:73–80.
Strcic J, Civljak A, Glozinic T, Pacheco RL, Brkovic T, Puljak L. Open data and data sharing in articles about COVID-19 published in preprint servers medRxiv and bioRxiv. Scientometrics. 2022;127:2791–802.
Faggion CM, Listl S, Smits KPJ. Meta-research publications in dentistry: a review. Eur J Oral Sci. 2021;129:e12748.
Puljak L. Evidence synthesis and methodological research on evidence in medicine—why it really is research and it really is medicine. J Evid Based Med. 2020;13:253–4.
Puljak L, Babic A, Pieper D. Limiting the search period in methodological studies. J Clin Epidemiol. 2020;123:175–6.
Puljak L. Research-on-research studies or methodological studies are primary research. J Clin Epidemiol. 2019;112:95.
Puljak L. Reporting checklist for methodological, that is, research on research studies is urgently needed. J Clin Epidemiol. 2019;112:93.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Sarkis-Onofre, R., Sofi-Mahmudi, A., Puljak, L. et al. The importance of meta-research in dentistry. Evid Based Dent 24, 98–99 (2023). https://doi.org/10.1038/s41432-023-00880-w