Pubmed, Embase, Cochrane Central Register of Controlled Trials and the Web of Science databases. Hand searches of the journals European Journal of Orthodontics, Journal of Orthodontics, Journal of Clinical Orthodontics, Seminars in Orthodontics, American Journal of Orthodontics & Dentofacial Orthopaedics and Angle Orthodontist.
Two reviewers independently selected studies. Randomised controlled trials (RCTs) and controlled clinical trials (CCTs) of orthodontic patients requiring extraction of the maxillary first premolars and closure of the spaces without anchorage loss were considered.
Data extraction and synthesis
Data extraction and risk of bias assessment were carried out independently by two reviewers. Meta-analysis and sensitivity analysis were conducted.
Fourteen studies; seven RCTS and seven CCTs were included. In total 303 patients received TISADs with 313 control patients. Overall the quality of the studies was considered to be moderate. Overall the TISAD group had significantly less anchorage loss than the control group. On average, TISADs enabled 1.86mm more anchorage preservation than did conventional methods.
The results of the meta-analysis showed that TISADs are more effective than conventional methods of anchorage reinforcement. The average difference of 2mm seems not only statistically but also clinically significant. However, the results should be interpreted with caution because of the moderate quality of the included studies. More high-quality studies on this issue are necessary to enable drawing more reliable conclusions.
The objective of the systematic review by Antoszewska-Smith and colleagues1 was to compare the effectiveness of orthodontic mini-implants (OMIs) as anchorage devices with conventional orthodontic anchorage methods in patients in need of space closure of extracted maxillary first premolars without losing molar anchorage. The difference in anchorage loss, ie mesial movement of the maxillary first molars, between these techniques was the primary outcome measure. For this commentary we assessed the quality of the systematic review. We used the AMSTAR and ROBIS tools to score respectively the methodological validity and the risk of bias in the systematic review.2,3,4,5 These instruments were applied independently to this review by the two authors (RMR and LD) of this commentary.
Limitations of the systematic review
Table 3 presents the rationale for the AMSTAR and ROBIS scores and also lists additional limitations of the review. These issues are explained here.
Prior to starting a systematic review investigators should (1) prioritise their research questions with pertinent stakeholders to assess whether their planned questions are necessary; 6 (2) assess whether the review has already been done previously. If so, authors should assess whether a new review is indicated, for example in the context of the limitations of the earlier review. The main text and the references of the review by Antoszewska-Smith and co-authors1 showed that they did not undertake such assessments. For example, they did not consider a recent Cochrane review by Jambi and co-workers7 that asked similar research questions. We consulted the PROSPERO8 register and various online protocol repositories,9,10 but were unable to identify a protocol of the systematic review by Antoszewska-Smith and co-authors.1 Not registering or publishing of protocols can introduce various biases such as selective reporting and publication of outcomes.11 Pilot testing of any of the research methods was also not reported in our appraised review.
Carefully defined eligibility criteria permit the reproducibility of a review and reduce sources of medical uncertainties such as the variability in participants, interventions, comparators, outcomes etc. Numerous eligibility criteria in this review were either not defined or were incomplete. For example: (1) a definition of eligible controlled clinical trials was missing; (2) eligible characteristics for participants such as age, sex, and other demographic items were not given. Restrictions for the type of setting were also not reported; (3) reporting on both the interventions and comparators was suboptimal. For example, the characteristics of eligible interventions with OMIs and conventional anchorage methods were not defined; (4) An eligible duration of treatment was not reported; (5) A clear time point for measuring outcomes was also not defined. It was not reported whether closure of extraction spaces with or without paralleling the roots was considered as the endpoint of treatment; (6) Eligible methods for measuring outcomes, eg model or cephalometric analyses were also not reported; (7) Language bias was a problem, because only articles reported in the English language were eligible
The authors screened a wide spectrum of electronic databases and also manually searched various pertinent journals and the references of included studies. However, they did not consult: (1) the Grey literature; (2) researchers and sponsors to obtain information on unknown or ongoing studies; (3) references of review articles on OMIs. The search strategy was probably sufficient, because it covered the same keywords as other systematic reviews on OMIs.7,12 However, we do not know whether this search strategy was pilot tested and whether an information specialist with expertise in searching the biomedical literature was consulted to validate the search strategy. A list of excluded studies with the rationale for exclusion was also not reported. Not including such a list makes the reproducibility of the review impossible and also introduces study selection bias.13
For the assessment of the quality of the eligible controlled clinical trials studies, the reviewers applied the Newcastle-Ottawa scale. This instrument lacks comprehensive manuals with instructions for users, which has resulted in low reliability between reviewers.14,15,16 To deal with these shortcomings, the Cochrane Bias Methods Group and the Cochrane Non-Randomised Studies Group developed a new tool for assessing risk of bias in non-randomised studies of interventions (ROBINS-I tool).17This instrument, previously known as the ACROBAT-NRSI tool18 should have been used for the bias assessment in the eligible controlled clinical trials. The authors assigned the quality of evidence as moderate for both the included randomised controlled trials and the controlled clinical trials, but this score was based on test methods developed by the authors themselves. Instead, they should have implemented a validated instrument such as the GRADE approach for assessing the quality of evidence.19
It was unclear whether conducting meta-analyses was indicated, because definitions of eligible outcomes were underreported in the eligibility criteria and in the review as a whole. It was therefore unclear whether all studies included in the meta-analyses had measured the same outcomes and with the same test instruments. It was also unclear whether outcomes on failed OMIs were included in the intervention groups. This is important, because excluding these outcomes can seriously upgrade the effectiveness of these devices. The same issue applies to excluding outcomes on poor headgear collaborators from the comparator groups. The authors also included a non-eligible study20 in two of the meta-analyses for a secondary outcome. The inclusion of this study is particularly problematic, because it skewed the meta-analysis on tipping of molars in favour of OMIs.
The authors of the appraised review did not assess any outcome on adverse effects of interventions. When considering the implementation of a new health technology, clinicians want to know both the benefits and the adverse outcomes of the intervention of interest. It is therefore mandatory in Cochrane reviews to assess the findings of at least one adverse effect as a primary outcome.
Competing interests can influence how research studies are designed, conducted and reported, which could divert outcomes away from the truth. In addition, it has been estimated that around 50% of the studies that involve researchers with conflicts of interests do not declare them.21 Full transparency on the role of a sponsor or funder during any part of the review process is therefore key.22 A statement on potential conflicts of interest was not included in the appraised review by Antoszewska-Smith and co-workers.1
Prior to applying the findings of a systematic review on a specific intervention to a patient, clinicians need to exclude numerous uncertainties. The assessment of the quality of the review is an initial step in this process. Quality assessments of systematic reviews are important as was explained in a recent investigation by Ioannidis.23 He suggested that 'possibly the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, and/or conflicted'.23 These three limitations were also found in the systematic review by Antoszewska-Smith and co-workers.1 First, the review reported no information on potential conflicts of interests. Second, assessments with both the AMSTAR and ROBIS tools identified numerous limitations. These shortcomings could have been resolved either during the protocol phase or during the conduct and reporting of the review. Supplementary files published online should have expanded on many of the underreported items. A good collaboration between editors, peer-reviewers, and the authors of this review could also have significantly raised its quality. Now only a new unbiased review team can address these shortcomings in a new systematic review. Third, before developing and conducting a new review one should first assess whether addressing the current research questions is really necessary, because the effectiveness of non-moving implants for anchorage purposes is obvious and probably does not need further research. Clinicians and patients are possibly more interested in reviews that assess how displacements of OMIs and therefore anchorage loss can be avoided. Prioritising review questions with pertinent stakeholders is the first step when considering a new review.6
Critical appraisals with the AMSTAR and ROBIS tools conducted by two reviewers independently, identified numerous limitations in this systematic review. These shortcomings should be carefully weighed prior to implementing the findings of this low quality review to our patients
Research on the obvious effectiveness of non-moving implants for anchorage purposes is probably not what patients and clinicians want. Prioritising review questions with pertinent stakeholders is indicated before designing and conducting a new review on orthodontic mini-implants and comparator interventions.
Antoszewska-Smith J, Sarul M, Łyczek J, Konopka T, Kawala B . Effectiveness of orthodontic miniscrew implants in anchorage reinforcement during en-masse retraction: A systematic review and meta-analysis. Am J Orthod Dentofacial Orthop 2017; 151: 440–455.
University of Bristol. Risk of bias in systematic reviews (ROBIS). ROBIS tool and the ROBIS guidance document. Available at http://www.bristol.ac.uk/social-community-medicine/projects/robis/ (accessed August 2017).
Shea B J, Grimshaw J M, Wells G A et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007; 7: 10.
Shea BJ, Hamel C, Wells GA et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol 2009; 62: 1013–1020.
Whiting P, Savovic J, Higgins JP et al. ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol 2016; 69: 225–234.
Cowan K and Oliver S . The James Lind Alliance Guidebook, 2013. Version 5. Available at http://www.jlaguidebook.org/pdfguidebook/guidebook.pdf (accessed August 2017).
Jambi S, Walsh T, Sandler J, Benson P E, Skeggs R M, O'Brien K D . Reinforcement of anchorage during orthodontic brace treatment with implants or other surgical methods. Cochrane Database Syst Rev 2014; 8: CD005098. DOI:10.1002/14651858.
University of York. PROSPERO: Centre for Reviews and Dissemination. Available at http://www.crd.york.ac.uk/PROSPERO/ (accessed August 2017).
OpenDOAR. The directory of Open Access Repositories (Open DOAR). Available at http://www.opendoar.org/ (accessed July 2017).
Registry of Open Access Repositories (ROAR). Available at http://roar.eprints.org/ (accessed August 2017).
Kirkham J J, Dwan KM, Altman D G et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ 2010; 340: c365.
Reynders R, Ronchi L, Bipat S . Mini-implants in orthodontics: a systematic review of the literature. Am J Orthod Dentofacial Orthop 2009; 135: 564.e1–e19.
Goodman S N, Fanelli D, Ioannidis J P . What does research reproducibility mean? Sci Transl Med 2016; 8: 341ps12.
Hartling L, Hamm M, Milne A et al. Validity and inter-rater reliability testing of quality assessment instruments. (Prepared by the University of Alberta Evidence-based Practice Center under Contract No. 290-2007-10021-I.) AHRQ Publication No. 12-EHC039-EF. Rockville, MD: Agency for Healthcare Research and Quality, 2012.
Hartling L, Milne A, Hamm M P et al. Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers. J Clin Epidemiol 2013; 66: 982–993.
Mertz D, Loeb M . Newcastle-Ottawa Scale: comparing reviewers' to authors' assessments. BMC Med Res Methodol 2014; 14: 45.
Sterne J A, Hernán M A, Reeves B C et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016; 355: i4919.
Sterne J A C, Higgins J P T . Reeves B C on behalf of the development group for ACROBAT-NRSI. A Cochrane Risk Of Bias Assessment Tool: for Non-Randomized Studies of Interventions (ACROBAT-NRSI), Version 1.0.0, 24 September 2014. Available at http://www.riskofbias.info (accessed August 2017).
Schünemann H, Brozek J, Gyatt G, Oxman A . GRADE handbook. Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach. Updated October 2013. Available at http://gdt.guidelinedevelopment.org/app/handbook/handbook.html (accessed August 2017).
Victor D, Prabhakar R, Karthikeyan M K et al. Effectiveness of mini implants in three-dimensional control during retraction - a clinical study. J Clin Diagn Res. 2014; 8: 227–232.
Dunn A G . Set up a public registry of competing interests. Nature. 2016; 533: 9.
Shamseer L, Moher D, Clarke M et al.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015; 349: g7647.
Ioannidis J P . The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. Milbank Q 2016; 94: 485–514.
Address for correspondence: Jan Łyczek, Department of Orthodontics and Dento- facial Orthopedics, Faculty of Dentistry, Wroclaw Medical University, Wroclaw 52-020, Poland. e-mail: firstname.lastname@example.org.
Antoszewska-Smith J, Sarul M, Łyczek J, Konopka T, Kawala B. Effectiveness of orthodontic miniscrew implants in anchorage reinforcement during en-masse retraction: A systematic review and meta-analysis. Am J Orthod Dentofacial Orthop 2017; 151: 440–455. doi: 10.1016/j.ajodo.2016.08.029. Review. PubMedPMID: 28257728.
About this article
Cite this article
Reynders, R., Ladu, L. Mini-implants for orthodontic anchorage. Evid Based Dent 18, 82–85 (2017). https://doi.org/10.1038/sj.ebd.6401257