Summary Review/Methodology

Evidence-Based Dentistry (2011) 12, 52–53. doi:10.1038/sj.ebd.6400796

Quality of reporting randomised controlled trials in major dental journals suboptimal

Questions: Can standards of recording and reporting of Randomised Controlled Trials (RCTs) in six dental specialties be improved?

Dominic Hurst1

1Queen Mary University of London, Barts & The London School of Medicine and Dentistry, Institute of Dentistry, London, UK.

Pandis N, Polychronopoulou A, Eliades T. Evaluation of the quality of randomised clinical trials published in leading dental specialty journals. J Dent 2010; 38: 713–721.

Address for correspondence: T Eliades, 57 Agnoston Hiroon, Nea Ionia 14231, Greece. E-mail: teliades@ath.forthnet.gr

Top

Abstract

Data sources

 

Hand searching of the most recent 24 issues of six high impact dental journals.

Study selection

 

RCTs involving only humans, from 24 issues of six leading specialty journals, covering a period up to July 2009 were included, including cluster randomised trials.

Data extraction and synthesis

 

Each article included in the study was assessed and scored independently by two observers, with any discrepancies being resolved by a third observer. In this study the modified CONSORT checklist was used to score each applicable item of data. The sum of the scores was converted to a percentage value for each trial. The data were analysed using descriptive statistics. Data regarding the publishing journal, country of origin of the trial, number of authors, involvement of statistician/epidemiologist, number of centres involved, ethics committee approval were subject to quantitative analysis.

Results

 

Ninety-five RCTs were identified with, according to the authors, generally suboptimal scores on quality reporting on key CONSORT areas. Significant differences in scores were found among the journals covering the named specialties. Overall there was a positive association between the quality score in studies with more authors, multicentre studies and studies in which a statistician/epidemiologist was involved.

Conclusions

 

The overall quality of reporting RCTs in major dental journals was considered suboptimal in key CONSORT areas. This is very important as the reported results of RCTs can have an impact on future patient care.

Top

Commentary

This study looked at the top ranking journals from each of 6 dental disciplines and extracted all the randomised controlled trials (RCTs) from each in their last 24 issues. They then assessed the trial reports according to a modified CONSORT checklist. The CONSORT statement is a reporting guideline for researchers conducting RCTs, which may also serve as a means for a reader to critically appraise a report.1, 2

The title of this paper demonstrates a flaw in this study as CONSORT is not a means to measure the quality of a trial, though this may, arguably, be reflected in the attention paid to its reporting.3 The authors used a scoring system, but this has not been validated as far as I can see and there is now unease about using scoring systems when assessing quality of studies.4 It seems logical that we should be circumspect about using them in reports of studies too.

However the percentage of the 95 trials adequately describing each of the 30 items from the checklist does allow one to see the omissions in the reporting of different stages in trial design and conduct for the group of trials. (Unfortunately we are unable to see this for each journal).

Only 50% of the 95 RCTs adequately reported a sample size calculation. Even though it has been argued that an underpowered study otherwise conducted with methodological rigour is better than no study at all, (as it can be combined with other studies at low risk of bias in a meta-analysis),5 the process of calculating an appropriate sample size – even with inaccurate estimates of effect size – makes scientific and ethical sense1 since it would seem to be more likely to produce a statistically significant result than if no calculation were done at all.

The finding that just 34% of the trials adequately described the generation of an unpredictable allocation sequence is probably the biggest shortcoming of this collection of trials. Randomisation eliminates selection and confounding biases, may allow blinding of the interventions to the patient, participant and assessor and allows researchers to apply probability theory to express the likelihood that any difference in outcome between intervention groups simply reflects chance.6 Every trial ought to be conducting and reporting randomisation procedures appropriately.

On the other hand the meaning of only 22% of trials describing adequately the allocation concealment is hard to interpret given that in the included RCTs it might have been impossible to blind the operator and patient to the intervention allocation, given the physical nature of many dental interventions. Likewise, the 84% of trials that did not adequately describe blinding of assessors may reflect either poor reporting or the impossibility of blinding an assessor when the intervention was clear in the patient's mouth. I suspect I am being too optimistic, but the study could have been improved by reporting itself on whether allocation concealment, blinding and the other items should have been expected for each trial.

The CONSORT checklist is a guide and a critical appraisal tool and its use as a scoring system ought to be avoided. In the interest of transparency, though, authors of trial reports could be encouraged to state why any items in the checklist were not reported, thus making the reader's task of understanding conformity to CONSORT more straightforward.

What do the results of this study mean for the clinician who encounters a RCT? Be familiar with the CONSORT checklist and other appraisal tools (e.g. CASP - http://www.sph.nhs.uk/what-we-do/public-health-workforce/resources/critical-appraisals-skills-programme), use your judgement in applying them and don't take trial quality for granted.

Top

References

  1. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Brit Med J 2010; 340: c869. | Article | PubMed |
  2. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Brit Med J 2010; 340: c332. | Article | PubMed |
  3. Richards D. RCTs in periodontology do not meet current recommendations on quality. Evid Based Dent 2003; 4: 3. | Article |
  4. Juni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. J Am Med Assoc 1999; 282: 1054–1060. | Article | ISI | ChemPort |
  5. Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet 2005; 365(9467):1348–1353. | Article |
  6. Schulz KF. Randomized controlled trials. Clin Obstet Gynecol 1998; 41: 245–256. | Article | PubMed | ISI |