Commentary

This study looked at the top ranking journals from each of 6 dental disciplines and extracted all the randomised controlled trials (RCTs) from each in their last 24 issues. They then assessed the trial reports according to a modified CONSORT checklist. The CONSORT statement is a reporting guideline for researchers conducting RCTs, which may also serve as a means for a reader to critically appraise a report.1, 2

The title of this paper demonstrates a flaw in this study as CONSORT is not a means to measure the quality of a trial, though this may, arguably, be reflected in the attention paid to its reporting.3 The authors used a scoring system, but this has not been validated as far as I can see and there is now unease about using scoring systems when assessing quality of studies.4 It seems logical that we should be circumspect about using them in reports of studies too.

However the percentage of the 95 trials adequately describing each of the 30 items from the checklist does allow one to see the omissions in the reporting of different stages in trial design and conduct for the group of trials. (Unfortunately we are unable to see this for each journal).

Only 50% of the 95 RCTs adequately reported a sample size calculation. Even though it has been argued that an underpowered study otherwise conducted with methodological rigour is better than no study at all, (as it can be combined with other studies at low risk of bias in a meta-analysis),5 the process of calculating an appropriate sample size – even with inaccurate estimates of effect size – makes scientific and ethical sense1 since it would seem to be more likely to produce a statistically significant result than if no calculation were done at all.

The finding that just 34% of the trials adequately described the generation of an unpredictable allocation sequence is probably the biggest shortcoming of this collection of trials. Randomisation eliminates selection and confounding biases, may allow blinding of the interventions to the patient, participant and assessor and allows researchers to apply probability theory to express the likelihood that any difference in outcome between intervention groups simply reflects chance.6 Every trial ought to be conducting and reporting randomisation procedures appropriately.

On the other hand the meaning of only 22% of trials describing adequately the allocation concealment is hard to interpret given that in the included RCTs it might have been impossible to blind the operator and patient to the intervention allocation, given the physical nature of many dental interventions. Likewise, the 84% of trials that did not adequately describe blinding of assessors may reflect either poor reporting or the impossibility of blinding an assessor when the intervention was clear in the patient's mouth. I suspect I am being too optimistic, but the study could have been improved by reporting itself on whether allocation concealment, blinding and the other items should have been expected for each trial.

The CONSORT checklist is a guide and a critical appraisal tool and its use as a scoring system ought to be avoided. In the interest of transparency, though, authors of trial reports could be encouraged to state why any items in the checklist were not reported, thus making the reader's task of understanding conformity to CONSORT more straightforward.

What do the results of this study mean for the clinician who encounters a RCT? Be familiar with the CONSORT checklist and other appraisal tools (e.g. CASP - http://www.sph.nhs.uk/what-we-do/public-health-workforce/resources/critical-appraisals-skills-programme), use your judgement in applying them and don't take trial quality for granted.