Send your letters to the Editor, c/o Kim Black-Totham, Evidence-based Dentistry Nature Publishing Group, The Macmillan Building, 4 Crinan Street, London N1 9XW E-mail: ebd@nature.com

Polyol-containing chewing gums: flawed evidence

Twetman1 based conclusions on “an excellent systematic review performed ... to high standards.”2 Unfortunately, Twetman seems no better able to distinguish rigorous reviews from flawed ones than Deshpande and Jadad.2 The reality is that the review was not even close to excellent, nor the standards even close to high. To start with, the conflict of interest is obvious when one author selects his own rating system for trial quality, despite the fact that this rating system has been amply demonstrated to be fatally flawed3, 4 because of its confusing necessity with sufficiency. Yes, randomisation, masking and appropriate handling of withdrawals are necessary, but they certainly are not sufficient, even taken together. There are many other key measures of trial quality that the Jadad score ignores altogether. For example, was the statistical analysis appropriate? The use of the Jadad score spares the reviewer from having to concern himself or herself with such trivialities, and clearly represents an effort to unburden the reviewer from having to do any real work to evaluate trial quality. The result is obvious. Beyond this, the convention that even a Jadad-3 trial is high quality says that these bare-minimum standards are not even necessary; we can compromise on two of them, as if there is some compensation in getting the other three right. Not that they even need to be right. Randomisation, for example, may be completely inappropriate (such as permuted blocks of size two in a trial labelled as masked but clearly unmasked by differential tastes or rates of specific side effects) and still receive full credit. So were there any high-quality trials on which to base the conclusions regarding the impact of polyol-containing chewing gums on dental carries?

Deshpande and Jadad's Table 12 reveals that only one study5 was rated a perfect Jadad-5: this trial was randomised by school, not by individual. There were five treatments and five schools, meaning that there was no true replication. Of course the schools differed from one another, and the design completely confounds treatment effects with school effects. The significant baseline imbalances across treatment groups (schools) in this one “perfect” trial5 may mean that other factors, not considered, also differed across schools (and, hence, across treatment groups). For example, did the students at some schools tend to floss more thoroughly than those at other schools? Did this baseline difference lead to the observed differences in dental carries across treatment groups which was uncritically attributed to the gums themselves? If not flossing, then some other feature of oral health that was emphasised more in some schools than in others?

Clearly, this is a possibility, and the result is that we have no basis for moving from our initial state of uncertainty regarding the impact of polyol-containing chewing gums on dental carries, which is bad enough. To pretend that we actually know more than we do is a disservice to your readers, and the record needs to be set straight. So what do we know? Certainly none of the trials were high quality. The evaluation of the quality of these trials was a failure, and the evaluation of the evaluation just carried this failure one step further.

Authors response

Dr Berger has made many good points and, as stated in my short commentary, I share much of his scepticism – for example, concerning the applied inclusion criteria and quality assessment.1 A first point is that I do not consider my initial sentence to be a “conclusion” but rather a descriptive introduction of the work. Systematic reviews are performed by various health technology agencies and research groups according to different standards, but a common mantra is that a systematic review will never be better than the research upon which it is based. I do think that the paper of Despande and Jadad2 represents a rigorous approach that is presented in a transparent way within its given methods. The authors have accepted and rated a number of studies with a high proportion of possible bias and confounders but this is done in a predetermined and defined way. This methodology of course can, and should, be discussed; the issues brought up by Dr Berger are highly relevant. In my commentary, several limitations are addressed and the conflicts of interest within the industry are stressed. It is almost a coincidence to find that the European Food and Safety Authority have recently published a scientific opinion that supports the claim that, “xylitol chewing gums reduces the risk of caries in children” 3 The statement is based upon 31 intervention studies, two observational studies and 16 narrative reviews, and is compiled by an expert panel at the request of chewing gum producers.

It is unfortunate if my commentary is amplifying poor “research” but a view should be taken by the readers themselves. One thing is clear: the scientific controversies regarding polyol-containing chewing gums in caries prevention seem to be an eternal issue.