Main

The Cochrane Collaboration (www.cochrane.org/) has led the way in setting standards for systematic reviewing. The Collaboration's database of systematic reviews, published as part of the Cochrane Library (The Cochrane Library is available free to residents in England and Wales through the National Electronic Library of Health www.nelh.nhs.uk and to clinicians in Scotland through www.elib.scot.nhs.uk. It is also available free in Australia, Ireland, Finland and Norway.), is established as a repository of high-quality information. Although the relevance of the systematic review database to dentists has been limited in the past, this is now changing rapidly, since over the past few years the work of the Cochrane Oral Health Group (www.cochrane-oral.man.ac.uk/) has seen the completion of 29 reviews over a wide cross-section of subject areas and over 100 more are in the pipeline.

The Cochrane Collaboration is constantly striving to improve the quality of their reviews and their standing has been shown to be high by a number of comparative studies.1, 2, 3, 4, 5 A more detailed study undertaken by experienced methodologists on a sample of reviews completed in 1998 was published in 20016 and found that overall the Cochrane reviews were of higher quality than other reviews, although some problems were still found. Unlike other systematic reviews and non-systematic or narrative reviews, those conducted by the Cochrane Collaboration are regularly updated and there is a facility to submit criticism. These facilities add to the robustness of the Cochrane approach, and indeed the paper by Olsen et al.6 has already resulted in changes to those reviews identified as having problems.

The Cochrane Library is updated quarterly and the first release of the library in 2003 saw the publication of eight new systematic reviews in the field of dentistry. They cover a range of topics, and all of them will be reviewed in the journal. The reviews of powered toothbrushes7 and fluoride toothpaste8 were released in January 2003. The different media response may seem a little odd at first for it was the review of powered toothbrushes that excited the media rather than the toothpaste review which showed the clearest evidence of effectiveness.

This review7 compared manual and powered toothbrushes in relation to the removal of plaque, the health of the gingivae, staining and calculus, dependability, adverse effects and cost. The reviewers concluded that powered toothbrushes with a rotation–oscillation action achieve a modest reduction in plaque and gingivitis compared with manual toothbrushing.

The lack of interest in the fluoride toothpaste review8 could be put down to various causes but it was probably due to the fact that the effectiveness of fluoride toothpastes in reducing caries is generally accepted. The media interest sparked by the powered-toothbrush review probably had as much to do with the battle for market share between two of the leading players in this sector (Gillette are makers of OralB and Royal Philips Electronics are the makers of Sonicare toothbrushes) as the actual results of review. Both companies issued press releases following the release of the review which placed emphasis on different aspects of the review. Gillette's press release was headlined, “Landmark study proves Oral-B® power toothbrush technology superior in reducing plaque and gingivitis” and, “Rotation oscillation toothbrushes proven more effective than ‘sonic’ technology”. Royal Philips Electronics, in contrast, led with, “Clinical Studies Prove Leading Power Toothbrushes Are More Effective Than Manual Toothbrushes, Contrary to Media Accounts of Recent Cochrane Report”.

Reports of the review rapidly spread throughout the world, appearing in many major papers and websites and even achieving a small piece in Newsweek. It is interesting to compare the manufacturers' views with those of the American Dental Association (ADA) whose article on their website is entitled, “Are power toothbrushes better? Cochrane group reviews controlled trials”, and the BBC website, which used a more negative headline, “Thumbs down for electric toothbrush”.

Actually, none of these interpretations of the Cochrane report is technically incorrect. However when the evidence is weak, incomplete, or the benefit small, it is more open to interpretation.

Looking at the review in more detail, the counter rotational brushes provided 7 and 17% reductions in plaque and gingivitis respectively, over and above manual brushes. Therefore it can be said that the brush is more effective than a manual one. The ADA recommends,9 however, that only oral care products that can prove a 20% improvement may publish a claim for superiority over other products.

The included studies are also of short duration, so we do not know whether this benefit is maintained over time, and as the improvements in plaque control and gingivitis over manual brushes are small they may be of borderline benefit to dental health. In addition, the available evidence only supports the marginal superiority of one type of powered brush. The patients or consumer may then legitimately ask, “If the difference is not that great and they cost that much more, should I get one?” Because of this, the questioning headlines of the ADA and the more negative spin of the BBC headline are valid. The increased brushing effectiveness may not be the only reason for buying a powered brush, however, because many patients find the larger handles better to use and smaller head sizes more comfortable than a manual brush.

The powered toothbrush review, as with so many of the Cochrane reviews, raises as many questions as it answers. The review was challenged by Royal Philips Electronics regarding many of the decisions taken during the review process and both the questions and the responses can be seen on the Cochrane Oral Health Group website (www.cochrane-oral.man.ac.uk/). One of the strengths of the Cochrane approach to reviews is the clear, explicit methodology and the ability of anyone to make comments about the reviews that are considered when reviews are updated. The updating usually takes place after 2 years and is an important feature of them.

Although the review does provide an answer, it highlights the poor quality of the toothbrush trials upon which every company's marketing claims are based. This is despite this being a huge industry, with an estimated $5 billion worldwide market in manual and power toothbrushes. The quality shortcomings in the majority of research in dentistry is a subject I have touched on in editorials before and these issues arise again in the toothbrush review. Recent trials did not adhere to modern reporting standards such as the CONSORT guidelines (see www.consort-statement.org), and there is no collective agreement on measuring outcomes that would enable different studies to be compared, whether this is between researchers or companies. Another issue is that there is currently no way of telling whether companies have suppressed negative trials that they have conducted, a situation that will only be resolved when all trials are publicly registered at the outset, just as some forward-looking pharmaceutical companies are now doing.