Reliability of Medline searches for dental randomised controlled trials

Key Points

• Describes the importance of locating evidence for clinical practice.

• Gives the search validity on Medline for different areas of dental research.

• Demonstrates the usefulness of Medline for rapid location of randomised controlled trials.

• Discusses possible problems in locating relevant randomised controlled trials on Medline.

Abstract

Objective To determine the validity of Medline searches for randomised controlled trials in dental research (RCT-Ds) using the medical subject headings (MeSH-terms).

Design The Medline database was searched for randomised controlled trials in dental research (RCT-Ds) published in 1999 and with MeSH-terms corresponding to different areas of dental research. All RCT-Ds were manually examined for relevance to the different areas of dental research and cross-tabulated against the Medline search results. The sensitivity, specificity, positive (precision) and negative predictive values, as well as the accuracy of the search results were calculated.

Results The highest validity in the Medline searches for RCT-Ds was seen for endodontics, followed by orthodontics, whereas the lowest validity was seen for pediatric dentistry and public health dentistry. For pediatric dentistry the MeSH-term searches had too low a sensitivity for adequate location of RCT-Ds.

Conclusions MeSH-term searches on Medline are a useful tool for rapid location of RCT-Ds in most areas of dental research. However, there is a vast variation in the search validity. More refined search strategies are required to locate RCT-Ds in areas of dental research with low search validity.

Main

Medline search validity for randomised controlled trials in different areas of dental research P Sjögren and A. Halling Br Dent J 2002; 192: 97–99

Amendment We would like to inform readers that the wrong name was credited to a recent research commentary. The comment in British Dental Journal 2001; 191: 563, was provided by Dr Heather Pontefract, Department of Oral and Dental Science, Bristol and not Martin Addy as stated.

Comment

A pre-requisite for evidence-based healthcare research is a comprehensive unbiased search for randomised controlled trials (RCTs) and this will include an electronic search of Medline. How successful we are in searching Medline will largely depend upon two key elements; namely, the National Library of Medicine's highly disciplined indexing system and the search terms we apply to search the database. Crucial to the process of indexing is the author/s' use of terms and descriptors. Lack of detail here may affect the quality of indexing and consequently the results of the search.

This paper by Sjögren and Halling examines the accuracy of Medline searches for identifying RCTs in different dental specialties. The authors conducted limited Medline searches and examined the retrieved reports to determine whether they were reports of RCTs and related to the specialty being searched. This is an important issue but fails to address the problem that, even when conducted by experienced searchers using optimally sensitive search strategies, Medline searches only detect, on average, 51% (range 17% – 82%) of all known RCTs.¹

The use of the term validity in this study must be questioned. Dickersin et al.¹ define validity, when related to searching for RCTs, as Number of RCTs identified / Total number of RCTs. Thus, when assessing the validity of electronic searches, they must be compared with a 'gold standard' which essentially must include handsearching the literature.

The authors' conclusion, that Medline searches based on MeSH terms provides an overall adequate method of identifying RCTs related to the dental specialties also has to be examined. This is because, although this study showed that most of the 'hits' from the searches were reports of RCTs related to the relevant speciality, it did not take into account the number of RCTs missed by the searches. Consequently any conclusions drawn from potentially incomplete searches are likely to be biased.

The Cochrane Collaboration has done much to assist researchers in evidence based healthcare by developing and applying to Medline a highly sensitive search strategy to identify reports of RCTs, and making these citations (along with searches from other databases and from handsearching journals) available through the Cochrane Controlled Trials Register (CCTR) in the Cochrane Library.² The Cochrane Oral Health Group's Specialised Register of reports of trials on oral health was similarly founded and currently has in excess of 11,000 citations.