Evidence-based practice

The concept of using research evidence to inform healthcare involving oral medicine, has a long history. One of the earliest accounts of research being undertaken to explore treatment options is the comparative clinical trial reported in James Lind's Treatise of the scurvy, published in 1753.1 Lind was a surgeon on HMS Salisbury. His book details his comparison of interventions for the treatment and prevention of scurvy, along with a critical and chronological account of what had been previously published on scurvy. Scurvy has multisystem manifestations including a number of oral features such as swollen and friable gingivae and spontaneous gingival haemorrhage. At the time it was a common disease among long distance sailors. Lind's work identified the superiority of the citrus fruits over other proposed treatments and, although recognised as important, it took more than 40 years before the results of Lind's experiments were acted upon; this time lag between research findings and changes in practice is still apparent today.2

This paper will briefly explore how the use of research evidence has developed since then to inform clinical practice focusing on oral medicine. It will also introduce the concept of core outcome sets to help improve future trial data reporting to allow comparison, contrast and combination as appropriate.

Despite the historic use of evidence in informing practice, the term 'evidence-based practice' (EBP) is relatively new, appearing initially as evidence-based medicine in 1992. Over the years EBP has evolved to mean many things to many people. Perhaps the most frequently used and feasible definition of EBP (first applied to evidence-based medicine) is: 'The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.'3

EBP is not about applying the results of research to all clinical settings and all patients without careful thought and evaluation. Instead it promotes the integration of best available research evidence with the individual clinician's experience and clinical judgement. In addition to this, the patient's expectations and preferences need to be taken into the decision-making process; a factor often not considered. Only when research evidence, clinical expertise and patient values are considered together can practice be considered evidence-based.

Clearly, in order to practice EBP, there needs to be a strong evidence base. That is, there needs to be relevant, valid research. Despite the promotion of EBP over the last 30 years, it has been estimated that only 15% of all clinical practice is based on sound research evidence.4 Variations in practice and the provision of inappropriate care continue. The incorporation of research evidence into the clinical decision making process may be hampered by several factors,5 not least the lack of reliable research addressing a clinical issue.

It has been suggested that the most realistic use of EBP by practitioners at the point of care involves the use of summaries of research evidence.5 Sources of secondary research, such as systematic reviews and clinical guidelines, may indeed have an important role to play in helping to close the gap between research evidence and clinical practice,2 providing they themselves are well conducted, address a clinically important question and that there are reliable studies addressing the specified questions.

The Cochrane Collaboration

One organisation aiming to improve the use of research evidence in clinical practice is The Cochrane Collaboration. Cochrane is a global independent network of researchers, professionals, patients, carers, and people interested in health. There are more than 37,000 contributors from over 130 countries. They aim to produce credible, accessible health information that is free from commercial sponsorship and other conflicts of interest. Cochrane believe the need to produce high-quality systematic reviews of research evidence is of increasing importance: 'As access to health evidence increases, so do the risks of misinterpreting complex content; meanwhile the likelihood of any one person getting a complete and balanced picture decreases' (http://www.cochrane.org/about-us). Cochrane is internationally recognised as the benchmark for high-quality information about the effectiveness of healthcare. It focuses predominantly, but not exclusively, on systematic reviews of randomised controlled trials (RCTs). It is acknowledged, however, that RCTs may not be the most appropriate study design to evaluate every clinical research question. Clinicians, policy makers and researchers need to be able to recognise the merits of different study designs in primary research for answering different types of clinical questions (whether they deal with evaluating the effectiveness of preventative or therapeutic interventions, the diagnosis of a particular disease or condition, incidence or prevalence, or perhaps cost-effectiveness of a defined management strategy).6

Cochrane Oral Health (COH)

Cochrane is made up of over 50 Review Groups, of which COH is one (http://oralhealth.cochrane.org).7 The scope of COH is to undertake systematic reviews (predominantly of randomised controlled trials) covering the prevention, treatment and rehabilitation of oral, dental and craniofacial diseases and disorders. The group has a worldwide network of over 1,650 members from 42 different countries. It maintains a register of references to clinical trials within the scope of the group. This register currently contains around 33,000 references. To date, COH have 154 published systematic reviews and 45 protocols. Their reviews have been used to inform guideline development by organisations such as the National Institute for Health and Care Excellence (NICE), the Scottish Intercollegiate Network (SIGN), the American Dental Association and the Scottish Dental Clinical Effectiveness Programme (SDCEP).

COH are committed to producing high-quality reviews that address priority clinical questions. They are currently working on an international priority setting exercise to ensure they address questions that are most useful for informing clinical decision-making by consumers and clinicians alike. In a previous COH prioritisation exercise carried out in 2014, oral medicine conditions featured in two distinct areas:

  1. 1

    Oral cancer (screening, diagnostic tests, clinical assessment)

  2. 2

    Oral conditions (temporomandibular disorders (TMD), lichen planus, leukoplakia, denture stomatitis).

In undertaking the reviews, COH are keen to ensure they work closely with all relevant stakeholders, in particular guideline developers, to ensure there is minimal duplication of effort and maximum uptake of the findings from their systematic reviews.

Oral medicine and evidence base

There are a number of Cochrane reviews in the field of oral medicine, covering a variety of topics (see Table 1). A key criticism of Cochrane reviews, particularly within dentistry, is that there are insufficient trials for the reviews to be useful. However, in the majority of oral medicine reviews, this is not the case with the number of trials included in the latest versions of the reviews ranging from 10–131. Despite the large number of trials in oral medicine, the results produced are often too heterogeneous to be utilised to inform clinical practice.

Table 1 Oral-medicine-related Cochrane reviews
Full size table

For example, with regard to recurrent aphthous stomatitis (RAS), there are over 25 trials in a review evaluating systemic interventions and 77 trials to date in an ongoing review evaluating topical interventions.8 The evidence base presented in each review is limited by several factors. Firstly, there is a wide range of interventions being assessed. Twenty-five trials of systemic interventions provide evidence on 21 different interventions; 77 trials of topical interventions provide evidence on over 60 interventions. In most cases, each intervention was assessed in only a single trial; where more than one trial evaluated the same intervention there was variation in dose and duration of the intervention and choice of control group. Such clinical heterogeneity precludes pooling of data. Often, interventions were evaluated with little clinical justification or discussion of biological plausibility and suggested mode of action.

In addition, there was substantial heterogeneity in type and timing of outcome assessment. When evaluating ulcers, this outcome was reported variously as number of ulcers, number of episodes, duration of ulcers/healing time, complete healing, size of ulcers, presence of ulcers, erythema, oedema, exudation, compound/summated RAS index, ulcer severity, site of ulcers, effectiveness index, onset of prodromal phase and recurrence. Other outcomes evaluated include pain (measured in a variety of ways), eating and drinking experience, brushing experience, tolerability/satisfaction with medication, recovery of function, adherence, and daily activity disturbance. The timing of assessments also varied. Some trials reported single episodes of ulceration and others reported multiple episodes (presenting data either cumulatively or per episode). Timeframes were as short as eight hours or as long as six months. In some cases, timing of outcome measurement was unspecified, or based around daily function (for example, before/after meal times). Unless the primary time point of interest is clearly specified a priori, there is potential for the reporting and interpretation of results to be data driven, that is, according to the time point that provides the most favourable result rather than at the time point which is of clinical interest or importance.

To assess the risk of bias in trials the Cochrane Risk of Bias Assessment tool was used (covering selection, performance, detection, attrition and reporting biases).9 Out of all 102 trials evaluated to date within the two systematic reviews, only one was considered to be at low risk of bias overall.10 One of the main areas of potential bias was selective reporting. Trials were assessed as being at risk of selective reporting bias due to the lack of reporting of important, expected outcomes (for example, pain), lack of reporting of outcomes listed in the methods section, outcomes reported but not at all listed time points, or summary statistics reported without measures of variability.

Issues regarding lack of accounting for multiplicity following measurement at many time points and reporting at an ulcer level rather than a person level were also apparent, as was analysis within the intervention and comparator groups over time, rather than comparing the groups at a specified time point.

The use of composite scales, providing a summated score based on categorical scores on a range of clinical domains (for example, number of ulcers, size, duration, ulcer-free period, site and pain) were also problematic. Unless scores for individual components are also provided, it is difficult to interpret the findings as improvement could be due to changes in any one of the domains measured or a composite. It is recognised that there is no cure for RAS and therefore all treatments are used to alleviate symptoms. The assessment of their impact therefore is best carried out by the patients themselves. The use of patient reported outcome measures is an important area to consider in RAS and many other oral medicine conditions. The use of a validated quality of life tool to assess the impact of chronic oral disease would be beneficial as described by Ni Riordain et al.11

Other limitations were apparent in the RAS evidence base, often linked to poor reporting. Although no formal assessment on reporting has been undertaken, there does not appear to be any consistent improvement in the conduct and reporting of RAS trials over time. Recent trials still lack clarity with regard to the inclusion/exclusion of participants within the trials, with the definition of RAS not always being apparent or whether individuals with systemic causes of RAS type lesions had been excluded. Many trials are limited in size, often with insufficient participants to be able to observe a statistically significant effect of treatment in the trial between interventions if one truly existed.

The Cochrane reviews of RAS are not the only reviews to have highlighted limitations of the evidence base in this area. Baccaglini et al. undertook a systematic review of RAS interventional trials published from 2005 to 2011.12 They identified considerable methodological flaws in the trial designs. They concluded, 'Improved design, analysis and standardised reporting of clinical trials are needed to maximise study quality, disclose potential sources of bias, and ensure complete assessment of product safety and effectiveness.'

These limitations are not isolated to RAS trials. Clinical trials in other areas of oral medicine have also been compromised due to methodological factors.13 Two recent systematic reviews assessing interventions for mucocutaneous pemphigus vulgaris (PV) and mucous membrane pemphigoid (MMP), carried out as part of the Sixth World Workshop of Oral Medicine (WWOM VI) and published in 2014, also concluded that there was inadequate quality of research supporting optimal PV treatment and a lack of high-quality research providing evidence-based MMP treatments.14,15 Both reviews revealed numerous methodological limitations including heterogeneity of outcome measures used.

Future trials

Several steps can be undertaken to help overcome the limitations identified. A number of these steps are clearly described in a recent paper by Innes et al.16 Prospective registration of trial protocols can help overcome issues of reporting biases, specifically outcome reporting bias. Careful attention needs to be given to the choice of intervention being evaluated; interventions should be relevant to current practice and compared with appropriate controls (that is, not sub-clinical doses of alternative treatments). Trials should be methodologically rigorous in their design and fully reported according to the CONSORT statement (www.consort-statement.org).

While single treatment interventions are the most common type of trial design, they do not necessarily reflect how the intervention is used within everyday practice, which often utilises multiple and/or sequential approaches to treatment. This has the potential to reduce the external validity of the study and so other types of trial design should be considered (for example, n-of-1 and stepped interventions) at the design stage.

In addition to examining the effectiveness of interventions, future studies should incorporate an appropriate economic evaluation. As stated in the Cochrane Handbook,17 the usefulness and applicability of Cochrane reviews can be enhanced by incorporating economics perspectives. Future trials should consider calculating and reporting the costs of interventions evaluated in order to better inform healthcare decision-making.

Outcome sets & the COMET initiative

Importantly, given the huge variation in the choice of outcomes measured, a set of standardised, clinically relevant outcome measures needs to be developed across different topic areas within oral medicine. As described below, this work is currently underway for RAS, registered with COMET (core outcome measures in effectiveness trials; www.comet-initiative.org), and was presented as a poster at the recent European Association of Oral Medicine (EAOM) meeting in Turin.18 We hope it will be used to inform trials and subsequently reviews in this area.

The COMET initiative is an international collaboration which aims to reduce the heterogeneity of outcome measurements used in trials (www.comet-intiative.org). COMET (core outcome measurements in effectiveness trials) 'brings together researchers interested in the development and application of agreed standardised sets of outcomes, known as a 'core outcome set.' These sets should represent the minimum that should be measured and reported in all clinical trials, audits of practice or other forms of research for a specific condition. They do not imply that outcomes in a particular study should be restricted to those in the core outcome set. Rather, there is an expectation that the core outcomes will be collected and reported to allow the results of trials and other studies to be compared, contrasted and combined as appropriate; and that researchers will continue to collect and explore other outcomes as well' (www.comet-intiative.org). There are a wide variety of core outcome sets that have previously been developed in areas outside of dentistry. These include the CROWN initiative (core outcome sets in women's in newborn health) (http://www.crown-initiative.org/core-outcome-sets/) and OMERACT (outcome measures in rheumatology) (https://www.omeract.org).

Using a suggested framework for the development of a core outcome set, a three stage approach was carried out in the development of a core outcome set for recurrent aphthous stomatitis (COSRAS):19

  • Identifying existing knowledge: a review of the existing outcome measures in use

  • Stakeholder involvement: the opinions of RAS patients with regards to outcome measures

  • Consensus methods: Delphi consensus process of clinicians involved in managing RAS patients.

RAS patients were asked about the outcome measures they thought were important and this information was combined with the results of a systematic review of outcome measures in trials of RAS treatments. Removing duplications, over 300 outcomes were condensed into 22 broad outcomes; these included the patient outcomes of choice. These 22 individual outcomes were then presented to oral medicine clinicians at a national speciality meeting (British Society of Oral Medicine) with the aim of gaining consensus on the outcomes to be included in a core outcome set.

The difficulty of recruiting adequate numbers of participants to clinical trials is well-known to any triallist and results in the risk of a study being underpowered. The use of a core outcome set in oral medicine trials makes the possibility of allowing combination of the results of different trials in a meta-analysis a realistic possibility, as well as allowing meaningful comparison of different interventions. Ultimately, the strength of the evidence base to guide clinical care will be improved.

Conclusion

High-quality research informs clinical guidelines and everyday practice. It is important that clinicians maintain up to date knowledge of their subject matter in order to provide the best care for patients. This is especially important in oral medicine as many of the conditions and interventions used to treat have considerable morbidity and in some cases mortality. Cochrane systematic reviews are a useful tool for evaluating and summarising the evidence for clinicians, however, the quality of the evidence produced by the systematic reviews is directly related to the quality of the trials included. Improvements to the methodology of oral medicine intervention trials as described in this paper, the use of patient-related outcomes measures (PROMS) and the future development and use of core outcome sets should improve the quality of the evidence produced to inform clinical care.