What is evidence-based dentistry anyway?

The best known and most widely accepted definition of evidence-based practice was suggested by David Sackett back in the 1990s; 'Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values'1 (see Fig. 1).

Figure 1
figure 1

Evidence based practice (EBP) relies on not only the 'best available evidence' but also successful incorporation of the clinician's expertise and each individual patient's values and preferences into the clinical setting

Full size image

Improving our standard of deliverable patient care requires maximisation of each of these three domains. This article focuses on the 'best evidence' domain.

What is the pathway of evidence production?

Oral and dental research shares the same goals as the rest of healthcare research – to benefit people and patients, whether at an individual or a societal level. Clinical trials form the backbone of our clinical evidence in relation to the effect of interventions. Trials investigate a new intervention (or an old one applied in a new way) with standard treatment, or they compare two or more standard preventive strategies or treatments. These comparisons allow us to work out which strategy or treatment has the best outcome that we are interested in for a particular condition.

Although evidence is produced for action, findings from a single study (primary research) are no longer enough to call for a change in practice. Figure 2 shows a simplified ideal pathway for synthesis of primary research studies into understandable and implementable evidence. In reality, this is more complex because there are stages before clinical trials, for example, investigations of new drugs in laboratories. Also, information actually feeds both ways in the process, with findings from trials being incorporated back into earlier stages to refine and improve interventions in future trials. Finally, not all interventions or preventive strategies can be tested in controlled trials – some can only be assessed through other types of research such as observational trials. Examples of these include linking smoking with lung cancer and using public health research to evaluate the introduction of healthy public policies. However, for the purposes of this paper, it is sufficient to consider controlled clinical trials and the pathway in Figure 2.

Figure 2: The flow of evidence creation from trials to guidelines.
figure 2

Pre-trial research and implementation findings informing future research have been omitted for simplicity

Full size image

Clinical trials

Well-designed and conducted trials will provide information to fill a gap or strengthen the evidence base. By using the same outcomes and outcome measures as other trials in the same area, the information can be assimilated with them. Complete and transparent reporting of the trial protocol and the trial itself (including descriptions of participants, settings, and interventions) gives context to the trial. This helps the findings and their relevance to be well understood by those reading it. This clarity is also essential to allow systematic reviewers to extract relevant data.

Systematic reviews

Systematic reviews (secondary research) compile and/or pool data from several trials looking at the same thing. The strengths of reviews lie in the richness of information generated through combining and contrasting research data from different researchers, involving diverse groups of participants and carried out in, often dissimilar, circumstances. All pertinent trials are identified and outcomes relevant to stakeholders analysed. Again, clear and thorough reporting of methodology, quality and bias in the incorporated trials is essential for interpretation of the results.

Guidelines

Guidelines comprise recommendations based on evidence. They translate research findings (both primary and secondary) into a digestible format for clinicians to implement. Ideally they too should be produced using a rigorous and transparent process. The shortcomings of contributing evidence are made clear. The guideline is tailored for the local environment and in usable format (paper or electronic).

Systematic reviews and guidelines are sometimes confused but they both serve different purposes. Systematic reviews present a summary of the evidence but do not present recommendations. However, the guideline development process includes decision making involving judgements around the evidence and the environment in which those recommendations are to be used. Such considerations include:

  • A balance between benefits and harms

  • Quality of the evidence from systematic reviews and other studies

  • Patients' values and preferences

  • Resource utilisation

  • Equity

  • Feasibility of implementation. The GRADE framework for moving from evidence to decisions is helpful in taking these steps.2

Appropriate dissemination of the guideline is essential for effective uptake and implementation of the recommendations; all of which have stemmed from the original primary evidence.

Is this pathway efficient and effective?

All of the research along this pathway should be fit for purpose and accessible. However, there are a number of points where it falls short in quality, resulting in a failure to translate research to health benefit, and wasting time, effort and money.3,4,5,6 In 2010, the cost of global life sciences research (mainly biomedical) was estimated at around US$240 billion.7 Less than 50% of the published biomedical literature is estimated to be adequate in conduct and reporting, making more than half of research insufficient in quality to be fit for purpose.8,9 This enormous wastage afflicting the biomedical research literature has been quantified as 85% of its investment; a staggering $200 billion dollars for 2010. This translates into tens of billions of pounds being wasted.9 In dentistry, there is a similar, if not worse, problem with poor conduct and insufficient reporting of trials.10

This ideal pathway of translating primary evidence to improved patient care can be compared to a system for producing drinking water. In the same way that a pipeline carrying water to a destination can leak water at various points along its journey, the quantity of evidence can be depleted at key points (failure to write or publish etc). Also, just as distillation of the water improves its quality, research has to be distilled through peer review, interpretation and syntheses to become useable clinical recommendations.

What is being done to improve the process of evidence creation?

Because there are many points of failure along the pipeline, by simply making very small adjustments to deal with these at each point, loss of information and improvement in the quality evidence can be introduced with little alteration to the status quo. An accumulation of these marginal gains can result in a huge overall improvement in the final product of 'best available evidence'. This is also known as the 1% improvement theory (Fig. 3), thought to initiate from Wilhelm Steinitz the first ever world chess champion (from 1886 to 1894) who developed modern chess game theory on the basis that small advantages accumulate throughout the game to give big advantages. This theory was brought to prominence by the success of the GB Olympics cycling team in 2012.11 Over the last 30 years, a number of separate collaborations and initiatives have evolved to do just this; to improve the evidence flow and distillation process by independently reducing losses in evidence quality and quantity in a small way to optimise the evidence playing into patient care.

Figure 3: Marginal gains in the flow of evidence.
figure 3

Small improvements are required at many stages from the primary research stage, through synthesis of these studies into systematic reviews and the incorporation of all of this evidence into guidelines preventing loss of integrity and quality in an incremental way

Full size image

Clinical trials: what can be done at the primary research stage?

Before a trial is even carried out, there is potential for the integrity of the evidence flow to be reduced. Funding for trials should be targeted to where gaps in evidence have been identified. Trials are often carried out with a providence that has more to do with happenstance and interest of the researcher than with efficiency and priority in mind. However, this has been changing in many countries with governments, research councils and charities (the three main funders of public research) identifying areas where evidence is needed and commissioning research for them. This drives funding towards need, although some have questioned whether this might stifle innovation.

When the area to be investigated has been identified and the research question formulated, the best design has to be chosen to answer that question. This is not a matter of simply picking a trial design from a shelf. No two trials are the same and there is an acknowledged element of creativity in the research process.12 In the case of trials, this is often determined by the environment and requires balance between a tightly controlled design, generalisability and practicality at many different levels.13 In primary research, there are a number of collaborative efforts to improve trial design, conduct and reporting. TRIAL FORGE14 is an initiative to establish a better evidence base behind trial design, set up, running and analyses processes. Better understanding of what works and what doesn't will improve inefficient mechanisms and prevent a cycle of repetition of the same mistakes. Increasing efficiency and effectiveness in trials, offers better value for money in clinical research, a growing concern in evidence production.14 The role of systematic reviews to identify research gaps16 and to use earlier trials to design future trials is well accepted although the advice is not well followed.17,18 Attempts to promote this have come through funders requiring reviews to justify the request for money and from ethics committees as part of the application. The trial should be necessary and designed to succeed in its aims.

Clinical trials should be registered publically a priori. The World Medical Association's Declaration of Helsinki (Ethical Principles for Medical Research Involving Human Subjects) states: 'Every clinical trial must be registered in a publicly accessible database before recruitment of the first subject'.19 There are different clinical trial registration sites but one that meets all of the criteria for the International Committee of Medical Journal Editors (ICMJE)20 and is free, is Clinical Trials.gov.21 The ICMJE also recommends that journals publish the trial registration number at the end of every abstract reporting the results of a trial. Unfortunately, it still has not been applied as a mandatory requirement for all dental journals to require a priori trial registration and some do not even make it a requirement for publication of the article, despite having it in their instructions to authors. Journal editors have a significantly important role in enforcing trial registration by refusing to publish trial reports that have not been registered. Many start to push this requirement by allowing researchers to register the trial retrospectively and go on to make it compulsory. The dental research community falls far behind our medical colleagues in this respect and it has important implications for transparency of trials.22 All Trials is campaigning for this and provides an excellent resource explaining the importance of trial registration and result reporting.23

Once the trial has been carried out (no mean feat in itself but not the focus of this paper), there is often a failure to report details in the manuscripts published in journals. Historically, the length of biomedical manuscripts has been limited by the shortage of physical space available in paper-based journals. A lack of available, detailed trial information not only makes it difficult for the readership to appraise the quality of the trial and decide whether the results are worth considering, but it can also make it difficult for systematic reviewers to extract relevant data. There have been two main ways of approaching this. Firstly many journals now offer the option of storing information associated with papers digitally, allowing the authors to make detailed methodology and datasets available online for readers to access. Secondly, there has been a move to push authors towards reporting a minimum set of pre-specified information in their paper. This minimum information set has been decided upon and itemised by CONSORT, the Consolidated Standards of Reporting Trials.24 Completion of the CONSORT statement checklist is a prerequisite of many journals before authors can submit their paper for approval to the journal.

Using different outcomes and outcome measures also compromises the capacity to combine evidence.25 The COMET Initiative, Core outcome Measures in Effectiveness Trials,26 are a group whose aim is to bring together researchers and stimulate the construction of core outcome sets (agreed standard outcomes for trials) to improve comparing, contrasting and combining trial results because variability in outcomes and outcome measures is one of the main impediments to using primary research in systematic reviews. COSMIN, COnsesus-based Standards for the selection of Measurement Instruments,27 is a linked collaborative effort focused on improving the selection of health measurement instruments, ideally to tie in with COMET. Mandatory trial registration, COMET, COSMIN and CONSORT aim to reduce poor reporting practices including non-reporting or selective reporting of information, inconsistent, biased or 'spinned' reporting.28,29 When reading the report of a trial in a journal, checking the trial protocol on a trial registration website against the report makes it is possible to see what was planned to be done, planned to be reported, and whether these were adhered to by the researchers.

Many established standards or guidelines in the field are further collected by the EQUATOR Network, Enhancing QUAlity and Transparency Of health Research,30 which is a resource aimed at supporting clear, accurate reports for all types of health research studies. Here, researchers can also find guidance on statistical analysis and handling of missing data etc. At the primary research stage, transparency should also be sought concerning potential conflicts of interest both financially and non-financially ('academic/professional bias').31

Systematic reviews (the secondary research stage)

Primary research is assimilated into secondary research; for example, systematic reviews or meta-syntheses. Systematic reviews allow an overview of all the trials on a specific topic. The narrative of a systematic review allows us to see how many trials there are, what they have looked at, the populations they have investigated, and the interventions they have studied. It contrasts results of different trials and allows identification of sources of heterogeneity. Any such differences in efficacy or effectiveness of interventions might guide implementation in different groups, or might point towards limited generalisability or different trial conduct.

If a meta-analysis (a statistical analysis of combined results of trials) is possible, there will be a statistical presentation of how the different trials' findings compare, what the relative contribution is of each trial, and in the end whether one intervention is better than another and in what context. One of the most reputable systematic review groups is the Cochrane Collaboration,31 a virtual organisation, comprised of many different groups, of which the Oral Health Group is one and populated by people from different countries and areas of expertise who come together to produce the review. Cochrane reviews are known for their thoroughness in review development, they insist on publication of a protocol prior to the review being carried out and the result is a review that carries weight in its findings as a product of the rigorous process.

An alternative for registering non-Cochrane reviews is the PROSPERO register.33 A priori systematic review protocol registration should be encouraged as it establishes which reviews are being carried out, as well as reducing the risk that other groups will address the same question. It improves rigor through transparency about the methodology of the review and any updates.

PRISMA (Transparent Reporting of Systematic Reviews and Meta-analyses) Statement34 has been developed to guide review developers by laying out a minimum set of items that should be reported in a review, in a similar way to CONSORT with the aim of improving reporting of research at the review stage. Eventually, review findings can be evaluated both regarding the direction of any potential recommendation towards an intervention, and the strength of evidence underlying that recommendation (that is, the confidence one can have into what was found by the review). To do so, the GRADE collaboration have outlined a detailed process35, and have recently launched an online tool allowing to bridge the gap between secondary research and guideline development.36

Guideline development stage

One way of making synthesised evidence usable in practice is to present recommendations for best practice in clinical guidelines. The AGREE II (Appraisal of guidelines for Research and Evaluation) instrument37 has been developed to evaluate the quality of practice guidelines. GRADE36 (previously mentioned) have developed a method of assessing evidence quality and linking it to clinical recommendations. More specifically, the DECIDE (Developing and Evaluating Communication Strategies to Support informed Decisions and Practice Based on Evidence) collaboration38 have produced evidence for working out the best dissemination strategies for recommendations to promote their use in practice.

Implementation and outlook

How can we ensure the use of these tools to improve clinical research design, conduct and reporting and to increase usability of research? There are a number of regulatory opportunities. Funders, who haven't already, should develop and enforce a system where trial design, conduct and reporting complies with standards, is transparent and justified. Journals should require authors to comply with reporting standards like CONSORT and PRISMA before manuscripts are even accepted for peer review. Although there is no formal requirement for editors of biomedical journals to undertake training before taking up an editorship, there are a number of forums that offer support, such as the International Committee of Medical Journal Editors,39 the World Association of Medical Editors,40 the Council of Science Editors,41 and the Committee on Publication Ethics.42

Dissemination of the tools discussed above needs to be more comprehensive. Perhaps making them available at a 'one-stop shop' and easier to find, would improve their uptake. In many, but not all, countries, ethics committees already require a priori registration of trials. Journals should support this move. Editors and journal peer reviewers should require the same for systematic reviews. Journal reviewers also should be educated on how to use existing guidelines, as they could act as facilitators or even change agents. Again, there are some resources available to train peer reviewers including Critical Appraisal Skills Programme (CASP).43 Making sense of evidence, World Association for Medical Editors (WAME)44 Resource for Evaluation of Research Articles, Publishing Research Consortium Sense about Science Peer review: the nuts and bolts45 and a programme of training materials for peer reviewers in the British Medical Journal.46

The roles of patients/participants, clinicians, policy makers and service commissioners in improving the quality of evidence is perhaps not as obvious as the roles of researchers. However, patients/participants and clinicians are well placed to suggest priority areas for research, based on experience, both as service users and clinical experience. Policy makers and service commissioners have a strong role to play in suggesting priority areas for research based on awareness of gaps in research to inform policy creation.

Conclusion

There are many points along the research process where research is lost and its quality is reduced. The myriad of tools to avoid this are already available and being implemented by many. However, there needs to be more universal usage by all those involved in the flow of evidence creation, that is, those who pay for research (funders), produce it (researchers), regulate it (ethics committees), take part in it (patients/participants) and use it (clinicians, policy makers and service commissioners). Incorporating these established improvements or standardisations – even though they are only small changes at many stages in the evidence flow pathway – will result in significant overall improvement in the evidence that we can use at point of care with our patients. We need to reduce research wastage. The water company might choose to accept the loss of water from their pipeline as necessary, it being too expensive to trace and fix. However, we must take steps to prevent the loss of research and research quality along the research pipeline and its associated wastage. Also, we owe it to the public whose money has been invested in these trials and whose goodwill and trust has been invested in volunteering as participants, on the understanding that this will result in benefit for them and for others.