Introduction

In the previous paper in this series we suggested that the identification of research questions that dentists feel are relevant to practice might help to promote dental practices informed by research evidence. We therefore set out to determine a means by which the research questions most relevant to practitioners could be identified and subsequently used to set priorities for the use of the research funds.

Thanks to Shirley Glasstone Hughes, a dentist, researcher and British Dental Association (BDA) member, the Shirley Glasstone Hughes (SGH) Trust Fund was established in 1990. This charitable trust has a remit to provide prizes or scholarships for dental research 'as trustees, in their absolute discretion decide'. Between 1991 and 2005, 41 research projects were funded, valued in total at £678,000.1 In 2005, the trustees commissioned an evaluation of the output from the fund. This assessed the relevance and usefulness to dental practitioners of the research that had been funded by the trust over the previous 15 years. This was to determine whether the relevance of the research funded by the SGH could be improved.2

Two key recommendations (of a total of seven) arose from this evaluation:

  1. 1

    The SGH Fund should continue to fund research led by and involving non-academic dentists, that is, primary care practitioners

  2. 2

    Greater effort should be made to ensure SGH funded work addresses themes of direct relevance to primary dental care.

Therefore in order to enact these recommendations it was necessary to explore mechanisms that would inform the trust about the day-to-day evidence needs of primary care practitioners. To this end a web-portal was established that gave practitioners the opportunity to nominate subjects that they felt were worthy of research funding. Thus a research agenda-setting process for general practitioners was created and the intention was to attract as many UK dental practitioners as possible to voice their opinion on what they need to know from research. The ultimate aim was to ensure that any research funded by the trust would be research that practitioners wanted, and found useful and usable.

A process to inform research agenda-setting by dental practitioners

The initial process for generating practitioners' unaddressed questions, and for prioritising and processing the relevant available evidence, is illustrated in Figure 1.3

Figure 1
figure 1

Feeding dental practitioners' views into research commissioning3

Full size image

Figure 1 shows that, for prioritised topics identified through the online topic nomination and voting system, a literature search followed by a 'rapid review' of the available research evidence was undertaken. The results of these rapid reviews were disseminated to practitioners and research commissioners through user-friendly research summaries. 'Rapid review' draws upon the structured processes of a systematic review, but is delivered with clearly identifiable constraint and short time frame. Rapid reviews undertaken in a period of one calendar month aim to determine and provide a narrative synthesis of the nature of the evidence available regarding the identified topic. This was fed back to practitioners (through the web portal and other media).

If research evidence available was insufficient or inconclusive to be of benefit to dental practitioners, the subject was fed to other funding bodies (NIHR, Cochrane, etc.) in an attempt to stimulate further systematic reviews or research funding in order to strengthen the evidence. However, if a topic considered a priority by practitioners was found to have no, or very poor research evidence to underpin practice and it was considered relevant to decision-making in practice, the topic was highlighted to SGH Trustees for potential SGH research funding. Overall, the idea was that once research on the practitioner-identified topic had been funded and carried out, the research findings would be disseminated to practitioners (via the web-portal and other media) and this would facilitate evidence-based practice. Thus, 'research waste'4 would be avoided.

Table 1 describes the process in detail and Figure 2 shows how the topics appeared on the website.3

Table 1 Description of components of the research agenda-setting process3
Full size table
Figure 2
figure 2

Screen shot of the online voting system

Full size image

Each time a new potential research topic was introduced on to the website by a practitioner, participants had an opportunity to vote. Practitioners could indicate whether they felt the topic to be important and relevant to them by giving a positive vote or, if they felt the subject to be unimportant or irrelevant to their day to day dentistry, they could give a negative vote. Thus the level of priority it was seen as having by the practitioners using the website was determined. Each month the most highly prioritised topic that is, the one with the most positive votes (after deducting the number of negative votes) went forward for rapid review of the evidence currently available relating to that topic.

Evaluation of the research prioritisation system

Assessment of the feasibility and effectiveness of new processes enabled the success of the initiative to be evaluated, in order to develop understanding about whether, how and why the system/process worked and in what way. Figure 3 illustrates this interactive evaluation3,4,5,6,7,8 and Table 2 highlights aspects of feasibility of the research agenda-setting process.3,5

Figure 3
figure 3

Interactive evaluation6,7,8

Full size image
Table 2 Aspects of feasibility of the research agenda-setting process3,5
Full size table

Evaluating the effectiveness of the system for promoting evidence-based practice in dentistry included consideration of the extent to which the website and prioritisation system encouraged or supported practitioners to utilise evidence to underpin their professional decisions. In order to do so it was necessary to assess the level of effectiveness of each of the key components of the system. Therefore we needed to evaluate how effective the system was at generating topic nominations from practitioners; how well the voting system worked in achieving nominated topic prioritisation; the extent to which the rapid reviews were read and remembered and finally, whether dentists and research commissioners valued the evidence statements.

The evaluation consisted of a mixed methods approach, using several forms of data collection tools. In depth qualitative semi-structured interviews were carried out with all those involved in the development and implementation of the system (commissioners, designers, creators and implementers). These data were then subjected to thematic analysis. We also analysed quantitatively the level and nature of online postings and also surveyed practitioners using two BDA omnibus surveys as a vehicle for data collection. An online survey of the SGH research commissioners was also thematically analysed. An integrative analysis across all findings was achieved through triangulation9 and perceived influences upon web-based system feasibility can be seen in Figure 4.

Figure 4
figure 4

Perceived influences upon web-based system feasibility3,5

Full size image

Results

Seven themes relating to the feasibility of the system were revealed via the analysis of the interview data, and these in turn fit into three overarching themes (technical, operational and economic; see Figure 4).5 Technical feasibility was seen as depending on the ability to have a technically-deliverable idea. However, there was clear recognition that technical feasibility did not necessarily mean that the objective would be delivered; that is, the website might 'work' but it would not necessarily involve practitioners sufficiently in research agenda setting to the extent that evidence-based practice would be influenced.

The evaluation of the effectiveness of the new website showed that 247 people registered on the website, with two thirds (199) of these occurring in the 6 month period following its launch.

However, the evaluation raised doubts about the website's ability to sustain interest once a practitioner had registered. It was recognised that the system would only continue to be a success if the site was continuously reviewed and updated. Thus a number of initiatives were added to the web portal in order to address the issues of long-term engagement and sustaining practitioners' interest.

Forty-eight topics were initially nominated. Topic nomination varied widely, with some registrants presenting specific research questions, while others put forward variable length general descriptions of areas of interest. Discussion among website participants was low with only 21 discussion comments listed. Comments varied widely in nature, from challenging the construction of a topic and what data might be relevant to it, to providing methodological background as to how the topic could be researched. Some comments offered 'answers' to the topics nominated for research.

The 2010 BDA omnibus survey showed that out of 241 respondents, only 29 (12%) were aware of the website. In contrast, nearly half (49%) of those respondents had read one or more of the evidence summaries and three quarters of them had found them to be very or somewhat useful. Three fifths (59%) agreed that the topics reviewed had been relevant to their day to day practice. Evidence summaries therefore appear to be recognised as a valuable aspect of the SGH web initiative and thus continue to be produced.

Finally, the online survey of the SGH research commissioners showed that all felt the system had helped 'a great deal' in providing a basis for making decisions about which research should be commissioned or funded. All commissioners felt that the initiative had given them a representative and much better picture of the research needs of practising dentists. They did, however, recognise that the web based system needed to grow its user level and that if this could be achieved, the success of the initiative would be greatly enhanced.

Conclusion

The new and ambitious initiative set up by the SGH Trust was introduced in an effort to involve and place primary dental practitioners at the heart of research agenda- setting. The evaluation has shown that it is feasible to set up and run a web-based system to generate practice-relevant research topics and themes and that this can be achieved with relatively modest resources and within short timescales. However, due to underuse of the system its effectiveness in achieving its purpose was limited.

The evaluation has clearly shown that it is feasible to design and implement a process to generate rapid reviews of the topics practitioners say are of interest to them. In addition, for those who accessed the evidence summaries, they appear to have been effective in as much as they were perceived as useful and informed decision making. However, the low level of participation in the website meant that dentists' awareness of the summaries was low. It is hoped that the redesigned 'Curious about' website (see below) and the addition of verifiable CPD will improve this.

In conclusion, the new process that was designed and implemented to generate user-friendly evidence summaries was feasible. However, the dissemination of information that dentists had said they wanted and needed was of limited effectiveness because of the low level of use by primary dental practitioners. The evaluation presented the information we needed in order to provide a usable, useful and economically viable means of involving dental practitioners in setting the research agenda and utilising research results in practice. It is hoped that practitioners' involvement will continue to grow.

SGH/BDA initiative: the 'Curious about' website

The latest version of the Curious about website offers verifiable CPD to practitioners accessing the rapid reviews. In addition to dentists being able to submit topics, and in an effort to maximise topic nomination and therefore support a practitioner-driven research agenda, BDA committee members (dentists acting on behalf of dentists), officers and staff can also submit topics to the SGH commissioners. BDA staff, especially the advice teams, are in continuous contact with members and are aware of the issues that practitioners have and the questions they would like answered. During the time the site has been live the nature of the suggestions has varied widely, and interest seems to be being maintained.

The BDA Curious about page, which is described in this paper, can be accessed at http://www.bda.org/dentists/education/sgh/about_sgh.aspx