Introduction

In September 1995, the Chief Dental Officer (CDO) Brian Mouatt wrote to all general dental practitioners (GDPs) detailing the introduction of a voluntary pilot scheme for clinical audit in general dental practice.1 One evaluation of this scheme was carried out by Fleming and Golding between 1998 and 2000.2 In March 2001 the then CDO Dame Margaret Seward announced a development of this scheme. She wrote to all GDPs in England informing them that from 1 April 2001, all GDPs working in the general dental services (GDS) would be required to participate in a rolling programme of at least 15 hours of clinical audit or peer review activity every three years.3 Throughout this report this general dental services clinical audit scheme is termed the GDS scheme. No formal evaluation of this scheme appears in the literature. There is clearly a lack of evaluation research relating to the GDS scheme and this study seeks in part to address this shortage.

A new dental contract was launched on the 1 April 2006. With it has come the devolvement of responsibility for clinical audit to Primary Care Trusts (PCTs) within their clinical governance arrangements.4 In this period of significant change in the GDS, the position and role of dental clinical audit appears unclear. In particular, the potential for development of clinical audit and the momentum that had been achieved thus far may be diluted by this lack of clarity and change of working arrangements.

Continual professional development is a requirement of every registrant with the General Dental Council (GDC) and clinical audit provides one of the mechanisms by which dentists and dental care professionals (DCPs) can achieve this. Having an effective system of clinical audit in primary dental care, which is subject to evaluation, seems a desirable goal. It therefore seems an appropriate time to be evaluating the GDS scheme that has been in operation to help inform the future development of clinical audit in dentistry.

The aim of this study was to use an evaluative research approach to undertake an evaluation of the GDS clinical audit scheme, particularly with regard to audit scheme structure, process and outcome.

Methods

Participants

A purposive sample was used for this study in order to interview people who were capable of providing answers to the evaluation research questions. The sample consisted of a group of eight GDPs who had undertaken clinical audit in the GDS scheme.

A range of criteria was used for this purposive sample to allow for maximum variation. Ritchie et al. describe maximum variation (or heterogeneous) sampling as being a deliberate strategy to include phenomena that vary widely from each other.5 The benefits to this sampling strategy are that detailed descriptions for differing cases are possible, allowing for uniqueness and the identification of central themes, which cut across the variety of cases or people.6 The criteria used are shown in Figure 1.

Figure 1
figure 1

Sampling criteria

Full size image

Procedure

Potential participants were recruited to the research study by contacting GDPs by letter. The letter detailed an outline of the project, the main inclusion criteria for taking part in the study and requested an expression of interest. It was sent to all GDPs in Essex. Included with this letter was a participant information sheet, which gave more detailed information regarding the study. Potential participants were encouraged to contact the researcher to indicate their willingness to participate in the study.

Each time a GDP met aspects of the selection criteria, a review was made of the emerging shape of the sample to identify where any gaps were developing that may have had a bearing on the further selection of participants. This helped to ensure that the full range of selection criteria was represented in the final sample. Potential participants that fell within the sample frames and who met the range of criteria required were contacted and a suitable time and convenient location for an interview arranged with them.

Design of interview schedule

An evaluation framework following the principles for the evaluation of health services originally proposed by Donabedian and promoted by Bowling was used to provide a structure to the study7,8 under the headings:

  • Evaluation of the structure of the audit scheme

  • Evaluation of the processes within the audit scheme

  • Evaluation of the outcomes of the audit scheme.

Research questions in each of the above evaluation framework areas were identified, and are outlined in Figure 2.

Figure 2
figure 2

Evaluation research questions

Full size image

An interview schedule was used when conducting the research interviews. Many authors advocate the use of some form of interview schedule when conducting semi-structured interviews, as it can act as an aid memoire to ensure that all the relevant topics are discussed within each interview, but should not be over prescriptive nor dictate how questions should be worded or the order of questioning.9,10 It also provides a mechanism for steering discussion, while ensuring flexibility to allow details important to the interviewee to be explored.10

In this study the main topic headings chosen were designed to cover the relevant areas of structure, process, impacts and outcomes from undertaking clinical audit. These main headings were broken down into subheadings to ensure that these areas were fully discussed. The interview schedule is illustrated in Figure 3.

Figure 3
figure 3

Interview schedule: evaluation of dentist experience of clinical audit

Full size image

Data collection

A semi-structured interview was chosen as the data collection tool for this study. The author, having previously undertaken a period of research interview training, carried out all of the interviews, which helped to provide a consistent approach to data collection. All interviews lasted for between 40 minutes to an hour. They were tape recorded, which allowed for later verbatim transcription of interviews to be made and served as a permanent record of what was and what was not said in an interview, also allowing the language of the participants including hesitations and tone to be captured.11 In addition, some field notes were made during or immediately after the interview.

Data analysis

The initial step in the data analysis was the transcription of the tape recordings. This process allowed for the initial immersion of the researcher into the data, allowing for the commencement of what has been termed familiarisation.12

The 'framework' approach to qualitative analysis12 described by Ritchie and Spencer was followed during the analysis phase of this study as it is systematic, comprehensive (in that a full analysis of all the data is made) and it provides a visible transparent process that is accessible to others, thereby helping to promote rigour and trustworthiness in a project. It incorporates several stages. Firstly familiarisation with the data; this involves reading a selection of the transcripts to continue the process of immersion and helps to start the process of identifying themes in relation to the objectives of the research. The second stage involves identifying a thematic framework; here using the transcripts and the topic guide, emerging themes were listed, grouped together and numbered to create an index or coding structure. The data was then coded, which involved systematically reading through each transcript and applying the index, so that blocks of text could be coded using the coding structure. The next stage was for the data to be charted. Here data was lifted from its original context and rearranged according to themes. Once the data had been charted according to core themes, the next stage was to map and interpret the data. This involved looking for similarities, differences, ranges, patterns of association and changes over time within the data and across charts.12 The software analysis tool MaxQDA2007 (Verbi Software, Marburg, Germany) was used to assist in the analysis.

Ethical considerations

Ethical approval was obtained from Essex 2 local research ethics committee (LREC). Approval to conduct the study was obtained from the central NHS Research and Development Committees for primary care in the region, to ensure that the conduct of the study conformed to the protocols of research governance.

No information relating to patients was collected during this project; the participants being GDPs. Written consent was obtained from those who agreed to participate. All data was considered confidential to the study and no details were used in any reports that could identify individual dentists or specific dental practices/health care settings. Data was held securely in accordance with the 1998 Data Protection Act.

Results

Scheme structure

Participants views on the theme of scheme structure fell into the following general areas; administration, motivators and barriers to engagement, and anonymity in audit.

Administration

The procedure of applying to undertake an audit in the GDS scheme involved using an application form and submitting it to a local assessment panel (LAP). Most participants indicated that the procedure was relatively straightforward. The types of clinical audits that respondents had participated in were varied, including single-handed audit projects, collaborative audits in small groups (up to eight participants) and projects with a large number of participants that had been organised within the GDS structure. A coordinator to take a hands-on approach to 'running' a project was a concept that appealed to many participants. Some because they didn't feel that they had the skill set to do this themselves and others who had participated in larger projects, appreciated an 'external' coordinator who had the role of ensuring everyone kept to task.

Motivators and barriers to engagement

Many participants had experience of using some form of template design, often referred to as a 'cookbook' to help them undertake a clinical audit. Several dentists had found this particularly helpful.

'I think that the fact that the clinical audit was designed by other people and that it was very straightforward means that you had less work in your busy professional lives...and you were much more likely to participate.'

A development on the use of a 'cookbook' experienced by some was the use of looking at samples of completed projects which could serve as a template or starting point, but further served to illustrate the level of competency that fellow dentists had achieved.

'I thought it was a good idea they had sample ones for you to have a look at, so you can see what other people have done and then you can adapt it and change things and do it your way.'

A strong motivator to engagement that emerged from the study was that of linkages to other parts of the dentists' practising lives. In other words, doing clinical audit 'fitted' with and/or reinforced other things that they were involved in, examples of this being;

To assist with achieving a further qualification such as the MFGDP(UK) (diploma of membership of the faculty of general dental practice), a combination of clinical audit with other practice initiatives, such as the BDA good practice scheme and the Denplan excel scheme, or a link between audit and practice organisation.

'We looked at it as an opportunity to structure our meetings...it was difficult to enthuse people, so a suggestion was made that we link it to audit and that went very well really.'

Many participants saw clinical audit with practitioners from other practices as an opportunity to 'get out' of their surgeries and that this was a good thing due to the potentially insular nature of practising dentistry in primary care.

'I think we have to keep in touch with what's happening around us 'cos we're quite insular in our surgeries, and it's fine to look at other views on how things are done in other surgeries.'

The mandatory element to undertake 15 hours of clinical audit activity over a 3 year period, which commenced in April 2001, served as a powerful motivator to undertake audit.

'We had looked at it before and sort of not progressed because, not quite sure, so the mandatory thing spurred us into action.'

For others it made them undertake something they had no prior intention of doing.

'I'd say I just did them because I had to.'

Several practitioners interviewed expressed a feeling of isolation in relation to undertaking clinical audit. This was especially so for the single-handed practitioner. In order to undertake some form of collaborative audit would involve having to contact and meet with colleagues from other practices, which some found a challenge.

'Working solo here, the other chap in the village I don't think was terribly interested in setting up a group, it was quite difficult work to get others around here to get going on it, or else I was asking and they had already done something.'

Several practitioners felt they lacked the necessary expertise to undertake a project without further assistance,

'We really didn't know what we were doing, you know, what we were supposed to set up and how we do it and so it was one that we put off for a long while.'

Even with that guidance, some were still uneasy as to what was expected of them.

'Although there were guidelines given we didn't actually know what we should do, would this be satisfactory enough, would this be too small is this too big, is this something that they would laugh at?'

In many cases this acted to delay participation.

'Like all these things you don't do very often, the fact that you hadn't done it, you need to find out and you weren't sure how to do it, was a deterrent.'

A particular criticism levelled at the various guidance that was available was that in an effort to encourage as many dentists to get involved as possible, the interpretation of what could be done was left vague, presumably to encourage dentists to participate by leaving them with a lot of choice about topic selection and how they could design their projects rather than being too prescriptive. Many dentists found this unhelpful.

'The stuff they sent through at the beginning didn't really help, probably nicely open in that you could almost do what you wanted, but that for me was too open...they sent you photocopies, things with ideas but they didn't say how long it could take, how long can you spend.'

Some practitioners cited that they were put off getting involved because of their perception that it would take a lot of time and effort, including time away from their clinical duties.

'You had to pick a topic, if you wanted to do a collaborative one you had to get all the dentists to do a particular topic, some to do one bit some to do another, some disagreed and said no that's a lot of rubbish lets do something else...I suppose the motivation of trying to get something done off your own back you know is harder.'

Anonymity

While many practitioners expressed the view that they had no problem with putting their own name to an audit, often the same practitioners also expressed the view that generally it would be better for the profession if participation in clinical audits were kept anonymous, as they would be anxious of who might get hold of the information and what might be done with it.

Respondents also felt that a lack of anonymity could deter potential participants from engaging in clinical audit.

'If there is a subject which you know you might be a bit weak on...and everybody was going to know how poorly you did on it, it might discourage you from taking part.'

Or that it might not be completed honestly.

'If you think you are answering it and somebody is going to work out who you are, and it could get out, and also... think this is like an exam... you might not get honest answers.'

Scheme process

Participants' views on the theme of scheme process fell into the following general areas; topic selection, data collection, data analysis, report generation and dissemination, feedback, re-auditing and post-audit activities, the role of collaboration and time commitment.

Topic selection

Dentists used a variety of strategies for selecting a topic for a clinical audit project. There were several examples of dentists asking their peers what they had done and then conducting a similar (or often the same) clinical audit project. The main purpose of this being to establish what was straightforward and quickest to complete. The justification for this being that the projects needed to be done quickly as the end of the mandatory period for completing 15 hours of clinical audit activity was looming.

'Did we choose a topic where we had a problem? We didn't really, we just looked at what would be the easiest one talking to practitioners.'

Over time, however, it became apparent as practitioners embarked on subsequent projects that many of them started to look at topics that would benefit their practice in some way rather than just being easy to complete.

'The topic we chose was because we were aware of the fact that within the practice there was a difference of opinion in prescribing a drug and of dosages.'

Data collection

Data collection from early clinical audits seemed to be undertaken in the main by dentists themselves.

'To be fair at that time it was mainly the dentists, because the dentist would take the radiographs and so they were assessing them and some of it was retrospective as well, we pulled out some historic X-rays, but that was mainly the dentist.'

But there was also evidence of a variety of team members being involved in data collection over time.

Data analysis

Data analysis of individual clinical audits and of small collaborative audits was carried out universally by dentists. In the case of small collaborative audits one member was elected to carry out the data analysis phase.

In the case of larger collaborative audits data were often collected up and sent away to be analysed by a distant third party. This proved to be a popular method because there was a perception that 'others' would be better at doing it.

'Get someone else to do it yes that's fine, their statistics is probably a good deal better than mine.'

Report generation and dissemination

Report writing was universally undertaken by a dentist. In the case of a collaborative audit this was usually undertaken by one nominated member of the group. Occasionally a non dentist member of the dental team was engaged in typing up the report before submission. Some dentists felt they would have liked more guidance as to what depth was required of them at the report writing stage.

Dissemination of project findings within practices occurred, often at staff meetings.

'As a result of the audits we did discuss the results and the changes that we are doing...and certainly the reasons why I was prescribing differently to the way I had before.'

Evidence of dissemination of report findings beyond the individual dental practices where the audit took place was limited to submission of the report to the LAP, a process necessary to record the level of commitment (in number of hours) that each participating dentist had undertaken and to allow for the authorisation of payment. Several dentists felt that further dissemination of project findings could be useful.

'The areas like the eastern deanery is such a big area, you've got so many PCTs, that if they... share their information, we would learn from that and we could all progress.'

Feedback

There was no formal feedback received by dentists within the GDS scheme who had submitted projects to the LAP, beyond acknowledgment of the report and verification of the number of hours undertaken.

Those dentists that chose to submit their clinical audits to the Faculty of General Dental Practitioners (FGDP) as part of their studies towards the MFGDP(UK) diploma did receive feedback about their report from the FGDP. Where this occurred it appears to have been helpful and there was a belief that it has lead to improvement in undertaking audit.

Many practitioners expressed ways in which they felt feedback might be used or developed, such as getting more information about the quality of their project; how well they had done, feedback from a perceived expert in the topic area.

'You need to have feedback from some outside source because you know the practice direction could be going way off without you ever knowing unless you had someone from outside saying: 'well actually lads you know the national guidelines or the national average is this and you need to pull yourself in a bit'.'

Re-auditing and postaudit activities

The purpose behind re-auditing a topic area was to ensure continuing improvement and the sustained maintenance of standards by participants. There was a lot of evidence of informal re-auditing continuing in the topic areas that had been audited.

'I have looked at my earlier X-rays compared to my current ones and there is a significant improvement...the number of mistakes now are few and far between.'

However, there was much less evidence of formal re-auditing. A reason for this appeared to be confusion over appropriate timescales for undertaking repeat audits on differing topics, some advocating 3-6 months, while others up to 7 years.

There was, however, some evidence of formal re-auditing of clinical audits. This was usually associated with another activity such as submitting a clinical audit for assessment as part of the course work towards the MFGDP diploma or as part of a vocational training project.

'Yes I did re-audit (for the MFGDP) and it was like a mini audit focused only on the things that were more relevant and to check that they were improving.'

The role of collaboration

Many interviewees had undertaken single-handed clinical audits. Where this had been carried out by one (younger) dentist in a multi-handed practice it was felt that it was difficult to make changes in the practice as others had not been involved. Where dentists had had experience of previously undertaking audit activities such as peer review or collaborative audits in addition to single-handed clinical audit, they expressed a preference for the group activities, citing interaction with others, and the social aspect of meeting one's peers as motivating factors.

'I found that stimulating as a single handed practitioner, you know I don't have that much contact with other practitioners and I found it good, I liked it, it's one of the drawbacks of being by yourself.'

Time commitment

There was some evidence of data collection occurring during the clinical working day. Where this was the case it was associated with data that lent itself to collection during clinical surgery time.

'I sat here and was watching the clock while I was waiting for the ID block to work and I had the data sheet there.'

However, much of the data collection by participants was done outside normal surgery hours, reasons cited for this were that there were less interruptions and a feeling that it was not easy to free up clinical time to do clinical audit work.

Scheme outcomes of clinical audit on dental practice

Participants' views on the theme of scheme outcomes fell into the following general areas; outcomes affecting participating dentists, practice teams and practices, patients, and catalysts and barriers to change following clinical audit.

Participating dentists

It was clear from the interviews with participants that there were many examples where undertaking a clinical audit had led to a change in practising behaviour among those interviewed. The responses indicated that these changes were not only short term but permanent.

'Without a shadow of a doubt, my dosages and the antibiotic drugs that I am prescribing now are certainly different to what they were before the audit.'

Practice teams and dental practices

Change resulting from undertaking audit activities also resulted in new protocols being developed for use within practices, such as to help ensure the quality of radiographs within practice. Where a protocol occurred this appeared to be a permanent change in most cases. The occasions where change although initially introduced did not persist, often occurred through difficulty in involving particularly non-clinical members of the team, who didn't fully understand the purpose of the audit.

Patients

Interviewees were able to give many examples of changes that had occurred for patients as a result of the audits they had carried out. These ranged from clinical issues, such as patients receiving consistently effective local analgesia, and convenience factors, such as dispensing antibiotics from the practice rather than the patient being required to attend a dispensing pharmacy for this, to introducing protocols to deal with the problem patient group who regularly fail to attend for their appointments.

Catalysts and barriers to change following clinical audit

Topics that were perceived as being important to a practice and where the potential benefits were obvious to the participants from the outset appear to motivate participants through the whole process including making changes following the audit.

The fact that you have actually got to do something within a clinical audit in comparison to hearing a lecturer on a particular topic telling you what to do, was felt by interviewees to make change more likely to occur. Collaborative audits across a whole practice were felt by many to facilitate subsequent changes within the practice more frequently than cases of individuals conducting their own audit projects.

'If you are comparing yourself to an idealised version its [sic] easier I think just to go along with what you're doing, whereas if you're involved with a group, I found it useful to just look at the tables...and you can see 'oh I've done badly there but then oh everyone's done badly so ok I need to change but that's alright.''

Other reasons which interviewees felt motivated them in making change happen included having a positive dynamic/culture within a practice and the involvement of the practice team within the audit. It was also felt that having an external rolling framework of audit in place would help ensure that change could continue to happen over time.

Discussion

All the data collection in this study was carried out retrospective to participants' engagement within the audit scheme. There is therefore the potential for participants' recollections of their involvement to diminish. The time delay occurred in this instance due to the lag between designing the research, gaining ethical approval and then embarking on data collection. An alternative evaluative approach, which would avoid this, would involve gathering data while a programme is running in a formative evaluation.13

In this study, dentists' experiences were used as a means of evaluation. Other options for research in this area could focus on patients' outcomes following clinical audit activity, PCT perspectives on the value of clinical audit, or the effectiveness of alternative quality improvement initiatives in dentistry.

Evaluation of scheme structure

It has been suggested that the administration involved in engaging in GDS clinical audit is both lengthy and bureaucratic, involving many separate steps with a LAP that is often geographically some distance from audit participants.14 However, most of those interviewed had not found this to be a problem. There was a desire among auditors that a central coordinating role within audit projects from the LAP could ensure they kept to task and help them with the process of undertaking an audit, along with more extensive guidance.

Many auditors interviewed had not had previous experience of undertaking clinical audit themselves. In their evaluation of the pilot clinical audit scheme for GDPs in England, Fleming and Golding acknowledge that the LAP's primary role is to assure the probity of the scheme, but that this has resulted in an inward looking reactive rather than proactive approach to the scheme and that this needed to be addressed.2 They recommended both a more proactive approach by LAPs and review of the scheme's guidance.

The launch of the mandatory scheme for clinical audit in 2001 has ensured that many more practitioners participate in clinical audit and this has undoubtedly been the factor for many that made them get on and do it. There are also those who do it only because they have to, this may be fuelled by a lack of appreciation of the potential benefits or a general apathy.

Where 'cookbook' templates for undertaking clinical audit were provided for participants, these were also appreciated by many auditors as it helped them understand the process, streamlined their participation reducing the time element involved and provided a benchmark for them to work towards. 'Cookbooks' were highlighted as a recommendation in a previous evaluation of dental clinical audit2 and developing a cookbook portfolio would seem particularly appropriate given the large number of practitioners new to undertaking clinical audit following the introduction of the mandatory scheme in 2001.

Isolation of practitioners both in terms of geographic location and single handed/small practices seemed to provide a problem for those practitioners who wanted to undertake some form of collaborative audit with the onus on them to find suitable collaborative partners being arduous.

Anonymity has been a central theme of the GDS clinical audit and peer review scheme throughout its pilot stages and mandatory phase.1,3 The purpose of this being to encourage the profession to get involved without fear of negative reprisals.

'Dentists can, therefore, undertake audit secure in the knowledge that there will be no breach of confidentiality.'1

The continued need for protection of the profession was expressed in many of the interviews but interestingly, on an individual basis most interviewed would have been happy for their results to not be anonymous, unless the topic area of an audit was particularly sensitive. The negative effect of anonymity of GDS clinical audit may be to stifle dissemination and all of its potential benefits. Others have highlighted this problem.

'The problem of anonymity had resulted in the loss of professional benefit of knowing what topics other people had audited and the process they had followed.'2

The profession may well not be able to throw off the perceived protection of anonymity in audit, but it does appear to at least acknowledge its shortcomings. Allowing individual practitioners to choose whether they wish to remain anonymous for a particular project may help generate more useful information from audit for dissemination, while providing reassurance for others.

Undertaking clinical audit projects on a single-handed basis can produce useful outcomes, but there are drawbacks. The most valuable feature, identified by participants of the pilot peer review scheme for general dental practitioners, was the opportunity provided for collaborative learning.15 This feature is obviously absent in single-handed audit, but in addition to this the absence of team involvement in single-handed audit may make future 'buy in' from the practice as a whole more challenging. Collaborative learning in a dental setting appears to offer advantages as discussed by Kleffner and Dadian16 and in addition, it would appear in the context of dental clinical audit to help auditors maintain their enthusiasm and implement change.

Evaluation of scheme process

Straightforward topics tended to be chosen by first time auditors, with a radiography audit being particularly popular, which coincides with the findings of Fleming and Golding2 and as such, provided a useful introduction to the process of audit. Subsequent audits were often chosen to look at an area of the practice that the practitioner perceived might benefit from audit. So there was evidence for some 'learning the ropes' from straightforward audit initially and then progressing to more complex use of clinical audit. However, the volume of guidance released relating to radiographic audit17 and radiographic cookbook audits have led some to consider that it may be time to move on from this as a specific formal clinical audit topic. This may be justified given that many practitioners have now undertaken a clinical audit topic so have experienced the process. There is also an argument that having many individuals formally duplicating the same audit topic and coming up with a similarly reproducible range of suggested outcomes is a waste of time and resources, when a set of simple guidelines and routine in-practice monitoring might be more appropriate.2

Data collection, analysis and report writing activities were generally undertaken in early audits by dentists themselves. This may well be because of the 'newness' of the process to many and the fact that the GDS scheme was very much geared towards 'dentist' clinical audit as opposed to clinical audit in dental practice. Indeed the guidance sent out to all dentists before the commencement of the mandatory GDS clinical audit and peer review scheme advised that;

'All dentists who provide general dental services will be required to demonstrate that by the end of a three year period, they have participated in a minimum of 15 hours of clinical audit or peer review.'3

It would also appear beneficial from the point of view of other dental team members that they be more involved in clinical audit as well. This could lead to educational benefits for team members, learning about the audit process as well as any education derived from a particular topic area. It could assist in the promotion of good team working and it could benefit a practice in terms of better use of financial resources and time management.

Dissemination of project findings within the practice setting was widespread and this would appear an obvious and useful place for it to occur, but there is little evidence of dissemination of project findings beyond the immediate practice or set of practices (in collaborative audit). The post-audit report to the LAP appears to be mainly to ensure probity and not for any further dissemination. Dissemination more widely of information regarding project design details and outcomes could provide an opportunity for comparison of the results in similar topic areas and the creation of guidance, which could be appropriate over a wide area. In general medical practice there are several examples where national results are available, which have led to benchmarks and standards that individual practices can use to compare themselves against, for example the General Practice Assessment Questionnaire (GPAQ).18

There was little if any useful feedback to practices following submission of a post-audit report and there was no prescribed mechanism for this within the GDS audit structure. Many dentists indicated that they thought that this would have been useful and where feedback had been received from another agency such as the FGDP(UK) this was indeed found to be useful. Feedback is acknowledged by many as having several benefits19 and in this setting could lead to better audit design and gaining further insight into a particular topic area from an expert and/or the basis from which to reflect more deeply on the results and outcomes of an audit. A recent Cochrane systematic review concluded that audit with feedback can be effective in improving professional practice.20

The advice for dentists undertaking GDS clinical audit has been that it should be a cyclical continuous process involving re-audit of the topic area at some future point.1 Evidence of reviewing audit in an informal way appears to occur but it is sporadic and dependant on such things as topic area and time commitment. Very little formal re-auditing was uncovered in this study. This echoes what has been found in previous studies;

'After one audit project dentists tend to move on to another topic rather than repeatedly re-examine the same topic.'2

Few would disagree that formal re-auditing makes theoretical sense, but the value of a formal audit cycle has been questioned.21

Dentists appear to prefer undertaking their clinical audit projects out of clinical working times, citing that they don't have time to undertake audit during their clinical working hours. The exception to this being data collection activities where the data collection tool can be completed during normal clinical sessions. However, clinical audit undertaken within this GDS scheme was remunerated, so this preference among GDPs for undertaking clinical audit activities out of surgery hours is not simply because it would prevent fees being earned for the practice if it was carried out during the working day. Perhaps the audit scheme was regarded as an 'add on' activity very much along the lines of CPD activities in relation to clinical dental practice. Historically a large proportion of continual professional dental education events have been held out of working hours. A glance through any deanery postgraduate dental education prospectus will confirm this. So the culture within the dental profession of 'doing' CPD activities out of normal working hours may be relevant here also.

Evaluation of scheme outcomes

This study provided a lot of evidence of change occurring in dental practices, which would have a direct effect on dental patients and much of this was perceived by the participating dentists to be beneficial to patients and to be permanent change. These changes could be due to an individual behaviour change, a change within the practice team or due to the development of protocols. Fleming and Golding2 noted in their evaluation of the GDS scheme before 2001 that the lack of improvement targets and assumptions about where quality of care could most be improved, significantly reduced the potential impact for improved services and care for patients. The findings in this study certainly echo the latter statement in that the breadth of topic selection available to participating dentists and the devolution of topic selection solely to the dentists moved the focus well away from specific areas that may need to be addressed. Where clear standards are available for a particular topic, such as those produced by the FGDP(UK) relating to dental radiography these appear to serve to help standardise the quality of care in an area. Guidance covering standards in many areas of dentistry has recently been produced.22 This comprehensive reference could aid the development of standards for use in future clinical audit activities.

Other outcomes noted included stimulation among some participants to get involved in further education and development. There are opportunities for general dental practitioners to develop themselves further in terms of study and research23 but these are very much down to the individual to organise and manage. The lack of a feedback or further development structure within the GDS audit scheme available to participants post-audit serves to stifle interested participants from further development at the structural end of a project.

An unexpected outcome from the GDS scheme was that of change occurring in non-participating practices based on the perception of benefits and change in other practices that had undertaken a project. This has the benefit of one audit project potentially leading to more patient benefit than just in the auditing practice, although the lack of formal dissemination of project findings must serve to inhibit such potential benefit. However, if dentists and their practices are willing and able to make service improvements based on other peoples audit project findings, it does beg the question is clinical audit by each individual practitioner the most efficient way for quality improvement to occur? Could a system of guidelines on a topic perhaps derived by an expert group including experienced auditor's and following review of relevant clinical audit projects and other evidence based sources be a more efficient and focused way of targeting quality improvement than getting every practitioner to undertake their own individual clinical audit project?

A dentist undertaking a clinical audit project who is not the practice principal may find it more difficult to make changes within the practice than if they are the principal. This may be compounded by such things as being recently qualified and lack of effective communication and/or leadership skills. In these circumstances the practise of individual clinical audits may not lead to as effective improvements in services and quality of care as might be achieved through practice wide audit.

Attitudes to quality improvement in healthcare

Quality improvement is widely acknowledged as having an important part to play in dental practice by the dental profession. There is a feeling that clinical audit is also a good tool to use for quality improvement. However, the profession as a whole has only been exposed to this one tool. The financial case for getting individual practitioners to duplicate work done by others in their clinical audits is very weak and the practical case for trying to squeeze every kind of topic that will be party to a quality improvement initiative into a clinical audit envelope is also weak. Fleming and Golding, list many alternative quality improvement strategies they feel could be appropriate for different circumstances and utilised by the profession, including benchmarking, critical incident techniques and action research.2

Whatever options are available for quality improvement within the profession, very little progress is likely to be made without some form of formal organisational framework, either centrally as was the case with the mandatory GDS audit scheme, or locally as with a PCT-based pilot audit scheme such as that undertaken in Essex.24 The present situation (post new dental contract April 2006) requires practitioners to engage in clinical governance activities undertaken by the PCT that they have a contract with.4 Clinical audit is mentioned under one of 12 headings in the 'Clinical Governance Framework' for primary care dental services.25 The level and consistency of quality improvement activity through clinical audit or other initiatives actually taking place at present varies markedly from one area to another. The lack of a clear organisational framework including some method of pooling of results, impacts and outcomes of initiatives, diminishes the opportunity for improvements in one area to be rolled out to others.

Conclusion

The major purpose of this study has been to carry out an evaluation of the GDS clinical audit scheme using evaluative research methodology; in particular, to look in detail at the evaluation of audit scheme structure, process and outcomes. It has focused on the experiences of general dental practitioners who have taken part in the scheme.

Participants found many aspects of it appealing and beneficial but also highlighted several areas in which it could be improved. It is hoped that consideration of the findings in this study will help those who develop dental clinical audit in the future.