Introduction

There have been a number of studies evaluating the efficacy of clinical audit (CA) schemes in general dental practice.1,2,3 These studies have assessed its value for improving service delivery in dental practices and its use as a clinical governance tool.4 This paper develops the discussion through an evaluation of CA as an educational activity within UK dental foundation training (DFT) from the perspective of the foundation dentists (FDs).

Background

Audit is a quality improvement tool (Fig. 1), which became mandatory in the NHS general dental services (GDS) in 2001.5 It has been defined as 'The process used by health professionals to assess, evaluate and improve the care of patients in a systematic way in order to enhance their health and quality of life'.6 FDs in the region of this study are required to complete a CA on a topic of their choice.

Figure 1
figure 1

The audit cycle

Full size image

DFT is a period of training following initial qualification that aims to 'produce a caring competent reflective practitioner able to develop their career in any branch of dentistry to the benefit of patients'.7 It is mandatory for UK graduates who wish to work in the NHS GDS.

Considerable time and resources are devoted to CA teaching within the training year but there has not yet been an in depth investigation into its educational benefit or the learning outcomes for young dentists.

The educational process should develop competencies within the domains of clinical, professionalism, management and leadership and communication skills.7

The aim of this qualitative study was to investigate and evaluate the educational effect of CA in NHS GDS from the perspective of FDs and their training programme directors (TPDs). The findings should be useful to FT trainers, TPDs and postgraduate deans, as well as to dentists and FDs themselves.

Methods

Participants

A purposive sample from two of the seven schemes was used to conduct focus group (FG) interviews. The interviews were triangulated with a third group consisting of TPDs to include their views about FD's learning.

The aim of purposive sampling is to focus on the characteristics which may affect perceptions and ensure that most views are represented. Groups were chosen to reflect the diversity of ethnicity in the region (Table 1), gender, place of qualification and 'A' level or graduate entry. This information was captured by asking the participants to complete a data sheet.

Table 1 Ethnic characteristics of the participants
Full size table

Group interviews

Group interviews were undertaken. These have been defined by Vaughn8 as 'an informal discussion among selected individuals about specific topics relevant to the situation in hand'. They allow data collection from a large number of subjects simultaneously9 and provide information about why people think and act as they do. They allow observation of the educational groups in context, allowing participants to express themselves more fully than in traditional surveys. They can encourage 'snowballing' where one response generates a 'chain reaction' from other group members.8 Some criticisms have been made of group interviews – participants may make up the answers, dominant individuals can influence the results and participants may tend to intellectualise.10 However, there is no single perfect method of evaluating complex social interactions and focus group studies can provide additional insight. The interviews with FDs were triangulated with those of their advisers to mitigate some of the weaknesses of group interviews.

It was decided to use pre-existing foundation groups as this produced a more naturalistic qualitative study.11 There are some disadvantages to using pre-existing groups, as opposed to the classic focus group format described by Krueger.10 Their well-established dynamics and communication may be hidden from the data collector (for example, use of the 'knowing look') but this is outweighed by the advantage of observing the educational group in context.

Recruitment

Letters of invitation and information were sent to the directors and trainees from the selected groups and the interviews were arranged at their study group centre.

Ethics

Group interviews require careful ethical considerations because the individuals are known to each other. This could deter them from taking part in the study or voicing their opinions. Invited participants who did not attend were given the opportunity to express their views in a written format, and after the interviews all participants were given the opportunity to post comments anonymously to the researcher.

Other concerns were addressed by producing a set of 'ground rules' and participant information leaflets. The study received guidance and ethical approval from a supervising university and advice from the local NHS research and development lead.

Design of interview schedule

Cannell3 developed a framework for evaluating CA, following the principles for evaluation of health services proposed by Donabedian.12 This was modified to further investigate the educational outcomes of CA teaching and research objectives were developed (Table 2). Qualitative research is characteristically flexible, and the focus and objectives of the study changed as it progressed.

Table 2 Research objectives
Full size table

From these objectives a series of semi-structured interview prompts were developed (Table 3). The prompts provided a framework for steering the discussion but allowed the flexibility to explore issues considered important by the participants. They were modified slightly for the TPD interviews focusing on their views of the FDs' experience.

Table 3 Prompts for group interviews to evaluate FD's experience of CA (from Cannell3)
Full size table

Data collection

The interviews were conducted in the early spring and summer of 2014, after each group had given their audit presentations. The interviews were moderated and digitally recorded by the chief researcher.

Data analysis

The data was transcribed verbatim by the chief researcher into QSR Nvivo computer-assisted, qualitative data analysis software. The ideas and concepts expressed were coded line by line. Controversy exists as to whether the unit of analysis in focus groups should be the group or individual participants.13 Analysis at group level allows a scrutiny of themes and interactions between groups. What individuals say in focus groups cannot be treated as personal disclosure because it is modified by the group interaction. Personal characteristics were looked at more closely when the background of the individual appeared to be significant. For instance, the graduates from European dental schools had not been introduced to the concept of audit as undergraduates and they valued the examples of previous FDs' work when grappling with the concept of audit. The data was indexed and categorised and a hierarchy of themes and sub-themes was developed. Relationships and comparisons were drawn out. Then, theories to answer the research questions were developed. The themes which developed for the analysis are summarised in Table 4.

Table 4 Themes and their descriptors20
Full size table

Results

The format used for reporting quantitative papers14 is not entirely suitable for qualitative data. There are circular linkages between the methods, data collection and analysis in qualitative studies.15

Learning about audit

From the interviews it was apparent that the module improved the FDs understanding of the audit cycle.

Topic selection

FDs selected topics that were original and important. TPDs identified that they had to be manageable in the time available. Originality, importance and feasibility could be mutually exclusive, and an important role of the trainer is to guide FDs in their topic selection. In 'good practices' FDs struggled to find 'something worthwhile that has not been done before', and while TPDs encouraged them to do something manageable and not overly complex, FDs viewed topics done 'a thousand times before' as boring.

One TPD described two kinds of audit, those that are 'diagnostic' – used to identify and solve a problem in the practice, and the 'routine, rolling audits for quality maintenance'. The former provided a richer, more rewarding learning experience for the FD, but were more difficult to do in the time available. Another TPD championed the value of going back and looking at topics that have been audited before in the practice to see if improvements have been maintained, so that audit becomes a 'continuous spiral of improvement'.

Setting the standard

In order to set a standard for good practice, FDs described their research for supporting evidence. There were issues with the availability of library and electronic resources. An interesting finding was that there was some distrust of published guidelines, which were generally easy to access, and FDs preferred to find the original papers, though this material was much more difficult to access.

Not many interviewees were specific about how they set a standard for their audit. There was some confusion about the difference between standards and targets and what percentage target should be set for the first cycle. Some FDs agreed a target for improvement at staff meetings but the directors thought that this 'tipped off' team members about the audit – 'There's an argument about having a discussion before you start the audit in a staff meeting, does that pre-prime people?'. TPDs were concerned about this 'Hawthorne effect' (changing behaviour by observation) but FDs did not think that this initial discussion of standards and targets changed practice.

Data collection and analysis

The FDs designed many different types of audit and collected various types of data such as patient questionnaires, clinical notes and date stamps on sterilised instrument packs. They learnt how to design data capture sheets, and adopted various strategies for collecting data. Some conducted all the work themselves, while others recruited other team members.

Some of them found the data analysis and manipulation difficult but most found ways of analysing and presenting their data, recruiting colleagues, trainers and others to help them. They did not think that additional lectures would have helped in this process but considered that: 'maybe a guideline booklet (would help). 'I don't know how I'd feel sitting through a whole seminar, cause [sic] really, you only pay attention when you've got to do it' (FD).

Presentations and feedback

After data analysis, the next stage was to make service changes, which involves team meetings or individual discussions. Some FDs find this daunting and appreciated the advice and support of their trainers before the meeting. TPDs thought that 'part of the benefit of audit is actually running a team meeting'. The dentists then re-audited their topic and gave a presentation to their study group. These presentations were marked using the criteria designed by the Faculty of General Dental Practitioners to assess audit. The best audit from each group presented their study at a regional meeting of all FDs. One cohort saw these presentations before conducting their own audits and found the example of previous FDs work helped them understand the principles of audit.

Despite some reported struggles with IT, most FDs managed to put together a presentation. 'Their presentations are usually outstanding. That's why I think their IT skills are good.' (TPD)

Outcomes

Overall FDs thought their audits had a positive effect on performance and brought about a positive change. For some it identified previously unknown weaknesses in their own performance, eg 'Yeah, it definitely changed my own idea, because I thought I was very good at record keeping.' (FD).

FDs cited improvements in: record keeping, instrument storage, compliance with 'Delivering better oral health' guidance on fluoride varnish use, giving oral health education, antibiotic prescribing, taking consent and other improvements in patient care. On the whole, they thought these changes would be permanent in their practices. TPDs were less confident of permanent change in service delivery. They cited FDs who had gone back and repeated audits done by previous FDs in the practice and found that standards had slipped

Learning through audit

The Foundation curriculum7 provides detailed descriptors of competencies within four domains. Evidence or discussion of such competencies provided one of the themes for the data analysis.

Clinical domain

There is some learning in the clinical domain and the small group teaching allows them to learn from each other: eg, 'I've already been speaking to... colleagues [other FDs in the group] and that about them and asking things that they've researched.' (FD).

They were keen to use audit to improve clinical outcomes and thought these topics more interesting than managerial or administrative subjects. However, TPDs thought choosing clinical or technical skills to audit could be problematic. In the early months of their training FDs are building their competence and confidence and may not have a large enough sample set to conduct an audit on a clinical issue. Data collection and outcome measures can be more difficult with some clinical topics. However, FDs who involved other clinicians from the practice conducted successful audits on topics such as antimicrobial prescribing and accuracy of alginate impressions.

Communication domain

CA enabled FDs to improve communication skills such as delivering oral health messages or gaining informed consent. It made them think about getting sensitive information from patients without implying criticism of colleagues. It encourages them to communicate with patients and colleagues, understand concerns and develop formal and informal communication skills, verbally and through print and other media (for example, by designing questionnaires and information leaflets). They learn how to lead meetings and give presentations to colleagues and provide written reports.

Management and leadership

CA is a very useful tool for learning practical leadership and management skills. Audit is time consuming and requires good time management. There was evidence of FDs learning skills such as delegation, and teamwork by recruiting the support of others to collect their data. Some FDs undertook their audit in Smile Month and found this to be an effective strategy to get the team engaged in audit.

Setting a standard encourages research into the evidence base and some FDs described detailed knowledge and improvement of processes such as infection control.

'Awkward' conversations

When there was variance from good practice, some, but not all, FDs intervened. Several trainees and TPDs talked about 'awkward conversations' they had with other team members. 'After I'd done my first cycle, I showed my trainer my results and he identified that some of the conversations would be awkward with some staff members, so he kind of selected like [sic], “I'll sit in with you on this conversation that you are going to have with this dentist, and we'll talk to him together about it.” So I think that's quite useful.' (FD)

Professionalism

The attitude of the trainer is important to FDs and an important role model in this domain. Occasionally, the worst performing dentist was the trainer; for example, in following current guidance on antibiotic prescribing. FDs worried about challenging their trainers on such issues but on the whole found their trainers' response mature and that they were willing to make changes. However, some trainers did not take their findings seriously and 'swept the findings under the carpet'. Having awkward conversations with colleagues can be unpleasant. However, the Francis Report16 on standards of care recommended that trainees should be encouraged to ask such questions and CA provides a framework within which such questioning can take place. FDs and programme directors both thought such managerial skills were important to develop, and that there are few other teaching experiences which allow this kind of learning.

If FDs have poor time management skills or have insufficient time to conduct audit, they can be tempted to behave unprofessionally. Choosing a difficult topic or not thinking through how data should be collected can have a knock on effect on their professionalism. For example: 'You just think, oh my God, people aren't handing out that, people aren't doing this, it's got to be in for this time, why don't I just make it all up? Not that I did, but...'.

Role of the trainer

FDs appreciated the support of their trainers but also valued being responsible for their own work, eg 'It was my idea and I ...pushed for myself to do it, and I got him [Trainer] on board, but as soon as I had chosen, he was like, “Right and now let's do this.” And I felt like, whoah! Let's tone it down a little bit.'

They appreciated being given space to try their own ideas out in the first instance: 'you need to try something, if it doesn't work, then go to your trainer and say, look, I've tried this data collection, it's not working! What else can I do? And then they'll chip in with ideas.'

Discussion

There is evidence that FDs gain a great deal from the CA module. Much of this learning would be useful to any dentist conducting CA with his or her team or engaging in team training. There is some learning within the clinical domain but the most learning takes place within the professionalism, communication, and especially management and leadership domains. FDs thought that the best way of learning about audit was to do it and valued the freedom to choose subjects they identified as important. The role of the trainers and TPDs is important here. Their aim should be to allow the FD to develop from their existing knowledge base, providing support when required but carefully gauging that support. If the trainer intervenes too early, FDs think they are taking over ('It's my audit!') but if they leave their intervention too late the FD can run into difficulties.

The idea of gauging the trainee's existing zone of knowledge, and helping them develop from it, fits well with existing educational theory,17 as does the idea of providing a scaffold of support when needed but withdrawing this as the trainee's skills and competencies develop.18 It should be noted, however, that sometimes this development is not a linear progression, and trainers should carefully observe and communicate with trainees so they can help if difficulties arise. Trainers should work in the 'Goldilocks Zone' when considering intervention: not 'too hot' with their support, nor 'too cold' but providing just the right amount of help when needed.

Using pre-existing groups allowed some observation of the group dynamic in education. FDs associated positive attitudes with small group work and reported learning from each other. Educational theorists think that social interaction with teachers and other learners plays a fundamental role in the development of understanding19 and this view would be supported by the comments of the FDs and TPDs in this study.

They appreciated examples of audit from previous FDs, who may have been closer to their own 'zone of knowledge' than their trainer or TPD. It is recommended that TPDs bear this in mind when planning CA programmes and ideally allow an FD from a previous year to give an example of their audit.

Many FDs developed their IT skills. Training providers should improve access to library and online resources and nhs.net email addresses would enable FDs to use the NHS Athens database more easily. It is interesting that personal interaction with trainers, other FDs, and team members etc were felt to be more important resources for their learning than IT. Audit is a very good way of developing teamwork skills.

Conclusions

The major purpose of this study was to carry out an investigation into the perceptions of FDs and TPDs about the educational outcomes of CA in DFT using qualitative research methodology. It has focused on what FDs think and what they learn. However, the findings should be useful to any GDP conducting an audit or involved in team training.

There is evidence of considerable learning in all the domains of the FD curriculum. CA can reach areas that other educational interventions cannot but there is some question about its long-term benefit to practices. Trainers and programme directors should consider time management, utilising support from practice teams, and associating CA with other initiatives for its most effective use. Improving access to library resources and original research would be appreciated by FDs. Trainers should give the right amount of support to FDs, not spoon feeding them, but being available when necessary, especially if awkward conversations with other team members are required.