Introduction

Qualified dentists who wish to work in National Health Service general dental practice undertake one year's vocational training (VT) where they have a placement in a dental surgery and are allocated a trainer from that practice. There is limited work evaluating the VT year1,2,3,4,5,6,7,8 and this relies on questionnaire responses from VDPs and tends to lack detail on links between ongoing practice and potential developments in training.

Method

In 2001, a pilot evaluation took place in one region. In order to explore more in-depth data, this pilot used focus groups and interviews to explore the VT scheme in the region. A report9 identified themes and issues. To increase generalisability it was decided to replicate the study using the same region plus two other regions, one more rural and one in Central London.

This study was structured using the four levels of Kirkpatrick's model of evaluation (Table 1).10

Table 1 The four levels of by Kirkpatrick's model of evaluation
Full size table

The first focus groups were carried out by a researcher from the earlier study, using themes from that study. Each group consisted of six VDPs, with a gender and ethnic mix in each group; all were present in each session, which was held during one of their study days. They had qualified in more than seven different dental schools and were training in a range of different sized practices. The second and third focus groups used structures suggested by earlier data and used relevant concepts from the literature. These concepts included Kirkpatrick's model, Lave and Wenger's ideas about socialisation into varying 'communities of practice' through work-based experience11 and ideas about practitioners' developing constructs of professionalism.12

Additionally, advisers from all three schemes were interviewed and two focus groups with trainers were carried out. Data also included an observation of assessed presentations on case studies and discussion on case study portfolios.

Focus groups and interviews were audio-recorded and transcribed. Verbatim quotes are reported in italics. To preserve confidentiality the source of each quote has not been identified.

Results

The data were analysed using Kirkpatrick's model.

Evaluation level 1: reaction

General

There was overwhelming support for the vocational training year. It is seen as 'a safety net' and an excellent transition into general practice.

In the final focus groups there was complete agreement about the value of having this opportunity to earn a standard salary, the day release programme, the social aspects and peer support and the 'trainer next door'.

The appointment process

The one area where extreme dissatisfaction exists is with the appointment process, from both the trainer and trainee perspectives: 'a nightmare', 'timing is very bad'. The problems included the schemes all coming out at different times, occasional gender discrimination (against female dentists) and being offered a place which was not the applicant's preferred choice.

'So it's difficult if you get offered a job right at the beginning... it's not your first choice. You're in a dilemma, whether I should accept this one because I might not get another one later.'

One trainee suggested that nurses should be involved in the selection process (though elsewhere this already happens). Trainees admitted that their choice of region was mainly to do with lifestyle expectations. All participants were asked for solutions to these problems but no group was able to offer feasible alternatives, though some thought that a one day recruitment fair in one location might solve some of the issues.

Induction process

There was general agreement that it is really important to spend a few days in the practice, getting to know the others perhaps on a social basis, before starting to see patients.

Tutorials

As in the earlier study, huge variations were evident in tutorial practice.

(a) Timing

Some had timetabled tutorials every week, eg Wednesday afternoon.

VDPs were not happy where tutorials started at the end of the working day:

'My trainer always wants it at four o'clock and he goes on and on and on, yeah, so if I say I'll go at four then I know that I'm going to be there till seven o'clock at night.'

Most were quite happy to discuss issues during their lunch hour but wanted that to be at their own initiative rather than the weekly tutorial or work discussion initiated by the trainer.

(b) Structure

Structured tutorials, with a list of topics to be discussed were appreciated, particularly at the beginning of the year. It is worth noting that some of the trainees had no experience of this and only had tutorials based on their own expressed needs. Most were able to request tutorials on specific topics. Topics included business as well as technical aspects.

Particularly appreciated were 'demonstration tutorials' where the trainer arranged an afternoon of appointments on particular treatments, and the trainee acted as nurse working with the trainer.

(c) Relationship

From comments made, it is clear that there is a wide variety of trainer/VDP relationships, from 'My tutor is a very good friend of mine', to 'I try to avoid him at all cost.'

The day release programme

The day release programme was highly valued in all three schemes:

'You should have them twice a week!'

'Gives you a break (from clinical work) and you can reflect back on the past week.'

All enjoyed problem solving sessions, based on their experience since the last 'day out'. The quality of speakers was commended. All three groups mentioned peer support, and two groups, the regional ones, mentioned social aspects on the night before the study day.

There were some problems with sequencing: 'litigation' and 'business' were appreciated later in the course. One scheme had a day where VDPs brought in their dental nurse and worked together on four-handed dentistry. This was highly valued but they wished it had been done much earlier in the year.

VDPs were uncomfortable where two lecturers overtly disagreed on one of the study days. They felt this undermined their confidence in choice of treatments.

Advisers

All five advisers working on the three schemes were highly evaluated. Having additional access to the regional adviser on the day release programme was seen as the optimum situation.

Advisers were seen as support for when there are problems with a trainer and VDPs liked having two available in case there were issues which were difficult to talk to one about, because of the adviser's practice interests.

Evaluation level 2: learning

In order to evaluate at this level, it is necessary to identify the intended learning outcomes and then to analyse whether they have been achieved. The objectives of the vocational training year are outlined in the professional development portfolio (PDP) but are rather vague for assessment purposes. A slightly more detailed model was developed, based on the earlier study and the first round of focus groups. This model was presented subsequently to both advisers and trainees. Additions were made but there was general agreement that the model described the intended areas of learning in the VT year.

Skills and knowledge acquisition: a model:

  • Clinical knowledge: diagnosis, treatment options, ethics.

  • Technical skill: appropriate use of tools, speed.

  • Social and communication skills: liaison with nurse, liaison with other colleagues, diverse patients (children, adults with learning difficulties, anxious, 'smelly', 'difficult'), discussing treatment options and associated patient charges.

  • Administrative/business skills: NHS paperwork, profit, charging, selling, Health and Safety at Work Act, selecting insurance, legal expectations.

Methods of assessment

As with any training course there are a variety of assessment options and a variety of players who are able to assess the different aspects. In this case, the person best qualified to carry out assessment of all aspects is undoubtedly the trainer. The adviser can also carry out assessment. Self assessment through the PDP is central to the scheme. At present very little use is made of assessment by peers, patients or nurses despite the fact that in Scotland there have now been several years of development of a patient assessment questionnaire (PAQ) which has provided useful feedback on interpersonal skills in vocational training.13

Assessing clinical knowledge

VDPs are already qualified dentists but there was general agreement that they needed to continue to expand their clinical knowledge. In particular they needed to develop knowledge of the treatment options available to patients under the National Health Service.

This is formally assessed through the case studies near the end of the course. There are many opportunities for informal and formative assessment by the trainer, but it was not clear from the data how this is achieved. Elsewhere, other models based on competencies and portfolios of evidence have been developed.14,15 These forms of assessment were not in use in the deaneries researched in this study.

In one mid year focus group, all six respondents claimed to have received a lot of negative criticism but no positive feedback from their trainers. The amount of direct observation used by trainers varied. In some practices the trainer constantly supervised, in others if 'passing... he would look' and, more worryingly:

'My tutor makes a point of not coming and intruding in my surgery and that would undermine me as a dentist in front of my patient as well.'

The technology used to prepare the case studies (cameras, X-rays, video clips) did not seem to be used in an ongoing way for trainer, peer or self assessment.

Assessing technical skill

As in the earlier study, the speed with which VDPs were expected to deliver treatment, in order to achieve income, came as an initial shock.

Appendix 1 in the PDP provides an ongoing record of weekly activity and therefore is a good assessment of the development of speed (linked to NHS income). Competence in the use of instruments can presumably be inferred from increasing speed and from observation by the trainer where this is standard practice. With speed, with or without positive feedback from the trainer, comes the increase of confidence.

Assessing social and communication skills

The dental nurse

One of the strongest themes in this study was the problematic nature of the VDPs' relationship with their dental nurses. It was evident all the way through and took many different forms. Most VDPs reported that they had had no experience of working with a nurse in their dental school, though some had practised four-handed dentistry with peers as clinical partners. Initially the relationship with practice nurses was ambivalent and the VDPs were unsure how to deal with difficult situations.

'You see I've never dealt with a person under my sort of employment...'

Many nurses received high praise in this study:

'I think on the timing you've got to look at the nurse's viewpoint as well. They're pretty good most of the time.'

'My nurse is brilliant actually. She's very good.'

'They're very sort of tolerant of us young pups.'

By mid year, relationships had settled down, though not necessarily in the same way:

'I had a very different opinion of nurses when I first came, I've got a completely different one now; before it's well I'm the dentist you're the nurse I want this done that done, I've completely changed. If my nurses are getting stressed out then I'll put the X-rays through the developer or help her, because it gets a good relationship she'll realise that she'll do me favours when I'm running stressed.'

'Yes, pretty much I let her do her stuff, I don't go to the waiting room to get patients or things like that.'

'My nurse is horrible she starts crying and stuff like that, she hit me behind a basin once, she was arguing and raising her voice and I thought it was all building up inside. And it was really unprofessional and I wasn't happy about it at all.'

The trainees' relationship with their nurse appears highly significant and as one trainer said: 'they see an awful lot more of their nurse than they do of their trainer – more than their family.'

However, nurses seem to be excluded from the assessment process. Two schemes had involved them in study days, one for four-handed dentistry and the other for a day on avoiding back problems, but there were no examples of VDPs using feedback from their nurses formally or informally, despite widespread acknowledgement that the nurses had specific useful skills that helped them through the year, eg making appropriately timed bookings, talking to children.

It is probably worth mentioning here that by the first focus group one VDP had trained a new nurse and by the final focus group a different VDP had trained five new dental nurses! It was generally agreed that working with one experienced nurse, with whom you had a good relationship was the optimum situation.

Liaison with colleagues and treating diverse patients

There were no instances of particular difficulties in these areas.

Discussing treatment options and associated patient charges

There was general agreement that patients in the dental school were 'nicer' than those in practice. Discussion indicated that this was because in practice charges were made for treatment whereas treatment in dental schools was free. (One trainer pointed out that dental schools in Eire do charge for treatment). By the time of the final focus groups, some VDPs were still not comfortable charging for NHS treatment.

'I feel guilty – especially poor elderly people, I feel so guilty. I hate them having to pay for it, when I know that they've barely got enough money to eat – and there has been a couple of times when I've done something and just not charged them for it.'

'It feels like we're holding them to ransom – we none of us went into it to do that.'

'I hate that, I leave it to my nurses or to my receptionist. It's like all of a sudden we're a sales rep.'

VDPs felt restricted in what they could talk about while treating a patient. They said they were careful to avoid certain topics because of patient reaction.

'I've stopped mentioning that I'm going on holiday – you can't say that to a patient, you can't make polite conversation, because they say 'Oh well I've just paid for your holiday.''

This awareness of patients' attitudes had led to one VDP never admitting he is a dentist in social situations and another cycling or walking to work and leaving his 'flash car' at home.

Assessing administrative/business skills

From the beginning of the VT year, a key aspect valued by all had been in the area of dealing with NHS paperwork. It was covered by trainers, guest speakers and nurses. It is easily assessed as it is a legal requirement that it is done correctly.

Self assessment and the PDP

In all the above categories it is valid to use self-assessment. This should be captured in the PDP. The PDP is an A5 clip file. It has a clearly defined structure, clear explanations for each section, built in involvement of trainer and adviser and is soundly based in reflective practice theory. The VDPs detest it! (This was also the case in the earlier study.)

In the first focus groups they were saying 'I just don't know what to write'; 'just racking your brain for the sake of it.'

By round two, when they had only to fill in two sheets of A5 a month: 'waste of time'; 'just silly'; 'just a chore'; 'it's a hassle'; 'that's rubbish'.

'...most people, they do it late on a Wednesday night or first thing in the morning in the car.'

In the final sessions, all three focus groups said that the PDP was one of the things that could be improved on the course.

'You just write what you can remember.'

'Load of rubbish – though it still creates a time when you and your trainer absolutely have to meet up.'

'We did the last nine months all last week!'

'Just make it up anyway.'

'Repetitive.'

'Not even a true reflection.'

'It was explained quite well.'

'It was done in a negative way though, 'cos you've got to fill it out.'

'I feel like I'm writing it down to keep them happy.'

'I've never liked writing feelings down... that was the hardest thing.'

'I cannot understand the action plan.'

'Would be OK but to do it every single week, or every single month, every single week in the first 12 weeks... that was a chore.'

There are two surprising aspects to this. Firstly these are all intelligent people and in other medical professions there is a much greater demand to achieve a reflective process than in this portfolio. Indeed, reflective practice is a key aspect of the post-technocratic curriculum, in which learning is based on professional practice.16 The PDP also gives permission to write the reflection in other ways if they do not like the given format. Secondly, the next stage in the reflective process based on the Kolb Experiential Learning Cycle,17 abstract conceptualisation (or the use of others' ideas), is highly valued by the VDPs; they look forward to discussing incidents, and these sessions are seen as high spots on their day release programme. The 18 VDPs were only able to give two examples of using the PDP to develop professionalism.

One adviser used the PDPs to evaluate the trainers he is responsible for. He confessed though that questions he writes in the PDP, in response to VDP reflections, are not necessarily followed up by him at a later date.

The PDP is designed to close the loop in the professional learning process but seems a long way from achieving this.

Trainer comment on assessment

The current regulations do not allow VDPs to fail the VT year. In effect they have a certificate of attendance, which allows them to practise as a NHS dentist. They are qualified dentists before they commence the year therefore it is argued that they cannot be failed at the end of another year's training. However, almost all the trainers in the study were quite clear that they had had VDPs that they did not consider competent at the end of the year. There were various reasons for this, both in terms of clinical/technical skills and communication skills. They were adamant that if they could have failed these individuals, they would have done. Some had counselled the incompetent general practitioner back into hospital practice or academic research. In one instance the VDP was being kept on as an associate so that his training could continue.

In essence this means that assessment is taking place, but in terms of evaluation, the course is not achieving the objective that 'the VDP should be eligible to practise unsupervised as a principal within the GDS.'18

Evaluation level 3: behavioural changes

In this post-technocratic curriculum, the VDPs are constantly implementing their learning though, as indicated above, they find it difficult to express their learning in reflective writing. It is clear from the data that what is happening through this process is the development of professional judgement. They commented: 'you don't need to be as efficient at dental school: a different ball-game'; 'at dental school you do what you're told.'

As their confidence grows they learn to trust their own judgement and not be bullied by demanding patients.

'Had to stop myself falling into the trap of OK we can do this for you, when it actually didn't need to be done.'

This process is congruent with Lave and Wenger's ideas about situated learning in communities of practice. They suggest that learning is a complex social process. At the beginning of training the trainee is an outsider. Being accepted as a VDP allows 'legitimate peripheral participation'19 into a new community of practice. As they practice they 'learn to talk' and to develop their identity as a general practitioner.19 In this instance the wider community of practice is NHS dentistry, the particular surgery is a more localised version of that.

Although some became used to 'selling' by mid year, the whole area of charging, which is central to this particular community of practice, caused difficulties.

'And yet they'll stick a hundred and forty quid's worth of tyres on their car. The mention of a seventy pound crown that can last them 15 years – ohhh!'

In round 2 they were asked 'Do you feel like a professional now?' At this stage about half the respondents said that they felt like a dentist and a professional

'Well I mean I guess you do feel like a professional when you're actually in your working environment... there are people in pain... and you can actually do something to help them out, so that way it does feel good.'

'Overall, not really... I don't feel like a professional. ...I still feel like a post graduate student so to speak and till I finish this, and maybe for me it's a personal thing, I get my own private space and be an associate and I won't feel like a professional until then.'

As the training neared its end there were examples of where VDPs had become aware of their own professional judgement. In one incident the VDP had stepped in to deal with a patient being belligerent to a receptionist and told her that if she did not calm down he would refuse to treat her. Another told of his pleasure when his trainer had started to ask his opinion on difficult cases. Another explained that she no longer made instant judgements, as to which treatments would be chosen, as patients arrived.

It seems clear from the data that as VDPs begin the vocational training year they are not just trainees but have been admitted to legitimate peripheral participation in a community of practice. On first inspection their training is in the hands of a trainer and an adviser. But it is much more complex than that, many others are involved as suggested by Lave and Wenger:

'This uneven sketch of the enterprise (available if there is legitimate access) might include who is involved; what they do; what everyday life is like; how masters talk, walk and work; and generally conduct their lives; how people who are not part of the community of practice interact with it; what other learners are doing; and what learners need to learn to become full practitioners.'20

The VDPs' learning is in the hands of trainer, advisers and also other colleagues, peers, nurses, receptionists, and patients. During this year they move from qualified dentist to professional general dental practitioner.

Evaluation level 4: results to the organisation

Forsyth et al. state that evaluation at level four is 'difficult and time-consuming'21 yet this is in effect the main purpose of this study. All those associated with the VT year value it highly as a supported transition to general practice. There was concern from both groups of trainers that the VT year is not only 'finishing school' for dentists but is now providing core practice and instruction which in the past would have been delivered in dental school. The implication of this is that the 'year' may need to be extended in the future.

It is also worth mentioning the benefits to the trainers under this heading. Being a trainer provides 'something else at work apart from clinical dentistry.'

'They question everything you do... makes you go back and assess what you do.'

'Breath of fresh air, having younger people in the practice... Good for the practices to have a VDP.'

'Tremendous satisfaction out of seeing them become confident, competent colleagues.'

Particularly in the rural areas, the VDP provides 'extra manpower in the practice' and is a good way of expanding the practice.

Recommendations

On the basis of the data it would seem sensible to continue the VT year in very much the same pattern as exists already. However there are possible changes that could be considered.

  1. 1

    The appointment process is the cause of stress and dissatisfaction for trainers and trainees. It is suggested that discussion continues to seek solutions.

  2. 2

    The day release structure is well received and should continue: maintaining the high quality of speakers, activities and visits; having two advisers available where possible and ensuring appropriate sequencing.

  3. 3

    The tutorials should be time-tabled in the working day as already recommended and be seen to have a structure, perhaps provided by the advisers. There should be at least one tutorial where the nurse(s) who works with the VDP is included and a dialogue can take place in a calm and professional way on how they can best work together.

  4. 4

    Assessment also appears to be an area that can be developed. Perhaps there should be a minimum percentage sample of treatments that has to be checked by the trainer (either in person or through photographic technology). Nurses and patients could be more involved in terms of informal feedback, particularly on communication skills. Communication skills as an area for assessment has also been highlighted in the development of the Key Skills portfolios in the West Midlands.15 The key question around assessment is whether the system should either fail VDPs or withhold certification until an acceptable standard has been reached.

  5. 5

    Reflective practice should be an integral part of the development of professionalism. The PDP appears to give a good skeleton for reflection but it is not working in practice. There may be several ways forward in this. Advisers could experiment with encouraging VDPs to experiment with different ways of doing individual reflection; a written journal; mind maps; before and after photographs; using prompts from a variety of models. It is probably most important for advisers and trainers to really believe in the process on which the PDP is based and it may be that they need training sessions and more experience of keeping up a reflective process themselves. They need to emphasise that professional development and reflective practice should continue throughout their career and that they will not always have the time and opportunity to enjoy the group reflective/problem solving sessions that they have in the VT year. Other experiments might involve a requirement that new learning theory is specifically linked to critical incidents, and that action planning can only be completed following a group problem solving session where others' ideas can be integrated.

Conclusions

The vocational training year is a very good learning experience for newly qualified dentists who wish to practise in the NHS or privately as general practitioners. There were many examples of good practice in all three schemes examined. There are inevitably areas where more thought and consideration could improve the experience for all those involved in the achievement of full participation in the general dental practitioner community of practice.