Introduction

Newly qualified UK trained dentists must complete dental vocational training, a 12-month postgraduate programme, to be eligible for an NHS list number. During this vocational training year, all trainees are based in general dental practice under the tutelage of an experienced dentist. However, while participation in this training programme is required across the UK, there is not an agreed national curriculum and the content and the evaluation of the training is not standardised across the UK. In particular, legislation in Scotland requires trainees to follow a structured curriculum and undergo mandatory assessment throughout their training year. This does not happen anywhere else in the UK.1,2,3,4,5,6

The current emphasis on developing quality standards in UK dental training has raised the question as to whether the rest of the UK should follow the more formal Scottish format, thereby incurring the costs required to support a formal assessment methodology.7,8 However, there is currently no evidence of the benefit of this system, in terms of better preparing dental trainees for their future careers. It could be argued that mandatory assessment may be more burdensome than beneficial for trainees. The aim of this study was to further an understanding of the impact of mandatory assessment on dental trainees.

In order to further this aim, an issue arose pertinent to the development of dental training in the UK, which is, what outcomes should be used when comparing the trainees - what is a better trained or prepared dentist? Since trainees outside of Scotland are not formally assessed after graduating, it was not possible to compare behavioural data across the geographical cohorts. A stakeholder consultation was carried out to identify appropriate outcomes to assess by questionnaire. Multidisciplinary teams involved in dental training and dental research (clinicians, economists, psychologists, statisticians), and 120 members of postgraduate dental deaneries throughout the UK were consulted. They determined that the vocational training year is meant to increase professional confidence and inculcate professional attitudes in a range of areas pertaining to the delivery of dental care, including clinical and non-clinical aspects of dental practice, continued professional development, professional identity, and their work in general. Support for using measures of confidence and attitude as outcomes for comparison is provided by the psychology literature, showing that greater confidence and more positive attitudes increase the likelihood of being able to perform behaviour. Furthermore, in our previous work we found measures of confidence and attitudes to be useful in understanding, predicting and influencing dental practice.9,10,11,12,13,14,15,16 Therefore, measures of confidence and attitudes relating to the stakeholder-identified outcomes were developed as a means of comparing the impact of the different assessment formats (see measures section for further details).

The following research question was examined:

Research question

Does the Scottish system, with its mandatory assessment, result in better outcomes, compared to dental vocational training in the rest of the UK?

Methods

This was a pragmatic experimental design. Data collection of three cohorts of dental trainees (2003-2004, 2004-2005, 2005-2006) was by postal survey. All dental trainees from the North and North Western Postgraduate Dental Deaneries in England, Wales, Northern Ireland and Scotland (approximately 40% of all UK dental trainees) were asked to complete questionnaires at the beginning and at the end of their vocational training year.

Measures

Questionnaire items were generated from a content analysis of the stakeholder discussions, which identified behaviours perceived as important (at least 80% agreement) to the delivery of dental care. The finalised questionnaire items were then circulated to stakeholders for comment. The response formats for the questionnaire items were based on standardised operationalisation protocols where available.7 All items assessing confidence used a Likert-type scale of 1 (not at all confident) to 7 (extremely confident). Higher scores indicate greater confidence. Items assessing attitude used Likert-type scales of 1 (strongly disagree) to 7 (strongly agree) or 1 (not important) to 7 (very important). Higher scores indicate more positive attitudes.

Confidence in clinical skills was the mean score of their confidence in performing 18 separate behaviours (Table 2).

Table 2 Comparing Scotland and other UK trainees on confidence in performing individual stake-holder identified behaviours. Baseline (pre-training) scores were taken into account as covariates in all analyses
Full size table

Confidence in non-clinical skills was the mean score of their confidence in performing nine separate behaviours (Table 2).

Confidence in continued professional development was the mean score of their confidence in performing four separate behaviours (Table 2).

Trainees were also asked how difficult they thought it would be to perform each of these behaviours on a Likert-type scale of 1 (difficult) to 7 (not at all difficult).

Attitude to vocational training and its assessmentwas the mean score of attitudes toward ten separate elements:

  • Maintaining a training record book

  • Reflecting on personal strengths and weaknesses during the vocational training year

  • Attending the DVT study day programme

  • Having regular tutorials with a trainer

  • Experiencing peer review during the vocational training year

  • Carrying out reflective self-assessment during the vocational training year

  • Completing a project during the vocational training year

  • Practicing as a vocational trainee before embarking on unsupervised practice

  • Being informally assessed in any way during the vocational training year

  • Being formally assessed in any way during the vocational training year.

Although all UK trainees do not experience all of these elements during their postgraduate training year, the stakeholders agreed that trainees should nevertheless have enough experience of each, including informal and formal (mandatory) assessment, to formulate an opinion as to whether they thought they were beneficial/harmful, useful/useless, or comforting/worrying.

Attitude to continued professional development was assessed as the mean score of how useful trainees thought it would be to perform each of four separate behaviours:

  • Keeping track of new technology about clinical practice

  • Keeping track of new evidence about clinical practice

  • Monitoring how new evidence about clinical practice matches what they currently are doing

  • Changing clinical practice in line with new evidence.

Trainees were also asked their intentions of performing each of these continued professional development behaviours and how stressful they think they would find performing each of these continued professional development behaviours.

Attitude to professional identity was the mean score of attitude toward being a caring dentist, a competent dentist, and an ethical dentist. Trainees were also asked how difficult it would be to have these professional characteristics.

Attitude to work was assessed as the mean score of 25 items adapted from an attitude to work measure previously used with dentists.17 For this measure, items were scored using a Likert-type scale of 1 (strongly disagree) to 5 (strongly agree). Items were reversed scored where disagreement signifies a more positive attitude, so that higher scores signify more positive attitudes (see Table 3).

Table 3 Comparing Scotland and other UK trainees on their attitude to work. Baseline (pre-training) scores were taken into account as covariates in all analyses
Full size table

Demographic measures included deanery, cohort, gender, age, and general health (GHQ-12).18

Statistical analysis

measures were tested for acceptable internal consistency using Cronbachs alpha. The impact of the training year was examined using paired t-tests. Group and cohort differences were examined for overall measure scores and at the item level using Chi square, t-tests and generalised linear models, with Bonferroni corrections where appropriate. Statistical significance was based on two-sided tests with p ≤0.05 as the criterion.

Results

In total, 555 trainees returned both the beginning and end of training year questionnaires, giving a response rate of 72% (n = 247) from Scotland and 64% (n = 308) from other UK (North Western England n = 58; Northern England n = 98; Wales n = 102; Northern Ireland n = 50). An outcome analysis revealed no significant differences between trainees who returned only the beginning of training (baseline) questionnaire (n = 710: response rate of 93%) and trainees who returned both questionnaires for any baseline measure (t-tests, p <0.05 level).

Cohort and area analysis

This study included three trainee cohorts from sequential years (Cohort 1 (2003-2004): n = 208, Cohort 2 (2004-2005): n = 168, Cohort 3 (2005-2006): n = 179). There were no significant differences between these cohorts in gender, age, general health, or number of returned questionnaires. There were no significant differences between Cohorts 2 and 3 in any measure. However, Cohort 1 was significantly more confident than either of the other cohorts at baseline in their clinical skills (F(1,554) = 8.02, p <0.001), and non-clinical skills (F(1,554) = 20.79, p <0.001), and had a more positive attitude toward their continued professional development at baseline (F(1,553) = 4.07, p <0.05). However, since there was no within cohort difference at baseline between trainees in Scotland and the rest of the UK, the three cohorts were combined for these analyses.

This study examined data from five UK dental deaneries. Bonferroni posthoc tests for multiple comparisons identified only one baseline measure showing a significant difference by area. Trainees from the Welsh Dental Deanery showed a significantly more positive attitude toward continued professional development at baseline than trainees from any other Dental Deanery.

Given that there were cohort and area differences at baseline, beginning of training year scores were taken into account as covariates in all analyses.

Confidence

a) Clinical skills

Generally, as can be seen by the overall means in Table 1, all trainees in this study were very confident in their clinical skills scoring means ≥ 5 out of a possible 7. The item analysis (Table 2) reveals that 17 of the 18 behaviours individually score means ≥ 4 out of a possible 7). The exception was performing an apicectomy.

Table 1 Outcome measures, example items, and GLM results comparing dental graduate training in Scotland (mandatory assessment) and the rest of the UK (no formal assessment) over three cohorts (2003/4, 2004/5, 2005/6). Baseline (pre-training) scores were taken into account as covariates in all analyses
Full size table

When compared to the rest of the UK, trainees in Scotland are significantly more confident in their clinical skills. The item analysis revealed that trainees in Scotland were significantly more confident in 10 of the 18 behaviours assessed (Table 2).

b) Non-clinical skills

Generally, all trainees were very confident in their non-clinical skills, with 8 of 9 behaviours scoring means ≥ 4 out of a possible 7. The exception was developing a business plan.

When compared to other UK trainees, trainees in Scotland were not significantly different in their overall confidence relating to non-clinical skills (Table 1). However, the item analysis showed that they were significantly more confident about knowing when prior approval of treatment from a dental practice board was necessary. There were also trends toward a significant difference (p <0.10) in their being more confident in completing the GP17 and being able to interact well with more experienced dentists.

c) Continued professional development

Generally, all trainees were very confident when it comes to continuing their professional development, with each of the four elements having mean scores ≥5. They also think that it should not be difficult to perform each of the elements (mean scores ≥4).

Most UK trainees (99%) said that they intended to change their clinical practice in line with new evidence, and that they are confident (80%) that they can do so. Although 90% of all trainees in this study think changing practice in line with evidence is useful, 36% think it is stressful and 41% that it is difficult.

When compared to other UK trainees, trainees in Scotland were significantly more confident in regard to their continued professional development (Table 1), specifically in keeping track of new technology relating to clinical practice and also in monitoring how new evidence matches what they do (Table 2). There was no significant difference between trainees in Scotland and the rest of the UK in their confidence in their ability to change what they do in line with new evidence, although trainees in Scotland did expect to find performing the continued professional development behaviours overall significantly less difficult (Scotland mean = 4.1 (0.6); rest of UK mean = 3.9 (0.6); F(1,550) = 6.68, p <0.01). Trainees in Scotland also had significantly higher intention to keep track of new technology about clinical practice (Scotland mean = 6.2 (0.6); rest of UK mean = 6.0 (0.5); F(1,551) = 5.3, p <0.05).

Attitude

a) Vocational training year

Generally, trainees have a positive attitude toward the training year and think that being a trainee before embarking on unsupervised practice is extremely beneficial (median = 7), useful (median = 7) and comforting (median = 7). At the item level, all of the ten training year elements identified by stakeholders were regarded as both useful (median scores ≥4) and beneficial (median scores ≥5).

When compared to other UK trainees, trainees in Scotland have a significantly more positive overall attitude toward the elements of training (Table 1). The item analysis showed that this difference was driven by their attitude towards being informally and formally assessed. When compared to other UK trainees, trainees in Scotland think being informally assessed is more beneficial (Scotland mean = 6.1 (0.9); rest of UK mean = 5.9 (1.2); F(1,550) = 0.34, p = 0.56) and significantly more useful (Scotland mean = 6.0 (1.1); rest of UK mean = 5.6 (1.4); F(1,546) = 6.15, p <0.01) and significantly more comforting (Scotland mean = 5.2 (1.3); rest of UK mean = 4.7 (1.5); F(1,546) = 5.74, p <0.05). Also, as Figure 1 demonstrates, when compared to other UK trainees, trainees in Scotland think mandatory (formal) assessment is significantly more beneficial (Scotland mean = 5.6 (1.9); rest of UK mean = 4.0 (1.9); F(1,546) = 60.87, p <0.001), significantly more useful (Scotland mean = 5.3 (1.5); rest of UK mean = 4.9 (1.7); F(1,549) = 7.474, p <0.01) and significantly more comforting (Scotland mean = 4.1 (1.6); rest of UK mean = 2.7 (1.6); F(1,547) = 57.47, p <0.001).

Figure 1
figure 1

Comparing the attitude of dental graduate trainees toward formal (mandatory) assessment during the vocational training year

Full size image

The majority of all UK trainees (83%) perceived practices were well prepared for their arrival and that the clinical skills of their trainers were good to excellent (75%), as were their trainers' teaching skills (64%).

The item analysis revealed that, compared to other UK trainees, trainees in Scotland perceived their trainers had significantly better organised their first week in practice (Scotland mean = 4.4 (0.9); rest of UK mean = 4.1 (1.1); F(1,177) = 4.10, p <0.05). There was no difference in perceptions of trainers' clinical or teaching skills.

b) Continued professional development

In general all trainees had a positive attitude toward their continued professional development. Trainees think that keeping track of new technology, keeping track of new evidence, monitoring new evidence, and changing practice in line with new evidence would all be useful to them (median scores ≥6). They also intend to perform each of the elements (median scores ≥6), and they feel performing them would not be stressful (median scores <4).

In general, trainees perceive they were under pressure to implement new evidence in practice from the GDC (86%), their peers (85%), their trainers (82%), but not as much from patients (66%). Trainees also felt motivated to do what the GDC (95%), their peers (90%), their trainers (98%) think, but not so much what patients think (76%).

When compared to other UK trainees, trainees in Scotland did not differ significantly in their overall attitude towards continued professional development (Table 1). The item analysis revealed no significant difference in perceptions of usefulness in any of the four continued professional behaviours assessed. However, trainees in Scotland believed that they would find significantly more stressful having to keep track of new evidence (Scotland mean = 4.6 (1.2); rest of UK mean = 4.0 (1.4); F(1,546) = 11.88, p <0.001), and having to monitor how new evidence matches what they do (Scotland mean = 4.3 (1.2); rest of UK mean = 3.8 (1.4); F(1,546) = 8.04, p <0.01). Nevertheless, there was no significant difference in how stressful they thought it would be to change their clinical practice in line with new evidence (Scotland mean = 4.1 (1.4); rest of UK mean = 3.8 (1.5); F(1,544) = 1.88, p = 0.17).

c) Professional identity

All trainees think being a caring, competent or ethical dentist is important, with all items scoring medians of 7 out of a possible 7. Furthermore, trainees tended to think it will be easy to be a caring, competent or ethical dentist (median scores ≥5). There was no difference in perceptions of importance or difficulty in being a caring, competent or an ethical dentist between trainees in Scotland and trainees in the rest of the UK.

The majority of all trainees also agreed that providing treatment that is best for the patient, no matter what the patient thinks, means they will think of themselves as someone who is doing what they need to do (74% agreement), a caring dentist (64%), a competent dentist (76%) and an ethical dentist (64%). Trainees also agreed that they would not be someone who is doing what they need to do (83%), a caring dentist (86%), a competent dentist (83%) or an ethical dentist (79%), if they consider a patient's ability to pay when making a treatment decision. There were no significant differences in these perceptions between trainees in Scotland and trainees in the rest of the UK.

d) Work in general

Trainees had a positive attitude toward their work in general, with a median score of 4 out of a possible 5 for this measure. However, the item analysis revealed that trainees did feel afraid of making mistakes, afraid of litigation and that patients could be too demanding (median scores ≤2).

When compared to other UK trainees, trainees in Scotland had a significantly more positive attitude to their work in general (Table 1). The item analysis showed that they felt under significantly less pressure at work, greater satisfaction with the variety in their job, less afraid of litigation, less likely to be regularly working beyond their capabilities, under less pressure to treat more patients and less worried about their future in dentistry (Table 3).

Impact of the training year

Immediately after graduation (baseline), all trainees were confident in their clinical and non clinical skills, as well as their continued professional development (medians = 5.0, 4.6, and 5.0 respectively, out of a maximum possible score of 7) and held positive attitudes toward continued professional development and their training year (medians = 4.9 and 5.5, out of a maximum possible score of 7) and toward their work in general (median = 3.7 out of a maximum possible score of 5). The medians for the end of training year scores ranged from 5.2 to 5.6. All of the measures showed significant increases from baseline scores (paired samples t- tests, p <0.01). There were no group (Scotland/other UK) by cohort interaction effects (GLM analyses).

Discussion

The results of this study revealed that there were significant differences between trainees in Scotland and other UK trainees. Trainees in Scotland are more confident in their clinical skills, at least as confident in non-clinical skills, feel better prepared for their chosen profession, and show a more positive attitude toward the training year, their continued professional development and their work. These results were robust over three cohorts.

It is recognised that this was a pragmatic experiment, not a randomised controlled trial. It is possible that many things might influence the differences we found in trainee confidence and attitudes. Differences in undergraduate training teaching quality, teaching time, even geography itself, may be mediating influences. However, the outcomes assessed in this study covered a broad range of areas identified from a review of dental and psychological literatures, as well as consultation with multidisciplinary teams and stakeholders from all areas of the UK. It is not unreasonable to attribute the observed differences in outcomes across three cohorts and three years to the consistently different assessment formats that they experienced. The results strongly suggest that mandatory assessment is beneficial and that the rest of the UK trainees may benefit from the introduction of mandatory assessment. However, before taking this conclusion as read, more evidence is needed. In future, opportunities to collect behavioural data as well as data from trainers will be explored.

Nevertheless, the results of this study are informative. One of the aims of continued professional development is to encourage dentists to be reflective about their work and professional practice. These results suggest that mandatory assessment makes trainees more willing and more accepting of the benefits of being assessed – a desirable trait given there will be changes in quality standards in care delivery throughout their professional careers. Mandatory assessment during the vocational training year may therefore help them meet the ongoing challenge of implementing evidence in practice.

The results relating to professional identity raised an issue relating to patient-centred actions. The majority of trainees in this study agreed that their opinion about treatment was more important than a patient's – and that they would feel caring, competent and ethical if they treated patients in a way not desired by the patient. The idea that patients should have a choice in what happens to them doesn't appear to have an influence in this thinking.

In general, at the very least, the results of this study suggest that UK dental vocational training is preparing its participants in accordance with stakeholder-identified aims and outcomes. Within and across cohorts, all trainees in the UK felt confident and prepared at the end of their training year. All trainees had a positive attitude toward their training, continued professional development and their work. It is not certain how the significant but small differences in confidence and attitude translate into clinical practice. Exploring methods of collecting behavioural data in future studies may help us understand this. Nevertheless, since mandatory assessment appears to have beneficial effects and occurs only in Scotland - it does appear to matter where dental graduates spend their training year in the UK. This study is an important step forward in understanding the possible impact of mandatory assessment, thereby informing the development of professional education programmes.