Introduction

Both the concept and requirements of mandatory continuing education have been debated extensively for many years within the dental profession. The National Voluntary Vocational Training Scheme was introduced in 1988, presaging the introduction of mandatory arrangements. Not until 2002 was comprehensive mandatory continuing professional development (CPD) in dentistry firmly established in the UK.

The ultimate goals of recertification and re-accreditation systems must always be the maintenance and improvement of standards of services in which patients can have confidence.

The impact of credit hours of traditional courses on the quality of practice is, however, disputable. It has been reported that traditional continuing medical education (CME) for doctors (equivalent to CPD) may have impeded development of more effective ways of promoting continued learning.1 In one study, the number of reported CME hours was found to be inversely correlated with competence.2 Therefore, the content, format, quality and outcome of postgraduate courses need to be continuously assessed if more effective evidence-based CME programmes are to be developed and implemented. The ultimate goals of recertification and re-accreditation systems must always be the maintenance and improvement of standards of services in which patients can have confidence.

Dental radiographic examinations represent one of the most frequently undertaken radiological investigations in the UK. A survey for the period 1997-19983 estimated that dentists were taking 19 million intra-oral radiographs each year and more than 2.9 million panoramic radiographs. The effective dose delivered per radiograph is very small, but the collective dose is significant4 and has been estimated to account for approximately 10 fatal cancers per year.5

It is a statutory requirement that all dentists and other health professionals who use X-rays should be adequately trained in accordance with the Schedule 2 and Regulation 11 of the Ionising Radiation Medical Exposure Regulations. In addition, knowledge in this field should be regularly reviewed and updated.4

Postgraduate dental deaneries within the UK have provided radiation protection courses for dentists for the last 25 years. In Wales, an average of 14% of all dental practitioners have annually attended radiation protection courses per year since 1999.

The name 'POPUMET' course is given to those courses which were delivered whilst the old regulations were in force (Protection of Persons undergoing Medical Examination or Treatment Regulations 19886 and the Ionising Regulations 1995). The name 'IRMER' course is given to those courses which cover the current sets of regulations (Ionising Radiation - Medical Exposure - Regulations 2000 and the Ionising Regulations 1999).7

The objectives of this study were as follows:

  1. 1

    To evaluate the effectiveness of a dental postgraduate one-day course in radiation protection (IRMER) run eight times in Wales in 2003-2004

  2. 2

    To identify the participants' areas of strength and weakness

  3. 3

    To compare the results with a similar study8 conducted five years earlier, which assessed a POPUMET course.

Subjects and methods

Two hundred and eighty-five general dental practitioners took part in this standardised CPD course, which was run eight times in six centres in Wales in 2003-2004 and always delivered by the same specialist teachers. Participants completed anonymously a validated multiple choice question (MCQ) test instrument before the teaching session and an identical MCQ at the end. The test instrument was a modified version of one prepared by Smith9 in 1991. It comprises 80 true/false items and is scored with negative marking, so the range of possible scores is from −80 to +80. Scores of 50 out of 80 (+62.5%) or above were regarded as satisfactory.

All analyses are based on paired morning-afternoon records available for 253 (89%) of participants. Year of graduation of each participant was also recorded. The 80 questions were categorised into seven subject areas, to enable assessment of areas of strength and weakness:

  1. 1

    Principles of radiation physics

  2. 2

    Risks of ionising radiation: biological effects

  3. 3

    Radiation doses in dental radiography

  4. 4

    Factors affecting doses: equipment and exposure factors

  5. 5

    Radiation protection: practical measures

  6. 6

    Statutory requirements

  7. 7

    Quality assurance.

Our previous study8 reported on six cohorts of dental practitioners attending the POPUMET ionising radiation regulation course, which was the predecessor to the course evaluated here. The analysis was based on 295 out of 304 practitioners who completed paired morning-afternoon records. The two sets of results were compared.

Statistical analysis

Scores were expressed as percentages of the maximum mark obtainable. Pearson correlations were used to assess the relationships between pre- and post-treatment scores, and of these and changes in score with the number of years since graduation. Heterogeneity of improvement between centres was assessed by one-way analysis of variance. A 95% confidence interval (CI) for the change in the proportion passing from pre- to post-training was calculated by method 10 as described by Newcombe.10 A 95% CI for the difference in the changes between IRMER and POPUMET courses was also calculated.10,11

Results

The mean (SD) pre- and post-course percentage scores were 33.8 (13.5) and 63.6 (14.6). The mean improvement was 29.8 percentage points (SD 11.7). At baseline only six (2.4%) of the 253 practitioners achieved the preset standard. After the course, 146 (57.7%) did so, an improvement of 55.3% (95% CI 48.9-61.3%).

Figure 1 plots post- versus pre-training percentage score, with a diagonal line of equality superimposed. As expected, the pre-post correlation was high (+0.66).

Figure 1: Scatter diagram showing the relationship between pre- and post-course mean scores (%).
figure 1

r = +0.66

Full size image

Figure 2 shows pre- and post-training mean scores for each of the 6 centres. There is slight evidence of heterogeneity of the improvement in score between these centres (F = 2.2, p = 0.06).

Figure 2
figure 2

Mean percentage scores in the six centres

Full size image

Table 1 and Figure 3 summarise the improvements in percentage score for the seven subscales. Changes from pre- to post-training are shown both as absolute changes and also as percentages of the shortfall from 100%, thus an improvement from 20% to 40% would appear in the final column as a change of 25%. Paired t-tests show that the improvement is highly significant (p <0.001) for each subscale. Even before training most participants were strong in practical radiation protection measures and quality assurance, but weak in understanding of exposure factors and their effects, effective doses and current regulations. The effect of training was greatest for biological risks, practical protection measures and least for radiation doses.

Table 1 Change in percentage score from pre- to post-training in 253 GDPs attending the IRMER course, by section of syllabus
Full size table
Figure 3
figure 3

Pre- and post-course mean score % for various groups of questions

Full size image

Participants' years of graduation ranged from 1962 to 2002, with mean and median 1984 (SD 10 years). Correlations of baseline score, final score and score gain with year of graduation were +0.02 (p = 0.71), +0.14 (p = 0.03) and +0.14 (p = 0.02) respectively. Thus the final score and score gain were slightly higher in more recent graduates.

Comparison of mean scores and pass rates between IRMER and POPUMET courses are shown in Tables 2 and 3. Pre-training, the dentists about to undertake the IRMER course scored slightly lower than those about to undertake the POPUMET course five years earlier. The post-training mean score was 3.4 percentage points higher after IRMER than after POPUMET (p = 0.007). When we allow for the initial score being lower, the IRMER course resulted in a 4.6 percentage points greater gain in score than POPUMET (95% CI +2.5 to +6.7, p <0.001).

Table 2 Comparison between IRMER and POPUMET results. Analysis based on mean percentage marks
Full size table
Table 3 Comparison between IRMER and POPUMET results. Analysis based on proportions scoring 50 or more out of 80
Full size table

The proportion of dentists passing the POPUMET course increased from 6.1% pre-training to 49.5% post-training, a gain of 43.4% (95% CI 37.5% to 49.0%). The increase in proportion passing for IRMER, 55.3%, was 11.9% higher than this (95% CI +3.4% to +20.3%).

Discussion

Continuing medical and dental education involves expenditure of great amounts of time and money on the part of both course providers and attendees, in the belief or hope that this will guarantee professional standards. However, there are few data on the objective benefits derived from such efforts.12 Furthermore, there is an inherent difficulty in using objective criteria to measure courses. Although completing multiple choice questionnaires is a poor measure of performance, more rigorous forms of assessment, such as practice visits and peer review, have proved too costly to introduce widely.13

We were interested in performance across a wide range of topics to cover both the subject and the requirements as laid down in the current guidelines. In that sense, a well-designed MCQ test should generate valid scores in that it has face validity as a measure of comprehension and knowledge application of the learner, and is objective and reproducible. The questions used in this study have been tested and validated.8,9 A quick, non-threatening test in the form of a quiz proved popular amongst practitioners.

Some educationists recommend the use of 'one-best answer items' as being less ambiguous than true/false questions, such as we used.14

In this study, levels of knowledge in radiation protection amongst dental practitioners were very low when measured at baseline. Some participants are self-selected and chose to do this course because they were aware that this was an area of weakness; others did the course in order to comply with the current recommendations. Only six (2.4%) of 253 participants achieved the threshold score of 62.5%, with an overall average of 33.8%. This compared to 146 (57.7%) participants with total average scores of 63.5% after the course, an increase of nearly 30%. While this is a substantial improvement, several issues ought to be considered and discussed.

The threshold level of 62.5% was chosen by consensus, in line with Smith9 who recommended that a practitioner with an adequate grasp of the 'core of knowledge' should be able to achieve at least this score - a high standard that perhaps many educators would like to achieve in their courses.

The results also showed that more recently qualified practitioners performed slightly better than their more experienced peers. One explanation may be greater familiarity with multiple choice questionnaires and recent learning experience.

It is important that our methods of assessment of competence and performance are evidence-based and promote self-directed learning.

One main advantage of such tests is that they enable identification of learning deficiencies in relation to patient care and thus planning for future needs. Competence (ie what we are capable of doing), is often confused with performance (ie what we do in everyday practice). Furthermore, professional accountability is crucial as public expectations and demands are growing and patients expect their clinicians to meet set standards. It is, therefore, important that our methods of assessment of competence and performance are evidence-based and promote self-directed learning.15 This study clearly demonstrated areas of deficient knowledge both prior to and following the course. This will help in future course planning and adoption of appropriate educational strategies, if the resources allow.

Course content is largely mandatory and inevitably a proportion of the subjects covered may appear dry and uninspiring to some. However, the course content should not be what is popular but what the profession needs. The Guidance notes for dental practitioners on the safe use of X-ray equipment4 published by the Department of Health in the U.K. (2001) outlined the essential elements from both legal and educational points of view. Course content was based on these training and statutory requirements.

The assessment of outcome in this study was performed immediately after the course. However, it is not clear how long this level of knowledge will be retained and how much self motivation and awareness have been evoked. Undoubtedly, refresher courses to reinforce learning are necessary, if a long-term benefit is to be achieved.

The study raises a question about the appropriateness of the traditional approach to continuing education. A one-day course, whether didactic or even with the attendees' participation, may not be adequate or ideal in delivering the IRMER requirements. Major literature reviews, relating to the effectiveness of education strategies designed to change physician performance and health care outcomes, reached similar conclusions.16,17 The three main aims of IRMER courses are to increase knowledge and understanding, raise awareness and change practice. Only the first two may be evaluated using this type of assessment; the third is more difficult to assess by this method.

The improvement in outcome comparing IRMER with earlier POPUMET courses is reassuring, indicating that we are going in the right direction. Nevertheless, in interpreting these results, it should be borne in mind that this study was not a randomised trial comparing two methods of training. In any case, much more remains to be achieved. Further study is planned to assess the long term effectiveness of this course.

Conclusion

The level of knowledge in radiation protection was very low at baseline. Attending an approved course improved this considerably but did not invariably achieve a satisfactory level of knowledge in fundamental radiation physics and regulations in all attendees, fewer than 60% of whom achieved the prescribed standard. Dentists who qualified in recent years obtained greater benefit from courses than those who qualified in earlier years. The increase in proportion passing was nearly 12% higher (p <0.001) for the IRMER course than for POPUMET.