Main

Competence, the ability to do a task to a pre-determined standard, is being increasingly assessed for a range of professions and occupations. In medicine the need for rigorous and objective assessments is heightened by the move towards greater accountability and revalidation. Surgical Dentistry is a new specialty and relatively little work has yet been done on the assessment of the competence of surgical dentists. This paper aims to discuss some of the issues that will need to be considered as assessment of surgical dentists' skills is introduced.

Definitions

In the past the terms competence and competency have been used synonymously, but a distinction has emerged. Competence is fitness for the purpose or the ability to do a task to a predetermined standard. Competencies describe the series of abilities that together make up a competent person.1

Competency standards are often thought of as a series of discrete task descriptions. There are however broader aspects to a competent performance e.g. planning and reacting to contingencies.1,2 The General Dental Council (1997) in its booklet on The First Five Years3 breaks down competence in to a series of desirable attributes:

  • knowledge of appropriate sorts

  • skills

  • attitudes

Separate assessments of these attributes do not show an individual's ability to integrate these skills or test their clinical judgement.

How do we assess competence?

1) Knowledge

This is of course already widely tested by a range of methods e.g. essays, short answers, multiple choice questions, viva voces either in examinations or in a continuous format linked to a planned review of progress i.e. appraisal. The pros and cons of the different methods are well established and have been extensively discussed.

2) Skills

a) Clinical skills and abilities. Traditionally these have been tested by presenting candidates with a patient or patients, asking them to take a history, conduct an examination, make a diagnosis and present their findings to an examiner. This approach has proven limitations in content, reliability and feasibility.4 In the last two decades considerable effort has gone into improving these forms of assessment, including written and computer-based patient management problems and standardised patient-based examinations. A significant development has been the introduction of Objective Structured Clinical Examinations (OSCEs).5 Various clinical skills are broken down into their constituent parts and candidates rotate around various bench stations completing set tasks at each one.

b) Practical or technical skills and abilities. All dental surgeons are expected to have a "good pair of hands" and to possess exceptional technical skills. Until recently such skills have tended to be assessed in a subjective judgmental style by the trainer. Interest in the objective assessment of these skills is growing rapidly.

3) Attitude

Less has been written in this area than the other two – possibly because it is the most problematic to define and assess. Attitudes are usually described as being in the 'affective domain' and are based on complex sets of values and beliefs. These are acquired throughout life and are based on a wide range of influences.6 The behaviour of surgeons must be compatible with their expected role.1 Attitude concerns the application of knowledge and skills. It could be considered to be synonymous with judgement but judgement in the narrow sense (forming a correct opinion) is only one of a number of factors to be considered. Attitude can be broken down into a number of discrete areas including the following:

  • clinical judgement

  • interaction with patients and relatives

  • ethics

  • reliability

  • professional development

  • teamwork

  • image or appearance.

Assessing competence in surgical dentistry

Surgical Dentistry is a specialty of dentistry, which deals with the diagnosis and surgical management of anomalies and pathological processes of the teeth and their supporting structures.

As we start to train surgical dentists it is essential that we also develop sound assessments and standards in as many of the competencies of the specialty as possible. Mossey et al.7 discuss the importance of integrating knowledge, skills and attitude into competence in dental students. Teaching and assessing operative skills at undergraduate level is uniquely a dental problem and methods need to be devised which will:

a)increase self-awareness by encouraging self-evaluation and learning by reflecting on experiences,

b)encourage achievement of competent core skills.

c)identify and help individuals who are not achieving or progressing satisfactorily at an early stage.

These methods need to form the basis of assessment during specialisation in surgical dentistry.

Assessment should record achievement of competence in as objective a manner as possible and encourage continuous self-evaluation. The students/trainees reflect on their clinical performance and in consultation with their supervisors they record their plans to improve their ability in that skill or procedure. Such supervisor–validated self assessment learning experiences are designed to progressively improve clinical performance and are described as formative. In contrast summative (regulatory) assessments are more formal evaluations6 e.g. qualifying examinations. In high stakes assessment, where results are important both to patient and examinee, it is important that there are high levels of agreement between examiners i.e. there is good inter-rater reliability.

Brown et al.8 and Davenport et al. 9 describe the use of OSCEs. They7 list the 18 stations they have developed to test second year clinical dental undergraduate students. They also describe in greater detail a dental pain station and its marking system. It is used as part of the student's summative in-course assessment. Davenport et al.9 found the main causes of unreliable assessment in OSCEs are patients who present inconsistently and examiners who do not adhere to agreed marking scales. OSCEs have yet to be developed for the surgical dentistry trainee but are likely to be appropriate in their assessment.

Theaker, Kay and Gill10 have tested inter-observer reliability when evaluating dental students' communication skills. The principles behind the assessment of undergraduate knowledge and skills will remain as these assessments are modified and developed for the trainee in surgical dentistry. As Harden5 recommends, a balance is sought between clinical, communication and diagnostic skills with clinical skills representing about 40% of the assessed content.

Testing practical and technical skills

The testing of practical and technical skills is a particularly complex subject, very relevant to surgical dentistry and meriting more detailed consideration. Under this heading we may consider a number of techniques.

1 Bench Models.

The reduction of technical evaluation into specific components could allow for the concrete assessment of surgical performance. Reznick and others11,12,13,14,15,16 adapted the OSCE to an OSATS (objective structured assessment of technical skills). They used bench models of surgical tissues to assess performance. Elements of simulated operations are selected for evaluation rather than the whole procedure. In this way the test environment can be standardised. (The skills were assessed using a checklist approach – ideal for providing specific feedback and a global assessment in a rating form. The latter focuses on surgical behaviour rather than specific manoevures and therefore looks at a different aspect of performance. (Fig. 1)

Figure 1
figure 1

Global rating scale of operative performance

Full size image

2 Virtual reality.

Virtual reality and associated technology offer enormous potential in early surgical training and in assessment of perceptual – motor competency skills17. However the highly refined models needed for complex surgical evaluation are still a long way off and effective assessment is required now.

3 Assessment in the workplace.

A successfully performed task comprises 75% decision making and 25% technical dexterity.18 It is for this reason that many argue that assessment of technical competence should take place (at least in part) in the work place.1,2 In this way judgement, attitude and the ability to cope with a contingency may be brought into the assessment. This is however more difficult and time consuming to apply. It is harder to standardise the environment and achieve good inter-rater reliability in the test.

One means of assessing technical skills in the workplace is by recording a log of operative procedures performed. This is however a record of the number and range of procedures done rather than the quality of surgery. If used with an assessment of actual performance logbooks can be used as a more objective measurement of competence in the workplace. When combined with self-assessment, logbooks are valuable training tools which can be applied by an individual for their own benefit.

Removal of third molars may be assessed using global rating scales, as described by Martin et al.13 and Dath and Reznick16 (Fig 1). Alternatively Figure 2 gives an example of checklist criteria that can be used to assess removal of impacted lower third molars. These may also be of value for assessment driven training.

Figure 2
figure 2

Objective assessment: removal of lower wisdom teeth

Full size image

Outcome audits may also be used to evaluate competence but are harder to use in the context of qualifying/accreditation examinations. They are more applicable when revalidation of established clinicians is required.

Peer Review can also be used to assess the competence of established clinicians in the workplace. Used when concerns regarding fitness to practise are raised, it is a very sensitive, complex, time consuming and costly exercise. However informal review between colleagues can lead to development of skills and be of benefit to both operator and reviewer.

Attitude

De Monchy et al. felt that the professional attitude of doctors was concerned not only with feelings, beliefs and behaviour towards patients but also towards other elements such as health care delivery, scientific interest and collaboration with other health professionals. They developed attitude scales which measure patient-centeredness (PCN) against doctor-centeredness.19 Batenburg et al found that attitudes were related to specialty. GP trainees showed more PCN than surgery trainees.20 They have yet to be tested on different specialties within dentistry.

In the past much store has been placed on image or appearance, but this must surely become secondary to more important attributes such as good judgement, interaction with patients and ethical issues.

Future Developments

Teaching systems in undergraduate, vocational and postgraduate specialties must provide curricula with core competencies and a means of achieving objective competency based assessment.7 Methods do need to be developed for assessing skills and attitudes in surgical dentistry, the former probably being much easier than the latter.

As already mentioned, in order to measure judgement, attitude and ability to cope with a contingency assessments should take place in the workplace.1,2 However the difficult question of exactly how these attributes will be examined and results standardised has yet to be fully addressed. The root of the problem of measurement is that both judgement and attitude are by their nature subjective and therefore difficult if not impossible to measure objectively. One can alter and develop attitudes, so it is important to encourage trainees to take control of their own learning process and to assist with continuation of learning and self-evaluation throughout life.

Conclusion

Assessment of clinical competence is both a valuable tool in its own right and a means of demonstrating to the public the continuing commitment of the profession to the highest possible standards. It will have relevance both to the teaching of undergraduates and trainees and to the revalidation of those who have already qualified. As we have shown, considerable progress has been made towards developing robust models for assessment and we can, in surgical dentistry, build on the work carried out in other medical and non-medical fields to develop specific batteries of tests.

Assessment, as with so many recent developments, will however be time-consuming for both trainer and trainees. There is a danger that over-emphasis on assessment may reduce exposure to procedures. We must, therefore, ensure that assessment delivers, in return, clear and unambiguous benefits to clinicians and that it solves real problems. We have said that the key areas to measure are judgement and attitude and that these are the most difficult to measure accurately. One approach to this challenge is to focus on assessment in the workplace. This is a key area to be addressed in future studies.