Introduction

To establish the correct level of orthodontic knowledge in dental students on qualification, adequate assessment and examination is necessary. Considerable changes have occurred in undergraduate orthodontic courses in UK dental schools in the last few years, following The first five years General Dental Council document,1 and university pressures.2 The increase in dental student numbers will put further pressure on UK undergraduate orthodontic courses, and may necessitate modification of methods of assessment and examination of dental students.

A detailed survey of undergraduate orthodontic teaching in UK dental schools has been carried out, with current aspects of orthodontic course content and recent changes to the courses reported in the BDJ in 2005.2 In this second part of the survey, orthodontic assessment and examination in UK dental schools is discussed. The survey compares the current internal assessments and BDS examinations, and assesses the changes which have taken place in the last three years, the reasons underlying these changes, and any further changes envisaged.

Method

Questionnaires were emailed to the undergraduate orthodontic course leaders in each of the UK dental schools. Twelve dental schools agreed to participate in this 2004 survey. Responses were checked for completeness, summarised and returned to each course leader to verify. These verified responses were then analysed. Course leaders were also requested to send copies of their course manuals to enable clarification of information as necessary.

Results

The results are presented under the exact headings of the questions included in the questionnaire, and are divided into two main sections of assessment, and BDS examination.

Number of students

Student numbers per year range from 45 (Belfast) to 155 (GKT), with numbers at most schools between 55 and 65 (Table 1).

Table 1 Talbe 1 Summary of survey results – internal assessment and examination (Yr = year, Q = question, PGpt = postgraduate patient, RA = removable appliance)

Assessment

Assessment – position in course

Assessment occurs in years 4 and 5 in five schools (Belfast, Dundee, GKT, Leeds, London), in years 3, 4 and 5 in three schools (Bristol, Glasgow, Newcastle), in years 3 and 4 in two schools (Cardiff, Manchester), in years 3 and 5 in one school (Birmingham), and mainly in years 3, 4 and 5 but with a minor contribution in years 1 and 2 in one school (Sheffield) (Table 2).

Table 2 Table 2 Assessment – position in course

Assessment – written examinations

Seven schools have a formal written exam either in year 3 (Sheffield), year 4 (Bristol, Cardiff, Manchester), or year 5 (Glasgow, GKT, Newcastle). Short answer papers are held in six schools either in year 3 (Cardiff), in year 4 (Dundee, GKT), or in year 5 (Birmingham, Glasgow, Newcastle). An MCQ is held in years 1 and 2 in one school (Sheffield).

In addition, two schools have essays either in year 3 (Manchester), or in year 3 and 4 (Sheffield), and one school has a project in year 4 (Manchester) (Table 2).

Assessment – OSCE/spotter

Six schools have an OSCE (objective structured clinical examination), either in year 3 (Bristol, Manchester, Newcastle), or in year 4 (Dundee, Glasgow, Leeds). The number of stations in orthodontics varies and may be only one station (Dundee). Three schools have a spotter exam, either in year 3 (Cardiff) or in year 5 (Belfast, Leeds) (Table 2).

Assessment – clinical

Eight schools hold a clinical assessment, either in year 3 (Cardiff), or in year 4 (Belfast, Bristol, Cardiff, Glasgow, Manchester), or in year 5 (Bristol, Glasgow, GKT, London, Newcastle), Three schools use an assessment of a treated case presentation, one uses a postgraduate patient only (Cardiff), one uses a postgraduate patient and a removable appliance (RA) patient (London), and one uses a postgraduate patient or a removable appliance patient (Sheffield). Two schools hold a clinical viva (Belfast, Newcastle), and in one school a viva is held only if candidates have failed an examination (Leeds) (Table 2).

Assessment – laboratory

Five schools use a laboratory assessment; this is mainly in year 3 (Birmingham, Bristol - 2 RAs, Glasgow, Newcastle), except for two schools where it is either earlier in year 2 (Sheffield) or an additional assessment later in year 5 (Glasgow) (Table 2).

Assessment – contribution to BDS

In seven schools the assessment contributed to the BDS mark, but the contribution was very variable, 20-30% in four schools (Bristol - 3rd BDS only, GKT - 5th BDS, London - 4th, 5th BDS, Manchester 4th BDS), 15% in one school (Birmingham), and less than 5% in two schools (Dundee 4th, 5th BDS, Sheffield - over all). Different methods were used to calculate the contribution to BDS making it extremely difficult to compare the contribution in each school (Table 1).

Assessment – need to pass to enter finals

Students need to pass internal assessments before they are allowed to enter for their final examinations in all schools except one (Glasgow), where students can still enter for the final examinations if they have compensated for the failure by carrying out extra work and attending extra clinics.

Assessment – formative

All schools except one (Bristol) use formative assessments during the clinical years. These formative assessments are considerably variable in the number and timing, and in the method and format. During clinical years, a grade may be given each clinical session in some schools (Dundee, Glasgow, Newcastle), or given every six months (Cardiff, London, Belfast - year 4 and once in years 3 and 5), or periodically in other schools. Time may be allocated in the student timetable for a reflective learning session (London). A logbook/portfolio is used for formative assessments and clinical requirements in two schools (GKT, Leeds). Formative assessments are similarly categorised into three components in some schools (Cardiff, Dundee, GKT) as: 1) knowledge or academic, 2) competence or clinical ability, and 3) professionalism or attendance and attitude. Formative assessments may be used to qualify academic grades (Dundee) or may be used only for student feedback (Glasgow, GKT).

Formative assessments are considered in the internal assessment in six schools (Birmingham, Dundee, Leeds, London, Newcastle, Sheffield) and only if the candidate is borderline in one school (Belfast). In four schools (Cardiff, Glasgow, GKT, Manchester) formative assessments are not considered in the internal assessment mark.

BDS examination

BDS examination – written examinations

All schools have a written BDS examination including one orthodontic essay question, except two schools that have two essay questions (Belfast and Bristol). In one school (London) questions are integrated with other subjects. The written BDS examination is held in year 5 in all schools except Manchester where it is in year 4, and Dundee where there is one question in both year 4 and year 5 examinations.

Six schools also have a short answer paper, in year 5 (Birmingham, Glasgow, Newcastle, Sheffield), in year 4 (London), or years 3 and 4 (Dundee) (Table 1).

BDS examination – OSCE

An OSCE is held in three schools (Birmingham, Dundee, London) (Table 1).

BDS examination – clinical examinations

All schools hold a clinical BDS examination (Glasgow, Manchester and Newcastle use patient records only). The number of students per year taking the clinical orthodontic/child dental health BDS examination (in year 5) varies from either all students (Belfast, Bristol, Glasgow, Manchester), one in three students (Cardiff, GKT, London), one in four students (Dundee), one in five students (Birmingham), maybe one in six students (Leeds) or one in seven students (Sheffield). An additional examination of a general dentistry clinical viva is held in one school (Manchester) (Table 1).

BDS examination – case presentations

A case presentation of an orthodontically treated patient is carried out by all students in two schools (Bristol, Newcastle), and is optionally an orthodontic case in one school (Cardiff) (Table 1).

Changes to the course

Assessment and examination – changes in the last three years

Assessment within the orthodontic course has been reduced in one school (London), while in two schools it has been increased, with the further inclusion of a year 5 diagnostic clinical viva (Belfast), and a year 3 OSCE (Bristol). Modifications to assessments have occurred in two schools (GKT, Leeds). In one school (GKT) students must pass a clinical assessment to enter finals. Log books/portfolios have been introduced in two schools (GKT, Leeds).

The orthodontic component of the BDS examination has been reduced in one school (GKT), with reduction in the number of orthodontic questions on the written paper from two to one. Modifications to the BDS examinations have occurred in three schools (Dundee, Glasgow, Newcastle). Changes to 3rd, 4th and 5th BDS examinations, and inclusion of a child patient in the clinical in 5th year BDS have been introduced in one school (Dundee), and the use of patient records for the clinical BDS introduced in two schools (Glasgow, Newcastle).

Assessment and examination – future changes

Future changes are an increase in assessment in four schools (Belfast, Bristol, Cardiff, Newcastle). Changes are a year 3 assessment (Belfast), an OSCE in all clinical years (Bristol), a Scot (structured clinical operative test) (Newcastle), and a more robust formative assessment (Cardiff). A change in the internal assessment from 0% to 10% contribution to BDS will take place in one school (Newcastle).

Future modifications are being introduced in six schools. These are a competency based clinical assessment (Cardiff), and introduction of electronic logbooks/portfolios (GKT, Leeds, and under trial in other schools). The BDS examinations are currently under review in three schools (Belfast, Glasgow, Manchester).

Discussion

There is a large variation in the internal assessments for orthodontics taking place in the UK dental schools. Student numbers on the courses currently range from 45 to 155, with such variation, clearly assessment methods that are suitable for use with 45 students may not be practical for use with 155 students. However, as with other aspects of the course, there were no common trends among the smaller schools, or among the larger schools. Factors other than student numbers alone will contribute to the suitability of a specific assessment.

The position of the assessment in the course depends on the years over which the course is held, with assessments taking place in years 3, 4 or 5. Only one school differs in having some input also into years 1 and 2 assessments.

There was considerable variation in the form of the assessment. Either a formal written examination paper or short answer paper were the most commonly used form of assessment, and were used in nine schools, with four schools using both. An MCQ was only used in one school, and then only early in the course. One factor may be the difficulty of testing in depth knowledge and understanding of orthodontics using the MCQ, making it more suitable for testing a broad range of basic knowledge needed in the early stages of the course.

Assessments of an essay or a project, requiring more scope for self-learning and research, were only used in two schools. These forms of assessment would require time within the student timetable and considerable staff time to mark them, neither of which may be currently available at many schools, with course hours and academic staff number reductions reported.2

An OSCE is used by six schools, and a spotter by three schools. However, the number of orthodontic stations contributing to the OSCE varies from school to school, and may be only one station. In recent years the OSCE has become a popular form of assessment in dentistry, testing a range of skills and knowledge. However, it is costly and time consuming, requiring a large number of qualified and consistent staff, all of which again may be difficult with large numbers of students. Restriction on timetable space and a limited number of staff available may, therefore, influence the form of assessment possible.

A clinical assessment is held in all schools except four, two of which use a viva instead. The use of a clinical assessment reflects the current emphasis on diagnostic teaching. A case presentation is only used as an assessment in three schools, and of these, two use a postgraduate patient treatment, one of which also uses a removable appliance case, and one uses either a postgraduate or removable appliance case. This may reflect the reduced patient treatment available to be carried out in many schools, due to increased numbers of students, the limited number of suitable patients available for simple treatment and the increase in observation of postgraduate patient treatment reported in many schools.2

Only five schools use a laboratory assessment. This tends to be relatively early on in the course, mainly in year 3. There is considerable variation in the content of the laboratory assessment in these five schools. The reduced laboratory assessment in many schools reflects the recent reduction in laboratory teaching in these courses, with only eight schools now having a laboratory component included in the course.2

The contribution of internal assessments to the BDS examination mark was extremely varied from 0% in five schools, to up to 40% in some schools. The complexities of the different marking schemes make comparison extremely difficult, with no apparent common percentage contribution. However, in all schools except one, students needed to pass their internal assessment to enter finals.

A formative continual assessment is carried out in all schools except one, but varied in the number and timing of these assessments. Formative assessments were only considered in the internal assessment in seven schools, but are considered an increasingly important student feedback mechanism in some schools.

Although the BDS written examination is common to all schools, the number of questions varied, and questions were integrated in one school. Therefore, the emphasis of the orthodontic content of the written examination may vary from school to school. Additional short answer papers are taken in six schools, and an OSCE is set in five of these six schools. The number and timing of BDS examinations is, therefore, not consistent with some schools having these additional examinations earlier in the course.

Although all schools hold a clinical BDS examination, only in five schools do all candidates see an orthodontic/child dental health patient. In other schools this varies from one in three students to one in seven students, meaning that some students will not be assessed clinically in orthodontics at BDS, and some students may not have had a formal orthodontic clinical assessment throughout the course. Increasing numbers of dental students increases the difficulty and total cost of holding a clinical examination, especially involving a child patient where problems of attendance and consistency of case are more common. There is a trend for schools to change to the use of patient records for the clinical exam rather than a live patient to overcome these problems. Only three schools use treated case presentations for BDS examinations. Again, this reflects the reduced numbers of cases treated on courses by undergraduate students.2

Considerable changes to assessment and examination have occurred in the last three years. Assessment and examination has been reduced in two schools, increased in two schools, modified in six schools, and is under review in three schools. Future changes show an increase and modification in assessment and examinations. More clinical assessment and competency, introduction of portfolios/logbooks for clinical work, and more OSCE and SCOT tests, show a trend for more emphasis on clinical assessment which reflects the changes in courses to more clinical diagnostic teaching. A change to more robust and formalising of formative assessments in the future in some schools emphasises a trend to consideration of the importance of more reflective learning and teaching, and will require more timetabled space in the course and more teaching staff.

Clearly the choice and form of assessment and examination, and quality of this assessment will depend on adequate course time allocated and staff time available. Changes to orthodontic assessment and examination have occurred in response to changes in the courses following The first five years General Dental Council document1 and university pressures. Changes in assessment may be influenced by practical matters of cost, size of school, and number of staff. Recent reductions reported by schools in course hours and number of teaching staff2 may influence the type of assessment possible, with increased numbers of students further increasing the pressure on orthodontic courses in the future.

Conclusions

Current orthodontic internal assessments in UK dental schools are inconsistent, with no common pattern across the courses. Inconsistencies also exist in the BDS examination with variation in the number and integration of questions in the written examination paper, a clinical examination which is not taken by all students at all schools, and various additional examinations of short answer paper, vivas, OSCE or case presentation taken in some schools.

Guidelines for assessment and examination are needed to ensure standards and consistency in the future.