Background

In the United Kingdom, institutions delivering a programme of study in dental surgery (BDS or BChD) are quality assured by the General Dental Council (GDC). The GDC document 'Preparing for practice'1 describes the outcomes which must be attained before registration. The domains of attainment in 'Preparing for practice' are: clinical, communication, professionalism, management and leadership. The GDC requires those graduating with a dental degree to have demonstrated that they have achieved the level of the 'safe beginner'. They define this as 'a rounded professional who, in addition to being a competent clinician and /or technician, will have the range of professional skills required to begin working as part of a dental team and be well prepared for independent practice.'1 The definition goes on to promote the importance of self-awareness and states that the safe beginner 'will be able to assess their own capabilities and limitations, act within these boundaries and will know when to request support and advice.'1 While this definition is helpful, it may be criticised in that it lacks both precision and detail, which impacts upon standard setting of thresholds.

The GDC require that assessments contributing to the professional examination of dental students 'should be rigorous, appropriate and reliable as a gateway for students to become qualified to practise independently.'1 Therefore, in the quality assuring of existing assessments and development of new assessments, educational providers must ensure that they are authentic, a reflection of current best practice, and that they accurately access the knowledge, skills and attributes required of new dental graduates.

Many academics involved in the development and delivery of an undergraduate dental degree course within dental schools are qualified as specialists in their own field, be that clinical, academic or both. As their clinical practice increasingly centres on their specialist areas, experience suggests that their focus when developing teaching material and assessments may also be biased in a manner reflecting this specialist level. Arguably, therefore, there may be benefit from not using academics as the sole determiners of the 'safe beginner' standard. While it is important to have this 'expert' level as high achieving or aspirational, it is crucial to balance this, both in terms of delivery of teaching and assessment. This will ensure the outcome requirements are that of a 'safe beginner', and not skewed towards that of a specialist. To this end, input into curriculum content, teaching and assessment design (content and standard setting), would greatly benefit from the input of those with a working and day-to-day awareness of both the standard and concept of the 'safe beginner'.

The vast majority of UK dental graduates embark on dental foundation training (DF), where they work with experienced primary care dental practitioners who are trainers within the DF programme. Involving primary care practitioners, with current or recent experience of DF training, together with academics in the development and delivery of both the undergraduate curriculum and its assessment, has the potential to ensure a more balanced, authentic and appropriate assessment at the standard of the 'safe beginner'.

A dependence on sessional clinical teachers from a primary care background is a recognisable feature across most institutions delivering medical and dental undergraduate education.2 These members of staff provide a variable number of sessions each week where their primary role is to supervise, support and aid in the learning and teaching of students in the clinical environment. When not in the dental school environment the majority of these practitioners work independently in primary care practice settings, these are often training practices where new or recent graduates are based. This experience with both undergraduate students and recent graduates makes this group of teachers, who have a wealth of clinical knowledge and experience in dental schools and primary dental care, eminently suited to contribute to the design, validation and standard setting processes of assessment. However, the nature of their employment contract may, at first glance, appear to prohibit their contribution to 'non-clinical' activities without resulting in the cancellation of clinical contact time. This would be both unpopular with students and undesirable for programme directors. In addition, many are employed on a 'term-time' basis, so they will not be present when the students have annual holidays.

This paper describes our experience of actively facilitating the involvement of sessional clinical teachers in the processes of assessment.

Identifying opportunities

In the academic year 2011–2012 the clinical discipline within the School of Dental Sciences with the largest number of sessional clinical teachers (restorative dentistry), implemented a process of active timetable management. Approximately 50% of all undergraduate clinical teaching in this discipline was delivered by 31 sessional teachers. One member of staff (HB) took responsibility for identifying under-utilised sessions within the undergraduate clinical timetable and coordinating alternative activities of clinical teachers.

Reviewing the timetable revealed opportunities when scheduled student clinics were legitimately cancelled or reduced in size for alternative educational activities. This situation potentially left clinical teachers without their usual teaching commitment. These instances included: tutorial afternoons (our sessional teachers do not have roles as personal tutors); and when students had timetabled written or practical examinations. Sessions were identified throughout the academic year, with the majority occurring in term 3, which is traditionally associated with the heaviest time commitment to examinations. Review of term 3 (2011–12) identified 119 sessions (16%) when sessional clinical teachers were not needed to deliver clinical teaching, due to student involvement in other legitimate learning and teaching activities. This is a consistent pattern and has amounted to between 71-119 sessions of sessional clinical teacher time in that term each year (Table 1). In financial terms, this staff resource has a cost, which based on current remuneration rates for sessional clinical teachers,3 is approximately £9,000 for this term each year.

Table 1 A summary of the sessions identified over a three year period with specific focus on the third term each academic year
Full size table

A proportion of these sessions were scheduled, as previously, to provide clinical teachers with opportunity to undertake mandatory tasks such as annual NHS Trust training, while others enabled full-time academic staff to be freed up from clinical teaching in order to undertake other aspects of their roles. Even taking this into consideration during the academic year 2011–12, 27 notional half-day sessions were identified as further opportunities for sessional teachers to contribute to the strategic development of assessment.

Opportunities for assessment development

Multiple stages in the development of assessments were identified that would potentially benefit from the input of sessional clinical teachers in terms of assessment preparation, through to standard setting, and in examining in summative assessments (Fig. 1).

Figure 1
figure 1

A representation of our stages in assessment development

Full size image

Having identified a need (additional input in assessment development), an opportunity (under-utilised sessions) and a resource (experienced sessional clinical teachers), we needed to consider implementation. Consideration included:

  • Identifying the need for staff availability; ensuring enough sustainable time to make investment in skills development worthwhile

  • Undertaking targeted training for sessional teachers to help them develop a greater understanding of assessment and utilisation of wider university resources in this process

  • Linking sessional teachers with academics within the school skilled in assessment development, thereby optimising the sharing of the skill sets.

Writing

Opportunities were identified where a number of sessional clinical teachers were available and initially dedicated to delivering workshops in question development. These were delivered by academic staff from the University Learning and Teaching Development Unit or facilitated by an experienced academic member of staff from within the school. Pairs or groups of sessional clinical teachers were then timetabled to work together to develop new assessment material targeted at specific knowledge based learning outcomes identified through the school's assessment blueprinting process. Multiple question formats (single best answer, extended matching item etc.) were constructed. On average each assessment development session produced two fully worked up questions.

Validation

The sessional clinical teachers also worked alongside academic staff to form groups reviewing previously prepared assessment material to determine the content validity.

Standard setting

Finally, groups were also formed to standard set previously developed elements of assessments using the Ebel method.4 Sessional clinical teachers contributed to both parts of the Ebel process; initially establishing which aspects of knowledge assessment are considered as an 'essential' contribution to the skill set of the safe beginner, and then secondly the determination of degree of difficulty of the assessment. This contribution was seen as being hugely valuable, as a means of tempering the opinion of experts.

Utilisation (examination)

In addition to assessment development, sessional teachers received training for acting as examiners during summative assessments. This involved initial observation of the examination process, reflection of the candidate and examiner performance and alignment to a more experienced examiner therefore preparing them to be examiners in future diets.

We have increased sessional teacher involvement across multiple stages of the programme in a range of clinical examinations, including end of year examinations for Stage 2, 3 and 4 BDS students and the Stage 5 (Final) BDS examination. Examples of examinations in which sessional clinical teachers participated, include in-course assessments of clinical competencies, OSCEs, case based discussions and examination of written reports of observational experiences of clinical interactions.

Table 2 shows how sessions for sessional clinical teachers were alternatively scheduled.

Table 2 To show how sessions were alternately scheduled.
Full size table

Discussion

Harden and Crosby5 describe one of the roles of a teacher as a 'resource developer', and in undertaking this initiative we have sought to extend this description to our sessional clinical teachers and their role within our BDS programme. Our drivers were two-fold:

  • enhancing quality in areas of academic assessment development

  • maximising the efficiency of utilisation of skilled clinical teachers.

The development of authentic, valid and reliable assessment material is challenging and resource intensive, but nonetheless a vital and ever present need if programmes are to consistently demonstrate continuing attainment of educational standards and learning outcomes. Moreover, secure, progressive and well-stocked question banks are essential to remain contemporary and avoid the need to frequently recycle previously used questions. This latter approach runs a risk of future cohorts of students gaining insight into potential questions, and taking a more strategic approach to learning and assessment.

There is little in the way of evidence detailing the role of sessional clinical teachers, beyond that of their supervisory role. However, the perceived potential advantages of the involvement of sessional teachers, who are primarily primary care practitioners, in the delivery of undergraduate programmes have been expressed. These focus on the situational primary care experience and knowledge these practitioners have.6,7 We believe, from the personal experiences of the authors that this can be extrapolated and that by including sessional teachers, we have been able to bring an authentic and practical application of clinical knowledge to enhance context appreciation, synthesis and application of theory to the questions they have developed.

This engagement has allowed us to draw out intrinsic skills and create assessments that have a greater breadth of context. At a similar time to increasing sessional teacher involvement, we recognised a need to increase the diversity and scope of the questions in our assessment bank. In addition to the pure clinical application, we needed to develop questions that addressed wider areas including: professionalism, leadership, management and communication. It was hoped that assessments would be developed that require students to draw information from different core courses, and apply that knowledge through synthesis within a valid context. There are many intrinsic skills that are part of a core course (for example, prosthodontics or radiology) which are not often explicitly identified as such, including skill components of those mentioned above. In addition we have identified new topics for assessment (and out-of-date material that needs to be dropped), increasing the diversity and applicability of the assessments in our question banks.

Establishing appropriate thresholds for assessment has been a continuing challenge for educators.8 Over the past decad e, the School of Dental Sciences at Newcastle University has been applying standard setting processes to better support the thresholds applied to the BDS and other learning programmes in the school. In the UK, a greater focus on standard setting for dental educators arrived with the regulatory body (GDC) embedding the process within their standards for education.9

The process of standard setting aims to estimate and quantify a performance level, or cut score, for grading categories of an assessment.8 The terminology applied by the UK regulator on a key threshold is the 'safe beginner' as an 'absolute standard', and determines the passing student.

Two regularly used methods, Angoff10 and Ebel,4 may, we believe, be actively enhanced by the involvement of primary care practitioners in conjunction with academic staff. McKinley and Norcini11 suggest a number of attributes for those who standard set. These can be summarised as:

  • awareness of the purpose of the test and the domain assessed

  • the consequences of passing the examination

  • familiarity with the content

  • familiarity with the level of those sitting the test.

We believe this description would accurately reflect our sessional clinical teachers. When determining 'safe beginner' assessment thresholds we have perceived that a further benefit of including sessional teachers with a base in primary care is their ability to effectively moderate the opinions of specialists to align to this key threshold. There remains an opportunity to further develop the skill set of this group to embrace post-hoc analysis of question performance which will in turn potentially lead to further improvement in question utility (specifically reliability).

The time necessary to develop assessment raw-material is valuable, necessary, and difficult to achieve solely by the full-time academic team who have competing demands on their time. As an organisation we became aware of an under-utilised staffing resource within the programme. We therefore feel this revised session management has proved mutually beneficial to the school and individual teachers; by extension we hope that our students will also benefit. Active session management has resulted in a fuller and more productive use of publicly funded teaching resources. In the current climate, the stewardship of resource is crucial, and reflects the core principles of the NHS in terms of ensuring 'best value for money' and 'most effective, fair and sustainable use of finite resources.'12

Delivery of training for sessional teachers has been a recognised challenge in other institutions.13 We felt that access to, and methods for, training were very much associated with the part-time nature of this staff group. Smaller group training was required that may have not been necessitated with full-time staff. This highlighted that training may be more accessible by utilising other delivery methods, that is, e-learning. Other than timing and method of delivery there was very little difference in the content of the training.

Annual appraisal discussions with our sessional teachers have highlighted their appreciation of, and interest in, continued involvement with all stages of the assessment process. Interestingly a number have commented that they believe their clinical teaching to have been enhanced. They anecdotally describe an increased awareness of both the curriculum (from learning outcomes) and the process of assessment that students are required to undertake. To this end, and in a similar way to acknowledging that assessment drives learning for students,14,15 we would suggest that involvement in the assessment process appears to support the enhancement of clinical teaching.

Identifying that time is available in an existing resource is important. To fully maximise the potential of this opportunity there is a need to invest in training. This is a principle which should be adopted across all staff in student assessment to improve consistency16 and we believe this supported development is worth investing in. This process of active session management has now been running for three academic years. In 2012–13 and 2013–14 fewer sessions were available for assessment development, this is almost certainly due to two large cohorts of students working through the programme. Nonetheless, our initial investment in training allowed our programme of new assessment development to continue, when perhaps in previous years little or no development would have been undertaken because of the increased pressure on teaching resource.

This work has led us to recognise the need for well designed, qualitative investigation into this area of clinical educational research.

Conclusion

We consider that it is realistic and beneficial to include sessional clinical teachers in developing assessment. Furthermore, this can be achieved without detriment to their primary role of clinical supervision/teaching. There is a requirement for investment in careful manpower planning and training to capture the enhancements that this group of clinical teachers can bring to programmes such as ours.