Abstract

Introduction

Seventy to eighty percent of healthcare errors are attributed to a breakdown in non-technical skills.

Objective

To determine the effect of simulation training on the acquisition of non-technical skills by undergraduate dental students.

Design

A parallel group, double blind, randomised controlled trial was designed. Simple randomisation, sequence generation and allocation concealment were undertaken.

Setting

University of Liverpool School of Dentistry, UK, 2017.

Materials and method

Eligible dental students had completed a minor oral surgery course, however, they had not completed a surgical extraction. The intervention group attended for a simulated, standardised minor oral surgery procedure. The control group attended for a video of such a procedure. Neither group received instruction in non-technical skills. Twenty-four hours following attendance, participants were observed completing a surgical extraction in a patient. Participants' non-technical skills were assessed by calibrated observers using a valid, reliable tool for their subjective assessment (NOTSS).

Results

There was a statistical difference between the groups in relation to mean NOTSS score (t = 2.2149, p <0.0439) demonstrating a significant effect of the intervention on the acquisition of non-technical skills.

Conclusion

Simulation training may support the acquisition of non-technical skills by dental students. Educators have an opportunity to diminish the risk of healthcare errors by incorporating simulation training.

Introduction

It has been reported that approximately 70–80% of all healthcare errors can be attributed to a breakdown in non-technical skills.1,2 The term non-technical skills is widely used within the aviation industry to describe general cognitive and social skills which enable pilots to monitor situations, make decisions, adopt the role of leader and communicate their actions within a team to ensure high levels of safety.2 There has been increasing interest in the development of non-technical skills acquisition within healthcare education.1

Simulation training as an educational tool has expanded in recent years within the area of healthcare education as a result of a number of driving forces.3,4,5,6,7 These driving forces have been refined with the development of both undergraduate and postgraduate education, coupled with societal pressures to promote a more safety conscious culture. Within the profession, it is recognised that we have a duty to educate undergraduate students with live patients to nourish their clinical development. As professionals, we also have a duty to ensure the safety and well-being of our patients and as such patients should not be regarded solely as 'teaching tools' purely for students to develop their skills.

Within Blooms Taxonomy, simulation allows the learner to move from knowledge to application, analysis, and even synthesis.8 Simulation training enables learners to actively learn from a given experience, in a way that direct instruction, or information delivery alone, does not permit.9 Simulation is based in constructivism, specifically social constructivism, and is based on Piaget's theory that describes learners constructing their understanding of the world through their interactions with it. Medical literature supports the use of simulation training to improve operating performance, with multiple randomised control trials demonstrating significant improvements in operating ability or technical skills.10,11,12,13

The simulation training literature has focused primarily on its use to improve technical skills.13 However, many surgical errors are due to a failure in non-technical skills including communication, team work, decision making and situational awareness.14,15 Those studies which do investigate non-technical skills do so within the context of an operating theatre, with anaesthetised patients.10,11,12,13 No previously reported studies have investigated the use of simulation to improve non-technical skills within the field of dentistry with a conscious patient. Indeed, dental education has primarily focused on the development of knowledge, technical skills and clinical expertise, and has paid less attention to non-technical skills.

Objective

The primary objective of this study was to determine the effect of simulation training on the acquisition of non-technical skills by undergraduate dental students in oral surgery. Secondary objectives were to assess the appropriateness of the research protocol of this feasibility study to aid the design of a future, larger randomised controlled trial, and to determine the acceptability of the interventions for the participants.

Materials and methods

A tool for the subjective assessment of non-technical skills has been developed and is freely available from the Royal College of Surgeons of Edinburgh. The tool, non-technical skills for surgeons (NOTSS), has consistent internal structure and a high inter-rater agreement.16 It has been demonstrated to have robust construct validity using confirmatory factor analysis.17 NOTSS is an observational behavioural rating system based on a skill taxonomy. The four skill categories are further divided into 12 elements (Table 1). Each of these elements are scored from 1 (poor) through to 4 (good).

Table 1 Non-technical skills for surgeons (NOTSS)
Full size table

Methods

A protocol was developed for a parallel group, randomised controlled trial design. The study received approval from the University of Liverpool Committee on Research Ethics. In the absence of previous trials to determine effect size, the inclusion criteria were applied to determine the eligible population. Eligible participants were undergraduate dental students in their fourth year of training at the University of Liverpool School of Dentistry. There was a single exclusion criterion of previous minor oral surgery experience (either observed or personal). The eligible population (n = 40) were invited to participate via university email correspondence. Participants were reassured of the confidential nature of their participation, and that no record of their performance would be placed on their academic log. This reassurance aimed to foster a transformative learning environment where participants felt safe to make mistakes and freely learn from those experiences.18 Thirty students provided informed consent for participation. Participants were allocated codes to prevent identification of individuals during statistical analysis, and to enable blinded assessment of outcome data. As part of the consent process, participants agreed that they would not reveal their group allocation throughout the study period. Participants were blinded to the purposes of the study until completion of data analysis and subsequent dissemination of the results.

Simple randomisation and sequence generation was undertaken using randomisation software, by a third party not involved in the study. Allocation concealment was with sequentially numbered opaque envelopes. Thirty participants were randomly allocated one of two parallel groups, Group A (experimental group, simulated teaching) or Group B (control group, video teaching). Participants in group A attended individually for a standardised one-hour simulated minor oral surgery procedure with the principal investigator (a clinical lecturer in oral surgery), during which no instruction was given in relation to non-technical skills. Participants in group B attended individually for a one-hour video demonstration of surgical procedures to remove teeth.19 Sequence implementation took place for each participant at the time of their attendance for delivery of the intervention.

Participants returned to the minor oral surgery clinic 24 hours following delivery of the intervention and were individually observed carrying out a surgical extraction, in a live patient, in the normal manner. The participants' non-technical skills were assessed by the blinded, calibrated observers using the NOTSS tool. The reliability and validity of the 12 item NOTSS tool was established during its design. Two members of oral surgery staff were recruited as observers. Calibration took place during the observation of three students not involved in the study, to establish inter-rater agreement in use of the NOTSS assessment tool. Inter-rater agreement was measured, and weighted Kappa was 0.72, demonstrating acceptable agreement.

Coded data were entered by a third party into SPSS Statistics, (Version 24, 2016), for statistical analysis. The data were stored on a secure database. The analyses were designed to be largely descriptive, and to establish the feasibility of conducting a larger future study. Normally distributed data were analysed with independent t tests. Pearson chi-square was used to determine independence for categorical variables.

Further description of the interventions

Experimental group

The intervention, (one-to-one simulated minor oral surgery teaching), was delivered in the ordinary environment in which routine clinical activity takes place by a single clinical lecturer in oral surgery. Group A participants were introduced to the simulated session and assured of the protected environment in which it would take place. Simulation models were designed and constructed at the prosthetics laboratory of the University of Liverpool School of Dentistry (Fig. 1).

Figure 1
figure 1

Simulation models

Full size image

The models incorporate plastic teeth fixed within an artificial bone matrix, covered with a plastic film to represent the mucosa. The models are secured into a mannequin head and torso that is fixed to the dental chair with straps. Standard operating procedures were employed. Participants were asked to complete the routine WHO surgical checklist and comply with routine health and safety procedures. Standard operating equipment was used, however, due to the nature of the artificial bone, irrigation through the surgical handpiece was not required (Fig. 2).

Figure 2
figure 2

Simulation training with surgical handpiece

Full size image

Control group

The control material (individual attendance for video demonstrations of surgical procedures), is produced by the British Association of Oral Surgeons. The videos are of routine surgical procedures being completed by a consultant oral surgeon on 'live' patients and contain verbal explanations of the procedures that provide advice in relation to technical skills required for completion of surgical procedures, as delivered during simulation training. Group B participants attended individually to watch videos in isolation for one hour. The videos contain no didactic teaching and the only voices audible are the surgeon and assisting member of staff.

Results

The mean cumulative NOTSS score for the intervention group was 42.6 ± 3.52. The mean cumulative NOTSS score for the control group was 38.07 ± 6.49 (Table 2 NOTSS Scores per Group). A paired t test was completed to compare the means of intervention and control NOTSS scores. There was a statistical difference between the groups in relation to mean NOTSS score (t = 2.2149, p< 0.0439) demonstrating a significant effect of the experimental intervention on the acquisition of non-technical skills in oral surgery. An inference of means was calculated that demonstrated a sample size of 42 would be required to power a future randomised trial at 95%.

Table 2 NOTSS scores per group
Full size table

Discussion

In this study, simulation training had a positive effect on the acquisition of non-technical skills by undergraduate dental students in oral surgery. There are several possible theories to explain this result. If we consider the Dreyfus model of skill acquisition, then perhaps the simulated training has transitioned students from novice to advanced beginner.20 At this stage, they have a greater awareness of their surroundings, (non-technical skill: situational awareness). It could be postulated that the students are no longer simply focusing on the tooth to be removed but have a greater appreciation for a holistic approach to patient management. Group A were able to practice teamwork skills, whereas Group B were passive in watching a video.

The research protocol of this feasibility study has been appropriate to address the primary objective, and to aid the design of a future, larger randomised controlled trial, following determination of effect size. Both the control and the experimental interventions were acceptable to participants. 73% (n = 11) of participants in the control group, and 93% (n = 14) of participants in the experimental group provided written feedback that they would welcome further training with their allocated intervention. A quarter of the eligible population declined to participate, reportedly as a result of the training for both groups taking place during lunchtimes. Therefore, in the design of a future study, training will be delivered during scheduled teaching sessions.

It is not known whether participants had previous exposure to oral surgery, as patients, or observers of online tutorials widely available online. The NOTSS tool does not account for the cooperation of conscious patients for treatment, hence it is possible that the participants were not exposed to patients who experience similar levels of dental anxiety. However, routine triaging of patients to the student surgical extractions clinic is likely to have avoided the introduction of any patient selection bias. Furthermore, the researchers had no part in allocating the attendance of patients to the clinic in which students from either group were observed.

The experimental intervention was appropriate for the investigation of the primary objective. The Best Evidence Medical Education collaboration have previously published guidelines to determine the features of high fidelity simulators that lead to effective learning.21 The five key elements of a successful learning experience from simulation were described as feedback, practice, validity, fidelity, and simulator learning. These elements were incorporated into this study design. During simulation, students were provided with informal verbal feedback. Providing students with verbal feedback may be associated with improved learning.22 Students were provided with an hour of simulated teaching with the opportunity to practice multiple variations of minor oral surgery and provided with immediate feedback. This is an example of deliberate practice as described in the literature.23 The same outcomes were achieved throughout simulated sessions as are achieved in regular teaching sessions, that is, teeth were extracted on live patients. It is fair to consider, therefore, that the simulator had appropriate validity. In comparison to computerised simulators available, the mannequins utilised are of low fidelity given the lack of blood and patient reaction. However, the literature supports the use of low fidelity models in improving performance, and indeed demonstrates how high fidelity can impinge on learning.21,24 The models had high fidelity for the surgical elements involved in minor oral surgery. It is ideal that fidelity should match the outcome assessed.1 The acquisition of non-technical skills may aid students in becoming consciously incompetent surgeons. This is in line with the Conscious Competence Learning Model, as attributed to Noel Burch, which starts with the unconscious incompetent (novice) and progresses through four stages to unconsciously competent (expert).25 The models (identical for each participant) simulated surgical procedures of varying complexity. It is unlikely, therefore, that students would be able to 'learn the simulator'. Successful simulation training, which aims to incorporate the characteristics of a conceptual framework, includes retention of technical proficiency, during task-based learning within a professional context.24

Conclusion

A breakdown in non-technical skills may contribute to three quarters of all healthcare errors. This randomised trial has generated the first documented evidence that simulation supports the acquisition of non-technical skills by undergraduate dental students in oral surgery. Dental educators have an opportunity to diminish the risk of healthcare errors by developing educational theory, and by incorporating simulation training into practice. Revalidation of the NOTSS tool for dentistry will be addressed in a future study. This study will be repeated with 95% power, to support the effect of simulation training on the acquisition of non-technical skills as described, in comparison to a true control group. A pathway for the delivery of simulation training in a variety of dental settings will be explored.