Introduction

Cataract surgery is one of the most commonly performed surgical procedures worldwide [1]. The most common intraoperative complication is posterior capsule rupture (PCR), with rates reported between 1.92 and 4.1% [1, 2]. Good visual outcomes can still be achieved in as many as 90% of PCR cases, but this is dependent on effective complication management [1, 3]. Opportunities for trainee surgeons to develop the skills required to handle such events are limited, given that the supervising surgeon will usually take control when PCR is recognised [4]. Having few opportunities to manage PCR inevitably means that the trainee may fall short of the standard required for independent practice by the time they finish training [4].

The Halstedian model of apprenticeship training has traditionally produced highly proficient ophthalmic surgeons, however, the challenges of modern surgery, reduced training hours, senior-led practice, increased public expectations and technological advances necessitate new and innovative training methods [5]. Surgical simulation is now being utilised across all surgical specialties to address this [4]. Simulation provides an opportunity to practice surgical skills in safe, controlled, and reproducible environments without compromising patient safety [6]. However, surgical simulation has predominantly focussed on the development of a surgeon’s technical skills (TS) whilst omitting the inclusion of non-technical skills (NTS) [7]. NTS include leadership, teamwork, communication, situational awareness, decision making and stress management; all of which are fundamental for the calm and effective handling of adverse events [8]. Contrary to traditional views that NTS were innate and therefore untrainable, studies have repeatedly shown that they can be taught, learned, assessed and refined through didactic teaching, e-learning and simulation [9]. Studies have also demonstrated strong correlations between a surgeon’s NTS and TS abilities, however, there is a paucity of assessment tools focussed on NTS [10,11,12]. Furthermore, there remains little integration of NTS with training models and simulation in ophthalmic surgery [7].

We developed an innovative, high-fidelity and fully immersive simulation-based training model, designed for training, rehearsing and assessing the NTS required for intraoperative PCR management. The objectives were to outline the devised simulation-based training model, analyse participant performance using validated measures to assess changes in participant’s NTS and TS, and present questionnaire data relating to validity and educational impact.

Materials and methods

This simulation study was performed in the operating theatres of Sussex Eye Hospital, University Hospitals Sussex (UHSussex) NHS Trust, United Kingdom. The Medical Research Council Health Research Authority (MRC HRA) decision analysis tool determined that ethical approval was not required. Consent forms approved by the UHSussex information governance team were used by faculty and study participants.

Design of simulation scenario

The first draft of the simulation scenario was devised by a focus group, which consisted of one Academic Foundation Trainee Doctor (internee) (TCW), one Ophthalmology Registrar (specialty trainee year 6), one Consultant ophthalmic surgeon with specialist interest in cataract and oculoplastic surgery (SR), and one Consultant ophthalmic surgeon with a specialist interest in cataract and anterior segment surgery (MAN). All members of the focus group had specialist academic interest in human factors and surgical NTS training. The NTS required for managing intraoperative complications in ophthalmic surgery have been outlined previously; the HUman Factors in intraoperative Ophthalmic Emergencies Scoring System (HUFOES) is a content-validated scoring system for NTS using PCR as a focus [13]. Using guidelines and recommendations set by previous studies in combination with the skills taxonomy outlined by HUFOES, the first draft of the simulation scenario was devised [13]. The scenario focussed on an elective cataract surgery case, during which an intraoperative PCR occurred. The remainder of the scenario was intertwined with extensive NTS stressors designed to challenge the surgeon’s handling of the case. The scenario underwent iterative revisions through which extensive amendments were made. This ensured that the scenario would be realistic with the addition of NTS stressors. Iterative suggestions were made to address the feasibility, deliverability and content validity of the scenario.

One consultant ophthalmic surgeon and one ophthalmic surgery clinical fellow then participated in a recorded pilot of the simulated scenario. The videos produced were analysed by focus group members. Further refinements were made from pilot simulation feedback in order to improve the scenario’s realism and timings.

Materials and faculty requirements

A comprehensive list of materials required for the simulation was established (Supplementary Box 1). This included materials for the participant to use during the scenario, and the means by which the simulation could be recorded. For each simulation with each surgeon, video recordings were captured live on ZEISS Callisto system attached to ZEISS Lumera operating microscope (Carl Zeiss, Jena, Germany) and smotsTM camera system (Scotia UK PLC, UK) for profile views.

The minimum faculty requirements to allow the simulation scenario to run effectively were one scrub nurse, one wandering nurse and one simulation coordinator. A camera operator was necessary due to the use of high complexity camera equipment. The simulation coordinator or camera operator provided the patient’s ‘voice’. All faculty were required to know the intended complication events, the NTS stressors to apply, and responses to potential outcomes.

The ‘patient’ was created on the operating table using an OSILA anatomical head and tactile synthetic handmade surgical simulation eyes (Phillips Studio, Bristol, UK). A single simulation eye was used for each simulation.

Initial simulation

Ophthalmic surgeons of all grades from four hospitals in the Kent, Surrey and Sussex region (Sussex Eye Hospital, East Kent Hospital, Eastbourne District General Hospital and Maidstone and Tunbridge Wells Hospital) were invited by email to participate in this training model. Timeslots were allocated for specific days. Each participant underwent exactly the same scenario without knowledge of the events set to unfold. All participants had consented for their simulation attempts to be recorded by high-definition cameras placed strategically within the operating theatre.

Each participant received a brief outside the operating theatre before their first simulation attempt. This included information on the patient’s demographics and allergies, the intended surgery, time of day and the support available. They were informed about the simulation eye and its behaviour during phacoemulsification (lens nucleus freely mobile in the capsule bag, air bubbles in anterior chamber during phacoemulsification, posterior chamber positive pressure present with and without filling the posterior chamber with egg white, anterior capsule not staining with VisionBlue® (DORC, Netherlands) and the reuse of Alcon AcrySof MA60 (Alcon Laboratories, Fort Worth, Texas) IOLs between cases). They were made aware that the instructor may interrupt them in the middle of the surgery and instruct them on an evolving complication, after which they repeat the instruction to the instructor, before re-entering simulation mode to manage the complication.

They then went through to the fully equipped and staffed operating theatre, where the ‘patient’ was draped and ready. The surgeon was required to scrub and begin the procedure. WHO checks from the ‘patient’ notes were provided [14]. The scenario then continued as intended, with faculty guiding the events agreed by the focus group. Following completion of the simulation, each participant underwent a debrief with the facilitator regarding their performance. Furthermore, the video recording of each participant was emailed to them for personal reflection and learning. This routine applied to all participants.

Training intervention

After completion of their first simulation attempt, a date and time was established to deliver an interactive training intervention to all participants. This consisted of three presentations delivered remotely via Zoom (San Jose, California, USA), due to the social distancing requirements of COVID-19.

The first presentation discussed the TS considerations of managing PCR, with reference to controlling equipment, equipment settings and the timing of interventions. Surgical videos were included as examples.

The second presentation addressed the NTS considerations for managing PCR, with reference to forward planning, reducing multi-task interference, and integrating NTS with TS. Furthermore, the role of simulation in surgery was outlined, notably how it can be used for threat and error management, improving communication, and handling intraoperative complications. The human factor and NTS components were developed in association with a commercial aviation pilot, with experience in simulation and NTS training.

Following this training intervention, the presentations were distributed to all participants for self-directed learning. Educational components addressed during the training intervention are further outlined in Supplementary Box 2.

Repeat simulation

Times were allocated for all participants to undergo a repeat simulation within 3 months of the initial simulation. The format and events were unchanged; hence all participants followed the same brief and scenario as that outlined previously.

Video analysis

Video recordings of each surgeon’s initial and repeat simulation attempts were coded by an independent faculty member. Coded videos were distributed to two independent and blinded assessors in random order at the end of the programme. Both assessors were Consultant ophthalmic surgeons with specialist interest in cataract surgery, NTS and surgical education. To prevent observer bias, neither assessor knew which videos pertained to initial or repeat simulation attempts.

Outcome measures

To demonstrate the extent of change in participant’s TS and NTS abilities as a result of the simulation based training model, both assessors independently rated participant performance in each recording using HUFOES for NTS, and the Objective Structured Assessment of Technical Skill (OSATS) global rating scale for TS [13, 15].

Following their first simulation attempt, each participant was required to complete a questionnaire relating to demographics and prior NTS experience. Following the training intervention and the repeat simulation, each participant was required to complete a second questionnaire relating to the content validity, feasibility and perceived educational impact of the model overall. Additional comments could be provided in free text boxes. The current educational impact of the simulation model was analysed using an adaptation of McGaghie’s model of translational outcomes for simulation-based learning [7, 16].

Data collection and analysis

HUFOES and OSATS scores were inputted into Microsoft Excel 2019 (Microsoft®, Redmond, Washington, USA), and analysed using Microsoft Excel 2019 and IBM® SPSS® Statistics for Windows, Version 27 (IBM Corp, Armonk, New York, USA) [13, 15]. Mean HUFOES and OSATS scores generated by both assessors were calculated. Paired t-tests were used to establish the difference in mean HUFOES and OSATS scores before and after the training intervention, with p < 0.05 considered statistically significant.

Questionnaire data were collected on the online platform QualtricsXM (Qualtrics, Provo and Seattle, USA). Microsoft Excel 2019 was used for data storage and analysis. Data were presented in terms of participant numbers and percentages.

Results

The theatre setup for this simulation model is shown in Supplementary Fig. 1. The finalised simulation scenario is outlined in Supplementary Box 3 and demonstrated in Fig. 1. A shortened sample video of the simulation exercise is shown in Video 1. Demographic information relating to grade, previous encounters with intraoperative complications and prior NTS training are outlined in Supplementary Table 1.

Fig. 1: Simulation based PCR management exercise.
figure 1

Four separate camera angles used to comprehensively capture the participant’s TS and NTS.

Full size image

Seventeen ophthalmic surgeons participated in the training model. All seventeen surgeons were included in the video analysis in order to generate HUFOES and OSATS scores before and after the training intervention. However, four of these participants were removed from questionnaire data analysis due to incomplete responses.

Following participation in the training intervention, a statistically significant increase in participant’s NTS was achieved. Mean HUFOES scores increased from 48.7 ± 16.6 to 59.2 ± 14.8 pre- and post-training intervention (p < 0.001). TS improvements were also demonstrated pre and post-training intervention; however, these failed to achieve statistical significance. Mean OSATS scores increased from 16.0 ± 7.3 to 17.9 ± 8.3 (p = 0.07). This is displayed in Fig. 2.

Fig. 2: HUFOES and OSATS scores pre and post training.
figure 2

a HUFOES scores pre and post training. b OSATS scores pre and post training.

Full size image

Questionnaire response data are presented in Supplementary Table 2 and Supplementary Table 3. 100% of responding surgeons (n = 13) were in agreement that further training is required in order to prepare ophthalmic surgery trainees for managing intraoperative complications. All responding surgeons were in agreement regarding the importance of intraoperative NTS; 53.8% (n = 7) felt them to be extremely important, 23.1% (n = 3) felt them to be very important, whilst 23.1% (n = 3) perceived them to be moderately important.

The simulated scenario undertaken before the teaching intervention was considered very stressful by 23.1% (n = 3), moderately stressful by 53.8% (n = 7), vaguely stressful by 7.7% (n = 1), and not at all stressful by 15.4% (n = 2). Those who found the scenario not at all stressful were Consultants with >5000 completed cataract surgeries and >20 PCRs encountered, yet none had received any previous NTS training. Participating in the scenario itself was considered to be educationally beneficial by surgeons overall, with 69.2% (n = 9) and 23.1% (n = 3) agreeing strongly or somewhat respectively. Furthermore, exclusive participation in the scenario was considered to improve NTS even before the training intervention; 15.4% (n = 2) were in strong agreement that their NTS had improved, 53.8% (n = 7) somewhat agreed, whilst 30.8% (n = 4) remained neutral. Mixed responses were received when participants were asked how able they felt to manage PCR following the simulated scenario if it were to happen during live surgery. 15.4% (n = 2) stated they felt extremely able, whilst 38.5% (n = 5) stated they were very able, 46.2% (n = 6) felt moderately able. Those who felt extremely able were experienced Consultants with >5000 cataract surgeries performed and >20 previously managed PCRs.

The teaching intervention was considered extremely or very enjoyable by 69.2% of surgeons (n = 9), whilst 23.1% (n = 3) found it moderately enjoyable. After participation in both rounds of simulation, 84.6% (n = 11) were in agreement that the simulations were immersive and realistic. Despite the fact that the second scenario was identical to the first, the second simulation was rated moderately to extremely enjoyable by 92.3% (n = 12), compared with 76.9% (n = 10) for the first. This is similar to how challenging each simulation was rated; the second simulation was considered extremely or very challenging by only 7.7% (n = 1) compared to 38.5% (n = 5) for the first.

At the end of the simulation training programme, 53.8% of responding surgeons (n = 7) strongly agreed that their NTS for managing PCR had been improved, whilst 38.5% (n = 5) somewhat agreed. Furthermore, these skills were considered transferrable, with 76.9% of responding surgeons (n = 10) in agreement that the model had enhanced the NTS required for managing general intraoperative complications. 92.3% (n = 12) stated that the model had been educationally beneficial for them. 84.6% (n = 11) were in agreement that comparable simulation training programmes should be more widely available in order to improve the NTS required for managing intraoperative complications.

Free text responses giving positive feedback remarked on the quality of the focussed post-simulation debrief, and the opportunity to participate in a unique training model delivered at an appropriate time after the suspension of elective cataract surgery. The most difficult aspects of the simulation were reported as technical difficulties with the Phillips Studio Eye, difficulty selecting the correct vitrectomy tubing, the approach of the theatre staff (unprofessional, inexperienced or out of character), difficulties communicating with a talkative and nervous patient whilst managing a complication, and immediate lack of knowledge regarding vitreous loss management. Suggestions for improvements included addressing technical problems with the eyes, and more detailed feedback and debrief opportunities. Feedback was accumulated and will be used to improve future iterations of the simulation model.

The simulation-based training model has achieved both Level 1 (internal acceptability) and Level 2 (contained effects) according to McGaghie’s model of translational outcomes for simulation-based learning [7, 16]. This demonstrates participant satisfaction with the simulation model overall, and improved performance in the simulated setting.

Discussion

We report the development of a novel simulation-based training model designed to enhance the NTS required for managing intraoperative PCR. Participating in the initial and repeat simulations either side of the training intervention demonstrated a statistically significant increase in participants’ NTS; mean HUFOES scores increased from 48.7 ± 16.6 to 59.2 ± 14.8 (p < 0.001). Whilst the increase in mean OSATS scores from 16.0 ± 7.3 to 17.9 ± 8.3 (p = 0.07) reflected an improvement in participant’s TS, these were not statistically significant. Questionnaire feedback also highlighted the importance of NTS within ophthalmic surgery and suggested that more should be done to prepare ophthalmic surgery trainees for managing intraoperative complications. The simulated scenarios were considered to be stressful by the majority of surgeons, however on completion of the model, 92.3% (n = 12) agreed that their NTS required for managing general intraoperative complications and PCR specifically had improved.

NTS are fundamental for PCR management; the full taxonomy of NTS necessary for PCR management has been outlined previously [13]. The model developed in the present study consisted of two identical simulations designed to challenge the surgeon’s NTS during an unfolding complication. The simulations were separated by an interactive training session where the TS and NTS requirements of PCR management were discussed. Whilst TS are paramount for safe and effective surgeons, NTS are now recognised as core surgical competencies [9]. NTS are critical for successful intraoperative complication management, however implementing them during pressurised situations is invariably challenging [8]. Applying evidence-based NTS stressors throughout these simulations was therefore an important objective, particularly as surgeons are often reluctant to admit the effects of stress on their performance [17, 18]. This may explain the effects seen in our questionnaire data, where the scenario undertaken prior to the training intervention was reported as moderately, vaguely or not at all stressful by 76.9% of surgeons (n = 10). Two highly experienced surgeons did not report experiencing stress effects; this is understandable given that experience is a recognised moderating factor when handling stressful situations [18]. Interruptions, distractions, time pressures, equipment problems, on-call responsibilities and high procedural complexities are all recognised intraoperative stressors [9, 18]. Whilst the moderate application of stress can be beneficial, deleterious effects on TS and NTS occur when one’s coping ability is exceeded [19]. Variants of these factors were therefore included. Interestingly, two recurring themes from the free-text responses cited the difficult approach of the theatre staff (who were encouraged to appear inexperienced and unfamiliar with complex equipment), and the difficulties surrounding a conscious and anxious patient. These stressors were selected to address the fact that complications can happen when working alongside inexperienced and unfamiliar team members, which requires the surgeon to take on a leadership role whilst communicating standards and requirements effectively [10]. Effective communication is also required for the surgeon to convey clear messages to the supporting team, in order to initiate changes to the course of the surgery and request the preparation of any specialist equipment that may be needed [13, 20]. Given that cataract surgery is predominantly performed under local anaesthetic, careful verbal communication is required at all times [20]. This is especially important during evolving complications.

The results from this simulation model demonstrate significant improvements in participants’ NTS as a result of the training intervention, which focussed predominantly on the NTS required to manage PCR. Participant TS were also found to improve, however the improvement was not statistically significant. As intended, this highlights that the simulation model and training intervention have greater specificity within the NTS domain. Surgical specialties such as urology have repeatedly proven that NTS can successfully be incorporated with simulated scenarios, and simulation has been shown to be a viable mechanism through which they can be trained and improved [9, 21]. More recently, the incorporation of NTS and human factors within ophthalmic surgery simulation has been identified as a feasible method of training, and has an expanding evidence base [6, 7, 10, 20]. Studies have shown that trainees who undergo virtual reality (VR) simulation training prior to performing live cataract surgery have lower PCR rates than their non-simulator trained counterparts (2.2% and 4.8% respectively, p = 0.032) [22]. This indicates a clear utility for simulation-based training in complication avoidance. 45% of ophthalmic surgery simulations focus exclusively on the technical aspects of cataract surgery, with numerous studies utilising the extensively validated VR Eyesi Surgical (VRmagic, Manheim, Germany) [6, 7]. However, there remains a comparative paucity of simulations with a NTS focus in ophthalmic surgery, with only two previous studies including these competencies [7]. Saleh et al evaluated simulations for ophthalmic surgical teams, presenting them with different scenarios based on genuine patient safety incidents, the majority of which were perioperative (such as marking up the incorrect eye) [10]. They concluded that human factor-based simulations in ophthalmic surgery are a feasible method of training [10]. Furthermore, Henderson et al reported on the Virtual Mentor; a computer-based simulation designed to train the cognitive aspects of hydrodissection [23]. Participants in the Virtual Mentor group received significantly better post-test scores compared to those who received standard teaching [23]. These studies support the importance of training human factors and NTS in ophthalmic surgery simulations, however the simulation-based training model outlined in our study is the first of its kind to exist in this format. Surgeons participating in our training model perceived the simulation model to be educationally beneficial, reporting noticeable improvements in their NTS. This is consistent with studies reporting NTS improvements following simulation-based training in urology [21]. However, aside from those who were already experienced, our model was not considered to prepare trainees for independent PCR management if they were to imminently encounter one in reality. This reflects findings by a previous study, which stated that only 9.1% of trainee ophthalmic surgeons would feel confident managing PCR without senior support [4]. It must therefore be said that surgical simulations are not substitutes for reality; rather they form realistic opportunities to practice skills, rectify mistakes, and prepare for complications in safe and reproducible environments [18]. In comparison, the aviation industry uses high fidelity simulations to train (and retrain) core pilot competencies, of which NTS contribute significantly [24,25,26]. Pilots are required to routinely practice prevention and management strategies for emergency situations, in order to optimise actions and behaviours when these events occur in reality, such as engine failure situations [26, 27]. Further work must be undertaken to implement NTS focussed simulation training in ophthalmic surgery, in order to achieve safety standards comparable to aviation. It is therefore encouraging that the simulations within the present study were considered enjoyable, immersive and realistic, as studies have shown that skill development within simulation is impacted by its overall realism [28]. This constitutes a progressive step towards providing formal NTS training in ophthalmic surgery.

This study did have limitations. The primary limitation includes the financial and time expense of using operating theatres, staff, equipment and set-up. Occasionally, the Phillips Studio Eyes needed replacing during a participant’s ongoing simulation attempt. This was due to the development of severe posterior chamber positive pressure secondary to the model eye’s dynamics, which are slightly different with each eye as they are all handmade. Furthermore, there were occasional recording failures, however crucial footage was not compromised at any stage. Whilst the focus of this study was NTS improvement, the training intervention included TS teaching. Some participants may have gravitated to this whilst providing questionnaire feedback, however this is not detrimental to our findings as previous studies have demonstrated strong correlations between TS and NTS overall [11]. Furthermore, our data support that NTS improved significantly, whereas the TS improvement was insignificant. Due to incomplete responses, four participants were removed from the questionnaire responses, meaning there were fewer responses than participants overall. This is not detrimental as questionnaire data is a secondary outcome. Finally, HUFOES is a content-validated scoring system for NTS in PCR and cataract surgery [13]. No directly comparable system exists to measure the TS within this domain specifically, and therefore the OSATS global rating scale was used to score participant TS generically [15].

The strengths of the NTS simulation-based training programme were extensive. These included its realism in a fully immersive operating theatre with genuine surgical equipment. It applied evidence-based stressors, facilitated realistic communication with a conscious ‘patient’, and provided an interactive training session on the NTS required for PCR management. Whilst the current training programme has achieved Level 1 (internal acceptability) and Level 2 (contained effects), McGaghie’s model of translational outcomes must be utilised further to establish the programme’s downstream, target and collateral effects [7, 16].

This study has developed a novel, fully immersive and realistic simulation-based training model for managing intraoperative PCR with a NTS focus, which demonstrates internal acceptability and contained effects. Significant improvements in participant NTS have been objectively evidenced, and participants themselves considered the model to be educationally beneficial.

Summary

What was known before

  • Opportunities for trainee ophthalmic surgeons to develop the skills required to manage intraoperative complications are limited.

  • Surgical simulation has traditionally focussed on the development of a surgeon’s technical skills at the expense of non-technical skills.

  • There are strong correlations between non-technical and technical skills, however there is little integration of non-technical skills with training models and simulation in ophthalmic surgery.

What this study adds

  • This study developed a fully immersive and realistic simulation-based training model for managing intraoperative PCR with a NTS focus.

  • Statistically significant improvements in NTS were objectively demonstrated as a result of the simulation-based training model

  • The simulation-based training model was considered educationally beneficial by participants overall.