Introduction

Cataract surgery is the most commonly performed surgical procedure in the Western World [1]. The most common intraoperative complication is posterior capsule rupture (PCR); with reported rates ranging from 1.92 to 4.1% [1, 2]. The UK Cataract National Database of Cataract Surgery found that the ‘junior surgeon’ is the most significant risk factor for PCR occurring (adjusted odds ratio—3.73) [2, 3]. The visual outcomes for patients with PCR can still be favourable post-operatively in 90% of cases [1, 4]. However, favourable outcomes are dependent on effective complication management [4]. Junior surgeons have limited opportunities for managing cataract surgery complications, which makes these outcomes challenging to achieve [5]. A recent survey of trainee ophthalmologists who had performed the >350 cataract surgeries (the number required by the Royal College of Ophthalmologists to be considered competent for Consultancy) revealed that only 2/11 felt able to deal with PCR and vitreous loss without senior support [5]. It has been suggested that simple numerical markers of competence or experience should be replaced with targeted training approaches, as has been adopted by the Accreditation Council for Graduate Medical Education (ACGME, American College of Surgeons), with specific emphasis on intraoperative complication management [5, 6].

Whilst technical surgical skills are paramount for the management of intraoperative complications, the application of non-technical skills (NTS) (leadership, teamwork, communication, situational awareness (SA), decision making and stress management) is also fundamental. The consequences of NTS failures are increasingly widely reported; for instance, one study revealed that communication failures alone account for 43% of intraoperative errors [7]. Their application in surgical emergencies is even more critical in order to recapture a safe and successful outcome [8]. Paradoxically, the ability to employ them effectively in crisis management is invariably challenging due to time pressure, rapidly evolving situations, diagnostic uncertainty and human instinct [8]. NTS are now a fundamental component of competency frameworks of the Royal Australasian College of Surgeons, Royal College of Physicians and Surgeons of Canada, ACGME, and the Royal College of Surgeons (UK), but remain limited in ophthalmology despite the acknowledgement of a pressing need in 2011 [6, 8,9,10,11]. A conservative measure of 3.4% of all patient safety incidents (15,311/446,184) were attributed to ophthalmology on the Reporting and Learning System of the National Patient Safety Agency [12]. Failures in NTS are the predominant cause for wrong intraocular lens events, making ophthalmology the largest culprit for surgical errors across all surgical specialties [13, 14]. High patient turnover, use of complex equipment, detailed data analysis and necessity for fast decision making mean that NTS failures will continue to occur in ophthalmology with increasing magnitude and severity, unless training modalities and interventions are introduced [11, 15].

Non-technical skills for surgeons (NOTSS), observation teamwork assessment for surgery (OTAS) and NOTECHS are NTS assessment tools that are used by surgeons or the surgical team in generic surgical situations [11, 16,17,18,19,20]. Anaesthetist’s non-technical skills (ANTS) and Scrub practitioners’ list of intraoperative NTS have been developed for anaesthetists and scrub practitioners, respectively [21, 22]. In highly specialist surgical domains, these generic tools do not apply. Increasingly specialised scoring systems have therefore been developed and validated, including Interpersonal and Cognitive Assessment for Robotic Surgery in robotic surgery, Behavioural Marker System for assessing Neurosurgical Non-Technical Skills and Endovascular Observational Teamwork Assessment Tool in endovascular surgery [23,24,25].

The primary objective of the present study is to develop the HUman Factors in intraoperative Ophthalmic Emergencies Scoring System (HUFOES), focussing on the NTS required to manage intraoperative PCR. The secondary objective is to obtain data on HUFOES’ validation and educational impact, as determined by a specialist panel of Consultant cataract surgeons.

Methods

This study was undertaken at Sussex Eye Hospital (Brighton and Sussex University Hospitals NHS Trust, Brighton, East Sussex, UK). The Medical Research Council’s Health Research Authority decision analysis tool determined that the present study did not require ethical approval.

Development of HUFOES

The development of HUFOES was undertaken in two stages.

(1) Focus group and literature review

HUFOES was proposed following a series of discussions amongst a small focus group consisting of one Consultant ophthalmic surgeon with a specialist interest in cataract and anterior segment surgery (MAN), one Consultant ophthalmic surgeon with specialist interest in cataract and oculoplastic surgery (SR), one Ophthalmology Registrar (speciality trainee year 5) (SM), and one Academic Foundation Trainee Doctor (internee) (TCW). The focus group was designed to represent doctors of different grades and experience levels involved in cataract surgery. All members had an academic interest in simulation and NTS training. A literature review of surgical NTS and their associated scoring systems was undertaken using Pubmed and Embase. This elicited an extensive number of accessible attributes, and the broad domains to which these belong. These were discussed amongst this focus group for their relative application in PCR management and were used as a platform to generate a novel taxonomy of NTS specific to this setting. Furthermore, the formats of each scoring system were discussed in accordance with their layout, perceived ease of use and user friendliness in order to generate an appropriate HUFOES format. Consensus was achieved amongst the focus group members for HUFOES’ first draft when no further improvements, additions or removals were recommended.

(2) Development and validation questionnaires

A development questionnaire was distributed to the Consultant ophthalmic surgeons working as trainee supervisors in the Kent, Surrey and Sussex Deanery for trainees (South-East England training programme, UK). Inclusion criteria required each respondent to be a Consultant ophthalmic surgeon in the Kent, Surrey and Sussex Deanery, a minimum of 3 years of experience in training junior surgeons and were a recognised clinical and surgical training supervisor of trainees. No limits were applied for prior experience or familiarity with NTS assessment tools. Those meeting exclusion criteria were those who were not in the Consultant ophthalmic surgeon’s post (including senior trainees and clinical or research fellows), non-cataract surgeons and those who were not supervising trainees. Each Consultant was invited to participate in the study by email, following discussions of the importance of human factors and NTS in surgery. Data were collected on each Consultant’s familiarity with NTS scoring systems, their prior encounters with intraoperative ophthalmic emergencies, and the approximate number of cataract surgeries they had performed. This questionnaire aimed to develop HUFOES further using the Delphi method; a scientific process used to reach a specific outcome through expert consensus, using multiple rounds of iteration until overall consensus has been achieved, using a recommended sample size of 10–18 experts [26, 27]. In this Delphi method, there is no limit to how many rounds there must be before consensus is achieved. Each participating Consultant was therefore asked to grade the importance of each HUFOES component on a scale of 1–10, before providing free text and ‘tick box’ opinions on matters including specificity, format and layout. Participating Consultants could also suggest the addition, removal or modification of individual components. If consensus amongst participants could not be achieved after the first round of the Delphi process, modifications proposed by the results of the first questionnaire were to be discussed and agreed upon by the focus group members. The accepted modifications would generate the second draft of HUFOES, and the second questionnaire would be amended to reflect this. The second questionnaire would be distributed to all Consultants who had provided answers to the first, which would gather data on the modifications. As per Delphi methodology, this step would be repeated until saturation of information and overall consensus had been achieved.

Once consensus had been achieved, a final questionnaire gathered data on the feasibility, educational impact and preliminary validation of HUFOES once the final version had been agreed through expert consensus.

Statistical analysis

Data were collected on the online platform QualtricsXM (Qualtrics, Provo and Seattle, USA). Microsoft Excel 2019 (Microsoft®, Redmond, Washington, USA) was used for data storage and analysis. Mean scores were generated for the importance grading of each HUFOES’ component, for which a score of 8/10 (agreed amongst the focus group) was used to merit inclusion in the next version of HUFOES. A Cronbach alpha correlation coefficient was calculated for each category as a means to assess interrater agreement on its importance; a score of >0.7 was used as the value for ‘good’ interrater agreement, and the statistical measure by which consensus had been achieved. The sample size was also considered sufficient once consensus amongst respondents was achieved with this measure. For Likert Scale responses, a consensus was defined as the majority either ‘agreeing’ or ‘somewhat agreeing’.

Results

Eighteen cataract surgeons participated in this study. Four responses were removed due to inadequate questionnaire completion. Feedback from 14 Consultant ophthalmic surgeons was therefore included for data analysis, including the two Consultants from the focus group. Hundred per cent (n = 14) were Consultants supervising cataract surgery trainees. Significantly, 64.3% (n = 9) of surgeons had completed >5000 cataract surgeries, whilst 28.6% (n = 4) had completed between 1001 and 5000. All surgeons had encountered at least six PCR’s in their career. Full participant data relating to cataract surgery experience and familiarity with NTS scoring systems are presented in Table 1.

Table 1 Respondent demographic data relating to cataract surgery experience and familiarity with non-technical skill scoring systems.
Full size table

HUFOES development

All HUFOES components were rated highly following the initial round of development. Each individual component achieved a mean importance grading of >8/10, which merited their inclusion in the final version. Furthermore, the interrater agreement of HUFOES overall was excellent (α = 0.953). The mean importance score for each HUFOES component and the Cronbach alpha for interrater agreement of each non-technical category are presented in Table 2.

Table 2 HUFOES component’s mean importance scores and interrater agreements.
Full size table

No more than seven individual pieces of free text advice regarding components to include or exclude from HUFOES were obtained overall. When reviewed by the focus group, it was agreed that the suggested components had already been included within HUFOES components, and components were reformatted to make this clear.

In all, 35.7% of surgeons (n = 5) strongly agreed that HUFOES components in their current form were specific enough for use in intraoperative PCR, 57.1% (n = 8) somewhat agreed, whilst 7.1% (n = 1) neither agreed nor disagreed.

In total, 21.4% of surgeons (n = 3) strongly agreed that the HUFOES components were measurable enough for an assessor, 42.9% (n = 6) somewhat agreed, 21.4% (n = 3) neither agreed nor disagreed, whilst 14.3% (n = 2) somewhat disagreed.

The layout of HUFOES was considered extremely easy to navigate by 28.6% (n = 4), somewhat easy by 64.3% (n = 9), whilst 7.1% (n = 1) remained neutral.

The HUFOES’ Likert assessment scale of 1–5 was considered appropriate overall, with 14.3% (n = 2) in strong agreement. In all, 64.3% (n = 9) somewhat agreed, 14.3% (n = 2) remained neutral, whilst 7.1% (n = 1) somewhat disagreed. The additional comments box was considered useful, with 50.0% (n = 7) in strong agreement, 14.3% (n = 2) somewhat agreed, 28.6% (n = 4) remained neutral, whilst 7.1% (n = 1) somewhat disagreed.

Given the strongly positive results obtained from the first round of Delphi, it was agreed that consensus had been obtained for all categories and components. All categories and components were therefore considered worthy for inclusion in the final version without further modification. Figure 1 displays the final version of HUFOES.

Fig. 1: Final components and layout of the Human Factors in intraoperative Ophthalmic Emergencies Scoring System (HUFOES).
figure 1

The nontechnical skill categories included are Teamwork and Communication, Leadership, Decision Making, Situational Awareness and Professionalism.

Full size image

Validation, feasibility and educational impact

All 14 Consultant ophthalmic surgeons who responded to the development questionnaire were sent the preliminary validation questionnaire, for which the response rate was 100% (n = 14).

Fifty per cent (n = 7) strongly agreed that current training does not go far enough to prepare trainees for intraoperative PCR; 35.7% (n = 5) somewhat agreed, 7.1% (n = 1) remained neutral, whilst 7.1% (n = 1) somewhat disagreed. This is reflected by the respondents’ own level of preparation the first time they were required to manage an unexpected PCR; 28.6% (n = 4) were extremely unprepared, 21.4% (n = 3) were somewhat unprepared, whilst only 50.0% (n = 7) reported themselves to be somewhat prepared. Given their current experience, those Consultant ophthalmic surgeons would be extremely (85.7%, n = 12) or somewhat (14.3%, n = 2) prepared to deal with an unexpected PCR today. Despite this, respondents reported that simulation training in conjunction with HUFOES would have enhanced their preparation for managing a PCR, with 57.1% (n = 8) and 35.7% (n = 5) in strong or moderate agreement, respectively. Respondents agreed that the introduction of HUFOES is needed to facilitate intraoperative NTS training, with 50.0% (n = 7) in strong agreement and 42.9% (n = 6) in moderate agreement.

Data for the perceived usefulness, potential benefit to patient outcomes, appropriateness for simulated or genuine complication scenarios, and potential content and concurrent validity are displayed graphically in Fig. 2.

Fig. 2: Graphs reporting HUFOES’ validity and educational impact data.
figure 2

a Perceived usefulness of HUFOES in preparing ophthalmologists for NTS required for intraoperative emergencies. b Agreement that HUFOES training opportunities would be beneficial to patient outcomes. c Participant comfort level using HUFOES after simulated complication scenario for feedback and training purposes. d Participant comfort level using HUFOES after a genuine intraoperative complication scenario for feedback and training purposes. e Extent of agreement that HUFOES has the ability to distinguish between those of different levels of surgical training. f Extent of agreement that HUFOES components would be able to accurately measure the components they aim to assess.

Full size image

Finally, HUFOES was declared as the preferred option for training NTS in intraoperative ophthalmic emergencies when compared with the current gold standard, NOTSS. In all, 78.6% (n = 11) preferred HUFOES, whilst 21.4% (n = 3) declared no preference.

Discussion

Decision making and judgement are NTS that underpin the technical skills required for complication management. Good judgement and correct and timely decisions will reduce the risk of further complications such as tear extension or a dropped nucleus during unexpected PCR [1]. HUFOES has therefore been designed as a tool for the assessment of the NTS, which are important causes of surgical error (both in ophthalmology and surgery overall) [7, 13, 14, 28]. These include teamwork, communication, leadership, decision making, SA and professionalism. Whilst these categories can be found in almost all other NTS scoring systems, HUFOES is specific to the management of PCR. Assessment tools for NTS must examine observable behaviours relating to the intraoperative phase of surgery, and should be specific for their intended surgical domain [16]. They should not assess personality or emotions [21]. They must be transparent, explicit, reliable and valid, whilst using appropriate language and terminology that both trainee and assessor can understand [16]. HUFOES fulfils these characteristics. Consensus has been achieved amongst the experienced Consultant ophthalmic surgeons that HUFOES components are measurable and specific for use in intraoperative ophthalmic emergencies. It has thus far achieved content validity, whilst overall agreement has been achieved for its ability to distinguish between those of different training levels and experience.

Teamwork, communication and leadership are of particular importance in cataract surgery, as the patient is usually awake and alert under local anaesthetic [11]. Interprofessional communication in all circumstances, but even more so, when complications occur must be instructive without causing alarm or distress to the patient [11, 25]. Furthermore, members of the surgical team are frequently unfamiliar with each other with varying degrees of skill and experience [15]. In the event of an emergency, calm, clear and explicit communication is vital. A change in operative course may require different instruments and preparing unfamiliar, complicated and specialised equipment (such as the phacoemulsification vitrectomy set-up.) [11]. The level and intensity of training needed for these NTS will vary between individuals, based on their current combination of instinctive and learned responses to unexpected emergencies.

SA involves the ability to anticipate problems, recognise an unfolding complication and adapt when emergencies occur. Successful management of PCR can be obtained through its prompt recognition and appreciating the risk of extension, however, subtle and unexpected events are often overlooked in stressful situations [15]. The surgeon must maintain awareness of the patient’s comfort and pain level, and manage these for what may be a longer and more invasive procedure, as absolute focus on the operative field can jeopardise outcomes [28]. Similarly, the surgeon must maintain SA of the theatre environment in order to ensure noise and disruptions are kept to a minimum, and the theatre team are primed for emergency management.

Stress levels of the surgeon and theatre team rise in complicated situations. The stress-response curve demonstrates that moderate stress can be productive through heightening concentration. However, technical errors and poor motion economy result when one’s ability to cope with stress is exceeded [29, 30]. Concerningly, surgeons are often reluctant to recognise their susceptibility to stress [31]. Given that inexperience is a risk factor for PCR, and 90.9% of ophthalmology trainees would require senior support during PCR management, asking for help, support or senior oversight is an important skill to employ [2, 3, 5].

To date, the only human factors simulation study specific to ophthalmology was undertaken by Saleh et al. [15]. They demonstrated that ophthalmology-based human factor simulations are feasible, and reported ANTS and NOTSS as the best performing assessment tools [15, 16, 21]. HUFOES builds on Saleh et al.’s results, whilst simultaneously responding to prior recommendations for an ophthalmology-specific NTS assessment tool [11, 15]. Respondents to our study reported that simulation in conjunction with HUFOES would have enhanced their ability to manage complications such as PCR if it had been available during their training, and were in strong agreement that surgical training does not go far enough to prepare trainees for intraoperative emergencies such as PCR. The introduction of HUFOES was supported, as it has the potential to be useful to trainee ophthalmologists and beneficial for patient outcomes. Encouragingly, our experienced trainer respondents considered HUFOES to be preferable for use in comparison to NOTSS, given its greater levels of specificity for the specialist environment for which it is intended.

We consulted experienced Consultant ophthalmic surgeons for HUFOES’ development and validation. They had limited familiarity and personal experience with NTS scoring systems, and therefore the data collected in this study is free from selection bias and may be considered akin to pure and real-world scenarios. The surgeons consulted were high volume Consultant cataract surgeons of the National Health Service and were experienced trainers in the Kent, Surrey and Sussex Deanery. They had extensive experience managing PCR and general intraoperative complications. Therefore, they had a good understanding of the role of surgical NTS. The validation of HUFOES undertaken in this study has been achieved through expert agreement and consensus, therefore necessitating further evaluation into its educational potential. The next phase of this research is to assess the practical application of HUFOES’ in high-fidelity cataract surgery complication simulations, with specific emphasis placed on its internal consistency, concurrent and construct validities. Whilst HUFOES has been developed using PCR as the specific intraoperative complication, it may be used for intraoperative complications beyond this complication alone. Further research into the wider application of HUFOES will establish this. Furthermore, an HUFOES guidance document is to be developed in order to support those using HUFOES as an assessment tool.

This study has successfully developed the HUFOES; the first focussed and specific NTS assessment tool designed to enhance the safe and effective management of the intraoperative complication of PCR. It is anticipated that HUFOES will be used as a platform to develop a structured NTS training programme for trainee ophthalmologists across the ophthalmology community, in order to enhance their ability to manage intraoperative ophthalmic emergencies. Developing effective NTS in the earlier years of ophthalmic surgical training would be aligned with the aims of other surgical training initiatives [6, 8,9,10,11]. HUFOES is designed to be an effective assessment tool to complement regular training sessions in fully immersive and high-fidelity simulations, in order to monitor and refresh the surgeon’s NTS. HUFOES can be used to assess the NTS of senior ophthalmic surgeons, as well as trainees, however, construct validity data must be obtained before this can be stated with confidence. Finally, HUFOES should not be used for supporting members of the theatre team, as specific behavioural marker systems already exist for staff including anaesthetists and scrub practitioners [21, 22].

Summary

What was known before

  • Nontechnical skills are a fundamental component of successfully managing intraoperative complications such as posterior capsule rupture.

  • Training tools to enhance non-technical skills are currently lacking in ophthalmology.

What this study adds

  • Development of the first nontechnical skills assessment tool specific to managing posterior capsule rupture, according to expert consensus through Delphi methodology.