Introduction

Eye-tracking devices are an innovative tool to understand provider behavior during stressful medical tasks,1,2 mainly tested on team leaders.3,4,5,6,7 Eye-tracking technology enables simulation instructors as well as researchers to analyze the focus of different health-care providers during critical simulated and real situations.8,9 Analysis of gaze behavior has shown that novice providers are more dependent on vital signs’ monitoring than experts, which leads to more distraction from clinical assessment of the newborn.4 In studies involving health-care providers wearing eye-tracking devices to provide real-time feedback during the clinical care of a newborn, providers were potentially distracted from clinical tasks in favor of monitoring systems.1,2 Such distractions can be potentially risky to patient safety. Yamada et al. reported that the majority of errors during neonatal resuscitation were due to lack of situational awareness (such as when being distracted) and insufficient knowledge.10 In contrast, being trained in a standardized visual search pattern, thereby minimizing distraction, led to a greater accuracy and efficiency during laparoscopic tasks.11,12

Experts usually focus more often than non-experts on the most task-relevant areas, such as the patient’s chest and airway during a pediatric emergency6 and are faster in attending to task-relevant information in critical situations.13 Eye-tracking devices can thus provide data that reflect health-care providers’ proficiency and expertise, with insight into their first-person view and visual attention during clinical care on different areas of interest (AOIs). This information is necessary to enhance our understanding of providers’ visual perspectives during clinical care and/or training and could be then beneficially used to augment and adapt clinical care and/or training.

Specifically, eye-tracking devices could be an effective tool during neonatal resuscitation. About 10% of neonates need help with initiation of breathing but only a minority needs intensive resuscitation, including chest compressions, emergency vascular access, intubation, and medication.14,15 Positive pressure ventilation is the most crucial part during neonatal resuscitation. Difficulties with bag-mask ventilation as well as mask leaks often lead to insufficient ventilation of the baby and therefore to an increase of hypoxia and bradycardia after delivery.14,15,16 While it might seem simple, adequate airway management is challenging and requires the providers’ undivided focus and attention.17 The simultaneous focus on the correct application of ventilations and gathering the quality of ventilations (i.e., vital parameters, chest movement) while coordinating ventilations with chest compressions and communication with other team members is a challenging task for every health-care provider.3 To accomplish this task effectively, it is critical that the provider focuses his or her gaze on the neonate as adequate chest rise serves as real-time feedback for adequate ventilations. This can be a challenging task, as providers might be easily distracted by other tasks being performed simultaneously by other team members as well as potentially focusing on the monitor rather than on the patient.

The aim of this study is to assess airway providers’ gaze behavior and usability of eye-tracking glasses during simulation-based neonatal resuscitation training. We focused only on airway management during this study and employed eye-tracking. We assessed the visual attention of health-care providers while ventilating at different times of interest (TOIs) in a simulated scenario including during ventilation only, chest compressions, umbilical vein catheter insertion, and endotracheal intubation. Furthermore, we evaluated how participants perceived the use of eye-tracking glasses to better understand the best use during simulation-based medical education in the future.

Methods

This study was an observational simulation-based study to explore participants’ gaze behavior and subjective experience with the eye-tracking device during neonatal resuscitation focusing on airway management. Primary outcome was gaze behavior during airway management. The study was conducted at the Yale Medical Simulation Center in January 2019.

Participants and procedure

Participants were medical students and residents who attended a simulated neonatal resuscitation training session during their obligatory education. They were grouped into teams, consisting of two to four participants each, to complete a neonatal resuscitation scenario. After briefing on the neonatal resuscitation algorithm, participants were assigned to perform an airway task, chest compressions, or other medical tasks. The team member assigned to the airway was wearing head-mounted eye-tracking glasses (Tobii Pro, Stockholm, Sweden) to record his or her gaze behavior during the scenario. Participants’ subjective experience with the eye-tracking device was assessed after the scenario using a questionnaire. Ethics approval for this study was obtained from the Yale University Institutional Review Board. This research was deemed exempt under 45CFR46.104.4

Scenario

The SimNewB (Laerdal Medical, Laerdal, Stavanger, Norway) was used for the scenario. The simulation scenario (see supplementary appendix) was a delivery of a newborn infant with unknown gestational age without prenatal care in a bathroom near an emergency room of a general peripheral hospital. Participants were given details about the setting and history of the patient, but there was no help from instructors during the scenario. Vital parameters were shown on the monitor after attaching the respective sensors. The mannequin was controlled by a technician according to the scripted scenario. The non-vigorous newborn presented floppy without any breathing effort and a cyanotic skin color. The initial heart rate was 50 beats per minute, respiratory rate was 0, and saturation non-measurable. Learning goals of this scenario were equipment check, assessment of a newborn infant, positive pressure ventilation, chest compressions, endotracheal intubation, and administration of epinephrine. After achieving all learning goals, the heart rate increased to 140 beats per minute and the saturation increased to 95%.

Data collection and processing

In total, 13 teams completed the scenario. Each participant assigned to the airway task wore Tobii Pro 2.0 eye-tracking glasses (Tobii Pro, Stockholm, Sweden). Eye movements were sampled at a rate of 50 Hz. The recording produced a first-person view video with an overlying pupil fixation showing where the participant was looking in real time. The data were then uploaded to a secured study computer and analyzed with the Tobii Pro Glasses Analyzer software (Tobii Pro, Stockholm, Sweden). Clinical experts of the research team determined four key AOIs before the study: (1) the newborn baby, (2) instruments (ventilation bag, laryngoscope, and endotracheal tube), (3) patient monitor, and (4) other personnel/staff. Furthermore, we identified four TOIs, representing critical parts during newborn resuscitation: (a) the time of ventilation, (b) the time of ventilation while other provider was performing chest compressions, (c) the time of ventilation while other provider was inserting the umbilical vein catheter, and (d) the time of endotracheal intubation with subsequent ventilation for 30 s. We analyzed total time spent on each of the above AOIs (dwell time) during the four critical TOIs.

After the scenario, all participants answered 14 items about discomfort associated with the glasses and reported how distracted they were because of the glasses and whether they would wear the glasses during a real scenario (Fig. 1). Responses were made on a 5-point Likert scale from 1 (agree) to 5 (disagree), with higher score representing less distraction and higher willingness to wear the glasses.

Fig. 1
figure 1

Participants’ subjective experience with the eye-tracking glasses during the scenario. *P < 0.05 (Wilcoxon signed-rank test)

Full size image

Statistics

Time spent on each AOI was analyzed as a proportion of total AOI dwell time because the completion time of the scenario varied across the teams. The Friedman test was used to assess significant differences in the overall proportion of time spent on the four AOIs. Kruskal–Wallis test was employed to control for significant differences among two-member teams, three-member teams, and four-member teams in the proportion of time spent. The Friedman test was again used to test significant differences in time spent on each AOI across the four TOIs. Finally, a one-sample Wilcoxon signed-rank test was performed to test whether participants’ subjective experience with eye-tracking glasses was different from 3 (=the scale median). All statistical analyses were performed with SPSS 24.0 (IBM Corp.; Armonk, NY, United States). The level of significance was set at P < 0.05 (two tailed).

Results

Data from 13 participants (8 female and 5 male) were collected. The majority of participants had prior experience in simulation-based training, also including the intubation of a simulated newborn, but they were largely inexperienced in intubating a real neonate patient. Few participants had prior experience with wearing eye-tracking glasses. Participants’ characteristics are shown in Table 1.

Table 1 Participants’ characteristics
Full size table

Gaze behavior

Overall, there were significant differences in dwell time within different AOIs during the scenario (P < 0.001), with participants spending twice as much time on the newborn and instruments as on the monitor and other staff (Fig. 2). There were no significant differences among two-member teams, three-member teams, and four-member teams with regard to the proportion of dwell time. However, the proportion of dwell time varied across the four TOIs (Fig. 3). Participants spent about 25% more time focusing on another provider while the provider was inserting the umbilical vein catheter than in all other TOIs (P = 0.04, Fig. 4). Furthermore, time spent on the instruments during intubation was higher than that during the other TOIs, but this difference just missed significance (P = 0.06). There were no significant differences in dwell time on the newborn and monitor across the TOIs.

Fig. 2
figure 2

Comparison of time spent in the different AOIs (as a proportion of total AOI dwell time) during the scenario

Full size image
Fig. 3
figure 3

Comparison of time spent in the different AOIs (as a proportion of total AOI dwell time) for the different parts of the scenario

Full size image
Fig. 4
figure 4

Exemplary heat map of participants’ gaze behavior during the time of ventilation while another participant was inserting the umbilical vein catheter

Full size image

Subjective experience with eye-tracking

None of the participants removed the eye-tracking glasses during the simulation scenario. The use of the glasses was perceived easy and not disturbing, as indicated by participants’ responses that were largely different from the scale median (Fig. 1). Participants also denied any discomfort related to the glasses and they did not report that it impeded patient care. However, participants were unsure if they would wear the glasses during a real scenario.

Discussion

In this study, we examined the feasibility and usability of eye-tracking glasses during simulated neonatal resuscitations analyzing the gaze behavior of the participants in charge of airway management. This is the first study to our knowledge using eye-tracking glasses during the airway management of a neonate. Previous studies utilizing eye-tracking were conducted during trauma and pediatric resuscitations,3,5,6 endotracheal intubation,18 education of nurses and paramedics,19 radiology,20 and surgery.21 Law et al. described the use of eye-tracking glasses in a pilot study during real primary care of newborn infants assessing what participants were looking at during a neonatal resuscitation.2

In our study, the utilization of mobile eye-tracking glasses during a simulated neonatal resuscitation scenario, including critical tasks such as emergency vascular access, intubation, chest compressions, and administration of medication, was feasible and participants’ responses in favor of eye-tracking glasses support this statement.

We were able to see what participants were looking at during the whole scenario and especially during critical TOIs. Thereby, we were able to use them as an instrument to help the instructors and learners identify possible critical parts of a newborn resuscitation in terms of suboptimal visual attention and furthermore to guide future educational training. This helped the instructors as well as the research team to better understand when and how participants were visually distracted from a specific task such as neonatal airway management. During the scenario, participants spent about twice as much time on the mannequin and instruments as on the monitor and other personnel, indicating that these AOIs were more salient to them. Participants presumably used chest rise and instrument position as a source of real-time feedback more often than the values on the monitor. This is educationally important information that would not have been available for discussion and learning without the eye-tracking glasses and it could be valuable for enhancing team training and focused task training.

Furthermore, we were able to show that participants were particularly distracted from their airway management task when another complex task was performed by another team member such as the insertion of an umbilical vein catheter simultaneously. We noticed that participants stopped ventilation for a period of time while being distracted by other tasks. However, as this was only a pilot, exploratory study, we did not measure the quality of ventilation quantitatively. The reasons for this distraction are unclear and we were not able to assess what prompted the participant to move the gaze to the distraction. Possible reasons are curiosity in the procedure, thoughts that the colleague might need assistance with the procedure, inexperience, feeling responsible for the overall management of the patient, and many others. Qualitative, structured interviews following eye-tracking scenarios could potentially help to better understand how to help participants stay focused on their task.

Our results showed that, in about 30% of the whole scenario, the participant wearing eye-tracking glasses was looking at the newborn, which is similar to the study by Law et al.2 in a real setting. We found differences regarding the gaze behavior on monitors and other participants, with 9% on monitors in our study (26% in the real setting, respectively) and 17% on other participants (5% in the real setting, respectively). In this study, the participant wearing the glasses was both the team leader and the designated airway provider. In our study, the designated airway provider was focused on the airway only with another provider functioning as the team leader. Prior studies have noted differences in gaze behavior depending on providers’ different experience levels.6

Simulation-based training is a well-established tool for improving health-care providers’ technical skills and non-technical.22,23,24,25,26 However, it is challenging to objectively evaluate non-technical skills, such as situational awareness. Gaze behavior analysis is a promising tool to enhance measurement and research on this non-technical skill.8

During resuscitation, different tasks require providers’ attention and gaze tracking can provide a description of providers’ patterns of gaze behaviors, such as fixation and distraction. We observed that participants were very focused and not visually distracted during very engaging skills, such as intubation. In the current paper, this finding was not statistically significant, but our study was not sufficiently powered for this outcome. Future studies could explore whether gaze patterns vary when a provider is allocated to a specific task (airway management only) or to several tasks at the same time (airway management plus team leader).

The subjective experience of wearing the eye-tracking glasses during the simulation was overall positive and the device was not perceived as disturbing or uncomfortable. At the same time, participants reported that they would be hesitant to wear the glasses during a real scenario, raising the question if they truly were perceived as non-distracting or if there are other reasons as to why there is a hesitation to wear the glasses during real cases, such as patient distractions. Qualitative interviews could help explore these reasons and help describe the optimal use of eye-tracking devices. Future work should study the impact of wearing eye-tracking glasses both on real patients and on patient outcomes. Furthermore, differences regarding experience (novices vs. experts) need to be acknowledged in future studies. However, we acknowledge that, owing to different settings and outcome parameters, it seems challenging to compare eye-tracking studies, therefore we recommend using similar settings and AOIs in further studies.

Limitations

There are two major limitations to this study. First, this was an observational feasibility study with a small number of participants that may limit the generalizability of our results. These data can be used to guide the power analysis of future interventional studies. Another limitation was that this was a retrospective study assessing videos after an educational event with limited influence on the case scenario and without a standardized debrief. In the future, a prospective interventional study design will help us better understand the nuances of how to best integrate eye-tracking devices into our teaching approach.

Conclusion

This study demonstrated that it is feasible to use eye-tracking glasses to describe participant’s gaze behavior during a simulated neonatal resuscitation scenario. Prospective interventional studies will better characterize the ideal use and application of this device and should also evaluate attention and distractions and correlate it with experience levels and quality of care.