The aim of this study was to investigate the effect of feedback devices on visual attention and the quality of pediatric resuscitation.
This was a randomized cross-over simulation study at the Medical University of Vienna. Participants were students and neonatal providers performing four resuscitation scenarios with the support of feedback devices randomized. The primary outcome was the quality of resuscitation. Secondary outcomes were total dwell time (=total duration of visit time) on areas of interest and the workload of participants.
Forty participants were analyzed. Overall, chest compression (P < 0.001) and ventilation quality were significantly better (P = 0.002) when using a feedback device. Dwell time on the feedback device was 40.1% in the ventilation feedback condition and 48.7% in the chest compression feedback condition. In both conditions, participants significantly reduced attention from the infant’s chest and mask (72.9 vs. 32.6% and 21.9 vs. 12.7%). Participants’ subjective workload increased by 3.5% (P = 0.018) and 8% (P < 0.001) when provided with feedback during a 3-min chest compression and ventilation scenario, respectively.
The quality of pediatric resuscitation significantly improved when using real-time feedback. However, attention shifted from the manikin and other equipment to the feedback device and subjective workload increased, respectively.
Cardiopulmonary resuscitation with feedback devices results in a higher quality of resuscitation and has the potential to lead to a better outcome for patients.
Feedback devices consume attention from resuscitation providers.
Feedback devices were associated with a shift of visual attention to the feedback devices and an increased workload of participants.
Increased workload for providers and benefits for resuscitation quality need to be balanced for the best effect.
Around 2% of pediatric intensive care unit admissions will experience cardiac arrest.1 Successful high-quality cardiopulmonary resuscitation (CPR) of in-house arrest may lead to the discharge of up to half of these patients.1,2,3 Training in a simulation setting improves the quality of CPR.4,5 Focus is on adequate chest compressions (CCs) (depth, frequency, recoil) and ventilations (frequency, tidal volume, leak, pressure) as well as few interruptions (short hands-off time). This may be achieved with real-time feedback devices used during resuscitation, which may assist in achieving a better quality of resuscitation.6,7,8,9,10,11 These devices relay information on compression as well as ventilation quality.12,13,14,15,16 Even though feedback devices are increasingly being used in educational and clinical settings, the resuscitation guidelines by the European Resuscitation Council still argue against routine implementation17 as evidence is still not clear-cut. Previous studies using the same feedback devices as in our study [QCPR Baby (Laerdal, Stavanger, Norway) for CCs and Neo Training (Monivent AB, Gothenburg, Sweden) for ventilations] have shown an improvement of resuscitation and ventilation performance.6,18,19 However, these devices may even be detrimental and lead to higher subjective stress.20 This increased mental workload may in turn lead to worse performance.21 Eye-tracking technology may allow the assessment of visual attention (VA) of healthcare providers during resuscitation.22,23 This may allow quantification of distractions during resuscitation and in turn lead to strategies to overcome these during resuscitation.
We therefore aimed to study the impact of feedback devices on resuscitation quality with eye-tracking to analyze participants’ performance when supported with feedback devices.
This study was a prospective randomized cross-over simulation-based trial conducted at the Medical University of Vienna, Austria. The study protocol was reviewed according to the Consolidated Standards of Reporting Trials (CONSORT).24 The ethics committee of the Medical University of Vienna gave this study an exempt status.
Medical students in their final year, fellows, nurses, and consultants from our Neonatal Intensive Care Unit were eligible for enrolment.
Study procedure and scenario
Participants received a brief review of the current pediatric CPR guidelines and an introduction to the feedback devices at the beginning of the study. Participants then signed an informed consent form and completed a questionnaire to assess demographic variables and expertise in pediatric resuscitation. Thereafter, eye-tracking glasses (Tobii 2.0, Tobii AB, Danderyd, Sweden) were calibrated as recommended by the company. Resuscitation teams consisted of the participant and a study nurse. Each participant completed a total of four basic life support scenarios according to the ERC pediatric BLS guidelines (15:2) in a cross-over setting.25 Each participant did CC twice, and ventilations (V) twice afterward (see Fig. 1).
The visibility of the feedback device was randomized (sealed envelope) within each pair of scenarios. Depending on group allocation (feedback condition vs. no-feedback condition), the feedback device was either visible or hidden from the participant, but always recording. Accordingly, the four scenarios were:
Participant performing chest compressions without feedback (=C).
Participant performing chest compressions with feedback (=CF).
Participant performing ventilations without feedback (=V).
Participant performing ventilations with feedback (=VF).
Each scenario lasted for 3 min and all tasks were done in one session. Immediately after each scenario, participants were asked to report perceived workload using the standardized NASA-Task Load Index (NASA-TLX).26 We measured the mark made by participants on the scale for the NASA-TLX score.
For CC scenarios we used the QCPR Baby manikin (Laerdal Medical, Stavanger, Norway), whereas for the ventilation scenarios we used the SimNewB manikin (Laerdal Medical, Stavanger, Norway) as it has no internal air leak. The QCPR Baby was connected to the SimPad Plus Skill Reporter device (Laerdal, Stavanger, Norway), which provided real-time visual feedback for CC performance. For ventilation feedback, a flow sensor (Neo Training, Monivent AB, Gothenburg, Sweden) was placed between the face mask (CareFusion Vital Signs Infant Face Mask, Chateaubriant, France) and the bag (Laerdal Silicone Resuscitator Pediatric Basic, Stavanger, Norway) to measure and show inspiratory (VTi) and expiratory tidal volume (VTe, graphically and numerically), peak inspiratory pressure (numerically), and mask leak (leak, numerically) in real time. A tablet (iPad, Apple Inc., Cupertino, CA) recorded the parameters from the sensor wirelessly for further analysis. Participants wore Tobii 2.0 mobile eye-tracking glasses (Tobii, Danderyd, Sweden) to record real-time gaze behavior in all scenarios.
The primary outcome was the quality of CCs and ventilations. The quality of CCs was measured with a total compression score (%), which is automatically calculated by the device including CC rate, CC rate compliance (percentage of correct CC rate), depth, depth compliance (percentage of correct CC depth), complete release, and hand position. Ventilation parameters included positive inspiratory pressure (<30 cmH2O),27 tidal volumes (4–8 mL/kg),12,28 and leakage (as low as possible). We evaluated whether the use of feedback devices improved or worsened the quality of CPR as compared to no-feedback conditions.
As secondary outcomes, we evaluated participants’ dwell time (amount of time looking at an area of interest [AOI]) and subjective workload. AOI describes a specific AOI in the video recording that is defined by researchers in the analysis process. For dwell time, the study group determined six AOIs: (1) feedback device (if available), (2) ventilation bag, (3) infant chest, (4) ventilation mask, (5) study nurse, and (6) others. Participants’ workload was measured with the NASA-TLX.26
The sample size calculation with the G*Power software for a paired t test with two repeated measures (feedback, no-feedback) revealed that a sample size of 34 participants would provide sufficient power (0.80) to detect an effect at the alpha level of 0.05 with medium effect size (dz = 0.50).29
Descriptive statistics were used to describe the sample and analyze the acceptance of feedback devices. Paired t tests were used to examine whether resuscitation quality, VA, and workload changed across the feedback conditions. Multiple regression analyses were conducted to test whether changes in resuscitation quality were predicted by participants’ expertise, VA, and workload. For VTe and VTi scores, χ2 tests were applied to compare the proportion of scores within the 4–8 mL/kg range across the feedback conditions. All analyses were performed with SPSS 24.0 (IBM Corp., Armonk, NY). The level of significance was set at P < 0.05 (two-tailed). Cohen’s dz (for paired t tests) were used as the measure of effect size, with 0.20, 0.50, and 0.80 indicating small, medium, and large effect, respectively. Parameters with a skewed data distribution were log transformed before analysis.
Data from 40 participants (25 females and 15 males) were collected. The recruitment period was from May 2019 to June 2020. Participants’ characteristics are shown in Table 1.
Participants showed significant improvements in several CC parameters when provided with real-time feedback (Table 2). In particular, they had a 22% higher total CC score and a 25% higher CC rate compliance than in the no-feedback condition (both P < 0.001). Self-reported CC quality increased by 7% (P = 0.03). Mean CC depth increased by 0.9 mm with feedback (P = 0.01), without having an effect on CC depth compliance. Years of clinical practice did not predict any changes in CC parameters, indicating that participants benefited from real-time feedback regardless of their experience.
Data of one participant were excluded because of technical issues with the flow sensor. Participants showed improvements in VTi, VTe, and mask leak when provided with real-time feedback (Table 3). The mean VTi improved with feedback, increasing from 18% without feedback to 33% with feedback (χ2 = 6.27, P = 0.012). The mean VTe was similar in both feedback conditions, but the proportion of VTe in the 4–8 mL/kg compliance range increased from 36% in the no-feedback condition to 67% in the feedback condition (χ2 = 16.05, P < 0.001). Mask leak was 7.5% lower with feedback (P = 0.009). Participants with less experience were lower than a VTe of 4–8 mL/kg limit more frequently than their more experienced counterparts (β = 0.45, P < 0.001).
VA in the CC task
In the no-feedback condition, infant chest was the most frequently attended AOI (73%), followed by ventilation mask (22%). The addition of a feedback device had a powerful effect on participants’ VA, with the feedback device being now the most frequently attended AOI (49%), followed by infant chest (33%) and ventilation mask (13%). Paired comparisons revealed that participants significantly reduced their attention to the infant chest and the mask in the feedback condition compared with the no-feedback condition (Table 2). Years of clinical practice did not predict changes in VA except for the feedback device AOI; more experienced participants tended to dwell on the device longer than less experienced participants (β = 0.34, P = 0.04). Furthermore, participants who dwelled longer on the device significantly reduced their mean CC rate (β = −0.29, P = 0.01), thereby exceeding the 120 CCs per minute limit less frequently and thus improving their CC rate compliance (113.2 with feedback vs. 120.2 without feedback).
VA in the ventilation task
Similar to the CC scenario, the infant’s chest was the most frequently attended AOI (52%), and the ventilation mask the second (40%) in the no-feedback condition. In the feedback condition, the feedback device was the most frequently attended AOI (40%), whereas participants’ attention to the infant’s chest and the mask declined significantly (Table 3). Years of clinical practice did not predict changes in VA except for the mask AOI; more experienced participants tended to dwell on the ventilation mask shorter than less experienced participants when provided with real-time feedback (β = −0.31, P = 0.03). Dwell time on the feedback device predicted changes in the mean VTe (β = 0.23, P = 0.03), with participants who dwelled longer on the device having higher mean VTe (7.0 mL/kg) than participants who attended to the device for a shorter time (6.6 mL/kg).
Workload in the CC task
The average NASA workload in the feedback condition was 37%, which was ∼3.5% higher than in the no-feedback condition (P = 0.02). This was especially due to the increases in mental demands (P = 0.006), performance demands (P = 0.05), and frustration (P = 0.002) (Table 2). More experienced participants experienced higher average workload (β = 0.37, P = 0.001), higher mental demands (β = 0.39, P = 0.003), and put more effort into the task (β = 0.26, P = 0.02) than less experienced participants. The average workload in the feedback condition did not predict changes in participants’ actual CC quality. There were only two effects of the NASA subscales on self-reported CC quality: higher performance demand and higher frustration were related to lower self-reported CC quality in the feedback condition (β = −0.44, P = 0.003, and β = −0.50, P = 0.001, respectively).
Workload in the ventilation task
The average NASA workload in the feedback condition was 36%, which was ∼8% higher than in the no-feedback condition (P < 0.001). This was especially due to increases in mental demands (P < 0.001), performance demands (P = 0.002), and frustration (P < 0.001) (Table 3). The increase in workload was unrelated to years of clinical practice. The average workload, performance demand, and frustration were related to a decline in self-reported ventilation quality (β = −0.58, P < 0.001, β = −0.63, P < 0.001, and β = −0.63, P < 0.001, respectively). In addition, higher frustration was related to higher VTi (β = 0.28, P = 0.04), with participants with VTi in the 4–8 mL/kg range being less frustrated than participants with VTi out of that range (28 vs. 53%).
The acceptance of feedback device
The majority of participants found the use of both feedback devices, SimPad and Monivent, helpful for visualization of the CC and ventilation quality, respectively. More than 90% of participants perceived the feedback from the SimPad device as easily interpretable. The answers to the Monivent device were more ambiguous, with 62% of participants rating the feedback as rather easy to interpret and 36% of participants perceiving it rather difficult. Participants further stated that using feedback devices would also be beneficial in an actual scenario in a pediatric patient (Table 4).
In this study, we observed the effect of feedback devices on the quality of pediatric resuscitation with eye tracking to analyze participants’ attention and workload. There were three main findings: (i) As in previous studies, we found that performance quality improved with real-time feedback for both CCs and ventilations. Feedback devices are designed to assist in delivering optimal resuscitation quality.6,7,8,9,10,12,13,14,15 A recently published systematic review also supported the use of feedback devices for pediatric as well as adult resuscitation support.30 The feedback devices helped the practitioners to appropriately recognize and optimize resuscitative tasks rather than serving as a distraction.
(ii) The addition of a feedback device influenced VA. When feedback devices were used, participants’ attention shifted significantly to the device (see Fig. 2) and away from the manikin consistent with former studies.31,32 However, this shift in attention was not harmful to performance in our study. We further observed that more experienced participants spent more time looking at the feedback device than on the manikin. These participants may have needed more time to interpret the feedback from the device or relied on their haptic memory requiring less confirmation of their task by looking directly at the manikin. However, also the non-feedback group’s dwell time was mostly on the infant’s chest and mask, likely to determine adequate chest rise. Probably, the participants were looking for feedback or needed feedback on how they were doing by paying attention to the areas that would tell them the quality of ventilation. While it sounds reasonable that participants are actively seeking feedback on performance, there is no literature that confirms this so far.
(iii) The addition of a feedback device increased subjective workload, but this increase was small33 and did not interfere with resuscitation quality. The increase was markedly higher when the participant had more years of practice, but only when performing CCs. Interestingly, a higher subjective workload was commonly paired with frustration when using feedback devices, but the quality of CCs and ventilation did not suffer in our study. The relationship between workload and clinical performance has been reported controversially in the literature. Previous studies have described a potential reduction of workload when using feedback devices,34 but also an increased workload with improved quality of CC.20 While feedback devices have beneficial effects on resuscitation performance, the effect might decrease once feedback devices are used on a daily basis and are not special to providers anymore.35 Ultimately, these devices make CPR performance measurably and their clinical use may allow the identification of the best practice during resuscitation.11
Overall, our findings support previous reports that feedback devices may be an important tool in CPR in situ as well as in real life,36 also with an improvement of return of spontaneous circulation rates.37
However, while we are not able to provide clinical data, it was interesting to see that while feedback devices increased subjective workload, resuscitation quality was mostly not influenced. Future studies should also focus on interviews with participants after the scenario, to identify why and what exactly influenced workload. Most providers were using feedback devices for the first time, which might in part explain the higher perceived workload. It is also reasonable that repetitive and regular training with feedback devices might decrease subjective workload while still impacting the quality of resuscitation positively. Alternatively, this increase in workload with feedback devices may be offset by a resuscitation coach who relays CPR quality information to the resuscitating provider.38,39 We assume that adding live feedback devices in real clinical situations, such as cardiopulmonary resuscitation and positive pressure ventilation, can help to increase the quality of patient care. However, clinical studies analyzing the utility of feedback devices in such situations including VA and workload are needed.
This was a simulation-based trial not involving any real patients. Furthermore, the study design did not allow for blinding. Participants knew they were being studied, which might amplify the shift of VA to the feedback device. In addition, we were not able to evaluate any other feedback routes such as voice or sound cues. Furthermore, scenarios were limited to 3 min, so we were unable to identify possible changes of workload with a longer duration of resuscitation. Workload measurement was based on subjective reports by participants.
We found that resuscitation quality significantly improved when using feedback devices in a simulated setting. VA shifted to the feedback device, when provided, and away from the simulated patient and other equipment. The subjective workload was higher with the feedback device. High-quality randomized controlled trials, as well as real-patient trials focusing on patient outcomes, are needed to determine the impact of VA and stressors when using feedback devices.
Slonim, A. D., Patel, K. M., Ruttimann, U. E. & Pollack, M. M. Cardiopulmonary resuscitation in pediatric intensive care units. Crit. Care Med. 25, 1951–1955 (1997).
Peddy, S. B. et al. Cardiopulmonary resuscitation: special considerations for infants and children with cardiac disease. Cardiol. Young 17(Suppl. 2), 116–126 (2007).
Gupta, P. et al. Outcomes following single and recurrent in-hospital cardiac arrests in children with heart disease: a report from American Heart Association’s get with the guidelines registry-resuscitation. Pediatr. Crit. Care Med. 17, 531–539 (2016).
Young, A. K. et al. Use of a simulation-based advanced resuscitation training curriculum: Impact on cardiopulmonary resuscitation quality and patient outcomes. J. Intensive Care Soc. 21, 57–63 (2020).
Moretti, M. A. et al. Advanced cardiac life support training improves long-term survival from in-hospital cardiac arrest. Resuscitation 72, 458–465 (2007).
Wagner, M. et al. Effects of feedback on chest compression quality: a randomized simulation study. Pediatrics 143, e20182441 (2019).
Goharani, R. et al. Real-time compression feedback for patients with in-hospital cardiac arrest: a multi-center randomized controlled clinical trial. J. Intensive Care 7, 5 (2019).
Cheng, A. et al. Improving cardiopulmonary resuscitation with a CPR feedback device and refresher simulations (CPR CARES Study): a Randomized Clinical Trial. JAMA Pediatr. 169, 137–144 (2015).
Bobrow, B. J. et al. The influence of scenario-based training and real-time audiovisual feedback on out-of-hospital cardiopulmonary resuscitation quality and survival from out-of-hospital cardiac arrest. Ann. Emerg. Med. 62, 47–56.e41 (2013).
Allan, K. S., Wong, N., Aves, T. & Dorian, P. The benefits of a simplified method for CPR training of medical professionals: a randomized controlled study. Resuscitation 84, 1119–1124 (2013).
Lin, S. & Scales, D. C. Cardiopulmonary resuscitation quality and beyond: the need to improve real-time feedback and physiologic monitoring. Crit. Care 20, 182 (2016).
Schmölzer, G. M. et al. Respiratory function monitor guidance of mask ventilation in the delivery room: a feasibility study. J. Pediatr. 160, 377–381.e372 (2012).
Khoury, A., De Luca, A., Sall, F. S., Pazart, L. & Capellier, G. Ventilation feedback device for manual ventilation in simulated respiratory arrest: a crossover manikin study. Scand. J. Trauma Resusc. Emerg. Med. 27, 93 (2019).
Gould, J. R., Campana, L., Rabickow, D., Raymond, R. & Partridge, R. Manual ventilation quality is improved with a real-time visual feedback system during simulated resuscitation. Int. J. Emerg. Med. 13, 18 (2020).
Bowman, T. A., Paget-Brown, A., Carroll, J., Gurka, M. J. & Kattwinkel, J. Sensing and responding to compliance changes during manual ventilation using a lung model: can we teach healthcare providers to improve? J. Pediatr. 160, 372–376.e371 (2012).
Perkins, G. D. et al. European Resuscitation Council Guidelines for Resuscitation: 2017 update. Resuscitation 123, 43–50 (2018).
Van de Voorde, P. et al. European Resuscitation Council Guidelines 2021: paediatric life support. Resuscitation 161, 327–387 (2021).
Lin, Y., Cheng, A., Grant, V. J., Currie, G. R. & Hecker, K. G. Improving CPR quality with distributed practice and real-time feedback in pediatric healthcare providers - a randomized controlled trial. Resuscitation 130, 6–12 (2018).
Gurung, R. et al. Effectiveness and acceptability of bag-and-mask ventilation with visual monitor for improving neonatal resuscitation in simulated setting in six hospitals of Nepal. J. Nepal Health Res. Counc. 17, 222–227 (2019).
Brown, L. L. et al. Impact of a CPR feedback device on healthcare provider workload during simulated cardiac arrest. Resuscitation 130, 111–117 (2018).
Byrne, A. J. et al. Novel method of measuring the mental workload of anaesthetists during clinical practice. Br. J. Anaesth. 105, 767–771 (2010).
Wagner, M. et al. Eye-tracking during simulation-based neonatal airway management. Pediatr. Res. 87, 518–522 (2020).
Law, B. H. Y. et al. Analysis of neonatal resuscitation using eye tracking: a pilot study. Arch. Dis. Child Fetal Neonatal Ed. 103, F82–f84 (2018).
Cheng, A. et al. Reporting Guidelines for Health Care Simulation Research: extensions to the CONSORT and STROBE statements. Simul. Health. 11, 238–248 (2016).
Maconochie, I. K. et al. European Resuscitation Council Guidelines for Resuscitation 2015: Section 6. Paediatric life support. Resuscitation 95, 223–248 (2015).
Hart, S. G. et al. Development of NASA TLX. Adv. Psychol. 52, 139–183 (1988)
Wyllie, J. et al. European Resuscitation Council Guidelines for Resuscitation 2015: Section 7. Resuscitation and support of transition of babies at birth. Resuscitation 95, 249–263 (2015).
Chowdhury, O. et al. Volume-targeted ventilation in infants born at or near term. Arch. Dis. Child. Fetal Neonatal Ed. 97, F264 (2012).
Ferguson, C. J. An effect size primer: a guide for clinicians and researchers. Prof. Psychol. Res. Pract. 40, 532–538 (2009).
Gugelmin-Almeida, D., Tobase, L., Polastri, T. F., Peres, H. H. C. & Timerman, S. Do automated real-time feedback devices improve CPR quality? A systematic review of literature. Resuscitation 6, 100108 (2021).
Herrick, H. et al. Provider visual attention on a respiratory function monitor during neonatal resuscitation. Arch. Dis. Child Fetal Neonatal Ed. 105, 666–668 (2020).
Katz, T. A. et al. Visual attention on a respiratory function monitor during simulated neonatal resuscitation: an eye-tracking study. Arch. Dis. Child Fetal Neonatal Ed. 104, F259–F264 (2019).
Mazur, L. M., Mosaly, P. R., Hoyle, L. M., Jones, E. L. & Marks, L. B. Subjective and objective quantification of physician’s workload and performance during radiation therapy planning tasks. Pract. Radiat. Oncol. 3, e171–e177 (2013).
Yamada, N. K., Fuerch, J. H. & Halamek, L. P. Ergonomic challenges inherent in neonatal resuscitation. Children 6, 74 (2019).
Troy, L. et al. Healthcare provider perceptions of cardiopulmonary resuscitation quality during simulation training. Pediatr. Crit. Care Med. 20, e473–e479 (2019).
Hostler, D. et al. Effect of real-time feedback during cardiopulmonary resuscitation outside hospital: prospective, cluster-randomised trial. BMJ 342, d512 (2011).
Vahedian-Azimi, A. et al. Effect of the Cardio First Angel device on CPR indices: a randomized controlled clinical trial. Crit. Care 20, 147 (2016).
Tofil, N. M. et al. Effect of a cardiopulmonary resuscitation coach on workload during pediatric cardiopulmonary arrest: a multicenter, simulation-based study. Pediatr. Crit. Care Med. 21, e274–e281 (2020).
Hunt, E. A. et al. Improved Cardiopulmonary Resuscitation Performance With CODE ACES(2): a resuscitation quality bundle. J. Am. Heart Assoc. 7, e009860 (2018).
We acknowledge support from Monivent AB (Gothenburg, Sweden), who provided us with training as well as technical support during the study. We further thank Lukas Zwingl and the entire pediatric simulation team for their continuous help during the study. This work was supported by ZOLL Foundation, grant number 74774258.2. The funder did not have any influence in study design, data collection, analysis, decision to publish, or preparation of the manuscript.
The authors declare no competing interests.
The ethics committee of the Medical University of Vienna gave this study an exempt status. As this study was a simulation-based study and no trial registration was done.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Wagner, M., Gröpel, P., Eibensteiner, F. et al. Visual attention during pediatric resuscitation with feedback devices: a randomized simulation study. Pediatr Res (2021). https://doi.org/10.1038/s41390-021-01653-w