Introduction

Less invasive surfactant administration (LISA) is a technique to administer surfactant to preterm infants with respiratory distress syndrome (RDS). LISA is performed by placing a thin catheter below the vocal cords of spontaneously breathing infants on non-invasive respiratory support.1,2,3,4 Since the 2016 update of the European Consensus Guidelines on Management of Respiratory Distress Syndrome,5 LISA has been suggested as the preferred method of surfactant administration in spontaneously breathing premature infants and is now widely used in neonatal intensive care units (NICUs) globally.6,7,8,9 However, the risks of airflow obstruction, apnea, desaturations, bradycardia, misplacement of the catheter, pain and discomfort persist.10 Video laryngoscopy may reduce some of these adverse events.11,12 Effective and safe performance of LISA requires an experienced operator and supporting clinical team.13 However, operator and team’s experience with LISA may be limited due to the relatively small number of infants eligible for LISA per clinician. Standardized simulation-based LISA education, including use of video laryngoscopy, has been recommended to improve LISA success rates.14,15,16,17 Nonetheless, to date, there is a lack of consensus on clinical practice and training required for gaining LISA proficiency.18 Therefore, clinicians would benefit from an internationally consented curriculum, based on sound clinical evidence and expert’s experience, that will serve as a template for LISA accreditation. Curriculum development should follow a structured approach starting with problem identification and a general needs assessment.19,20 Further, to support skills development, methods for assessment of LISA competence are needed to ensure high and consistent performance and to enable future mastery learning of the LISA procedure, where trainees practice with feedback until they achieve a predefined mastery learning level.21,22 This study aimed to provide international expert consensus on (1) the training curriculum for LISA operators and (2) what to include in the assessment of LISA competence.

Methods

Study design

From February to July 2022, we conducted a three-round iterative Delphi process using online survey questionnaires to collect information and establish consensus regarding the content to be included in (1) a LISA curriculum (LISA-CUR) and (2) a LISA assessment tool (LISA-AT) for the assessment of LISA competence. The Delphi process is a systematic, group facilitation technique using structured and semi-structured questionnaires to collect and organize the opinions from experts in the field until consensus is reached.23 The survey questionnaires for Delphi rounds 1–3 are available from the corresponding author. We defined consensus in rounds 2 and 3 as an agreement of ≥80%. Figure 1 presents an overview of the process.

Fig. 1: Overview of the three-round Delphi process from February to July 2022.
figure 1

All experts who participated in the first round were invited to participate in the following rounds. LISA less invasive surfactant administration, LISA-CUR LISA curriculum, LISA-AT LISA assessment tool.

Full size image

Data collection

We used the Research Electronic Data Capture (REDCap) system for data collection. Chinese experts could not access REDCap from mainland China and submitted their replies by e-mail. This information was subsequently entered into REDCap by the principal investigator (N.B.).

Sample characteristics

The LISA experts were identified based on their involvement in research, curriculum development, and clinical education in the LISA procedure.

Invitations to participate were sent via e-mail by the steering committee members using their professional networks. The initial invitation to participate was promoted by the European Society for Paediatric Research (ESPR) via its website, on social media, and in its newsletter. The invitation included a link to the online consent form and the survey questionnaire for round one. Participation was voluntary. All experts who participated in the first round were invited to participate in the following rounds.

Survey administration

A Delphi steering committee was established to facilitate and manage all the steps in the Delphi process, including recruitment of experts and data analyses. The steering committee included an international team of 16 members with experience in neonatal intensive care or postgraduate medical education. All steering committee members except N.B. and M.G.T. were invited to participate in the survey due to their roles as international experts on the LISA procedure. The survey was developed by N.B. and piloted by the steering committee before the data collection in each round.

Delphi round 1

The first round was a brainstorming phase where experts replied to the following statements: (1) “List, in free text, the specific knowledge and skills that you believe a newly certified LISA operator needs to have and should be included in a curriculum for LISA training”, and (2) “List, in free text, all relevant aspects of the LISA procedure to be included in an assessment tool to evaluate competence/certification of a LISA operator.”

Experts provided baseline information, including sex, age, specialty, and years of experience with the LISA procedure and answered questions concerning local LISA procedure practice and use of training equipment.

Data were gathered by N.B. and processed by the steering committee. Duplicates were removed. Teaching activities, learning-teaching strategies, and teaching methods were omitted as they were already described in other publications.15 The remaining items were analyzed using content analysis24 to identify repeated categories of procedures and various textual expressions for each unique item to consolidate the lists for the LISA-CUR and the LISA-AT used in round 2. Due to variations in phraseology in round 1, the steering committee was allowed to adjust the wording but not the content of each suggested item. Decisions were based on 100% agreement within the steering committee.

Delphi round 2

In this round, items generated in the Delphi round 1 were assessed for suitability for the LISA-CUR and the LISA-AT. The selection was based on importance by using a five-point Likert scale; (1) Not important at all, (2) Less important, (3) Neutral, (4) Important, (5) Very important. The experts replied to the following statements individually for each item: (1) “How important are the following items when learning how to perform the LISA procedure?” and (2) “How important are the following items when assessing the skill of an operator performing the LISA procedure?”

Items rated as important (4) or very important (5) by 80% or more of the experts were included in the preliminary LISA-CUR and LISA-AT that were sent back to the experts for final review and comments in Delphi round 3.

Delphi round 3

The experts were informed about the distribution of scores, the excluded items, and selected comments from other experts in round 2. Subsequently, they were asked to approve or reject and comment on the preliminary LISA-CUR and LISA-AT.

Ethical considerations

Ethical approval was waived by the Committee on Health Research Ethics in the Capital Region of Denmark (Journal number: 21051793). Experts were informed about the aims of this study, the importance of participation, and the detailed tasks in each step of the Delphi process. All experts provided electronic informed consent prior to study participation. The datasets were de-identified prior to analyses. All information was handled confidentially. Data management and processing were approved (ID-number: P-2022-11). This study is reported according to the Consensus-Based Checklist for Reporting of Survey Studies (CROSS)25 (Online Supplement, Appendix A).

Statistical analysis

Categorical data from Delphi round 1 are presented as frequencies (counts and percentages) and continuous data as medians with interquartile ranges [IQRs] and range as appropriate. Statistical analyses were performed using RStudio version 1.2.5001. There were no changes to study design after this study commenced. There was no imputation of missing data. The required response rate for each round was ≥70% to minimize non-response error.

Results

Expert demographics

One hundred fifty-three experts agreed to participate in this study representing 14 countries worldwide (Table 1). Experts were involved in research (48%), curriculum development (56%), and clinical education (87%) regarding the LISA procedure, and the median [IQR] time of involvement in the LISA procedure was 5 [4–8] years. Most were board-certified neonatologists (97%), 16% in pediatrics, 1% in anesthesia, and 1% in epidemiology. More than half the experts (54%) worked at hospitals with more than 10 individual LISA operators. According to the experts, 37% of LISA operators in their respective units performed less than one procedure per month, and 52% performed 1–5 procedures per month (Table 1).

Table 1 Experts’ baseline information.
Full size table

The LISA procedure

Based on the experts’ replies, median gestational age of infants considered eligible for treatment with the LISA procedure was 28 weeks (IQR 24–32 weeks, range 22–42 weeks) (Table 2). nCPAP was the most frequently reported mode of respiratory support used during LISA (84%), followed by NIPPV (46%) and HFNC (9%).

Table 2 Experts’ report of therapies applied during LISA.
Full size table

Forty-one per cent of experts reported no use of pre-procedure sedatives or analgesics, whereas 49% indicated the use of fentanyl as pre-procedure treatment (Table 2). Use of non-pharmacological measures was reported by 96% of the experts: 58% used swaddling/wrapping, 48% used sucrose, 46% used tucking and other interventions like skin-to-skin care, light and noise reduction, positioning, and pre-procedure non-nutritive suctioning.

Main findings

The final LISA-CUR and the LISA-AT, including response anchors for guided assessments, are available in Tables 3 and 4, respectively.

Table 3 The final approved LISA curriculum (LISA-CUR).
Full size table
Table 4 The final approved LISA assessment tool (LISA-AT) with response anchors one, three, and five after Delphi round 3.
Full size table

Delphi round 1

The experts suggested 807 and 658 items for the LISA-CUR and the LISA-AT, respectively.

For the LISA-CUR, 148 and 498 of the 807 items were excluded being duplicates or according to the following criteria: teaching activities, learning-teaching strategies, and teaching methods. The remaining 159 unique items were further condensed to 44 items and post hoc organized into the following nine categories to achieve a better overview: (1) Indications and contraindications, (2) Complications, (3) Familiarity with the equipment, (4) Drugs and non-pharmacological measures, (5) Airway management, (6) Respiratory support, (7) Monitoring and assessment, (8) Evaluation, (9) Other skills (Online Supplement, Appendix B).

For the LISA-AT, 256 and 314 of the 658 items were excluded for the same reasons as those listed for the LISA-CUR. The steering committee further condensed the remaining 88 unique items to 22 and post hoc sorted all items chronologically in descending order from procedure start to finish (Online Supplement, Appendix C). These two lists were used in Delphi round 2 and returned to the experts for assessment.

The list of training equipment used to practice LISA is available in the Online Supplement, Appendix D.

Delphi round 2

The response rate was 82% (126/153 experts).

For the LISA-CUR, 12 items were eliminated by consensus (consensus score <80%, Online Supplement, Appendix B). Between rounds 2 and 3, the steering committee decided to merge three of the included items based on the experts’ comments resulting in a preliminary LISA-CUR of 29 items.

For the LISA-AT, eight items were eliminated by consensus (Online Supplement, Appendix C), resulting in a preliminary LISA-AT of 14 items. One item, “Surfactant administration synchronized with the patient’s inspiration with a closed mouth”, was rephrased by the steering committee to “Surfactant administration: Slow infusion to allow surfactant to be inhaled by the infant” based on the comments received. Round 2 was repeated for this item alone.

The steering committee assigned specific response anchors on a five-point Likert scale to each item included in the LISA-AT based on the experts’ suggestions. The steering committee provided precise guidance on the “poor,” the “sufficient,” and the “excellent” performance (response anchors 1, 3, and 5, respectively) for each item to hopefully increase inter-rater reliability and provide more standardized assessments without restricting the use of the LISA-AT, as operators may perform very differently.

The complete overview of adjustments to the LISA-CUR and LISA-AT between rounds 2 and 3 is available in the Online Supplement, Appendices E and F, respectively.

Delphi round 3

The response rate in Delphi round 3 was 83% (127/153 experts). The LISA-CUR and LISA-AT achieved consensus without changes (100% and 98%, respectively). Furthermore, the rephrased item was included in the final assessment tool as 86% of the experts rated the item 4 (important) or 5 (very important).

Discussion

In this Delphi process, 153 LISA experts from 14 countries were involved in a three-round iterative process to gather consensus regarding the training and assessment of LISA operators. Together, we developed the LISA-CUR consisting of 29 unique items, and the LISA-AT consisting of 15 unique items with response anchors. There are several benefits of implementing the LISA-CUR and LISA-AT in standardized training after adaptation to the local setting to increase operator competence and patient safety.

Until recently, a lack of formalized training18 has been a barrier to the implementation of LISA, as confirmed by experts’ in our study. Furthermore, the majority of the LISA experts in our study only performed 0–5 LISA procedures per month. Lack of formal training and infrequent clinical exposure to the LISA procedure may challenge adherence to clinical guidelines.15,26

A combination of theoretical aspects with practical training for learning the LISA procedure is recommended.14 So far, a consensus on a LISA training curriculum or assessment tool to evaluate operator competence has yet to be published. In 2021, Liebers et al.17 developed a LISA training program including multimedia materials, checklists, pocket cards, and team briefing, which increased the mean success rate of LISA from 62% to 92%, defined as no need for any of the following: intubation within the following 72 h, a second surfactant dose, or termination of the LISA procedure. In 2019, Vento et al.15 published a practical recommendation for surfactant administration specifying the characteristics of a trainer and a trainee and how training should be structured. Three training modules for operators and assistants were proposed, including (1) pre-course preparation and online test, (2) LISA training on a mannequin, and (3) post-course LISA consolidation by performing several procedures in the clinical setting with supervision combined with periodic refreshers on a mannequin. Recommendations by Vento et al.15 and Reynolds et al.27 suggest treatment thresholds, exclusion criteria, personnel, monitoring, equipment, non-pharmacological measures, and analgesics, including doses. The LISA-CUR may work with these existing recommendations to optimize and standardize LISA training in the future.

The consensus-based LISA-AT consists of 15 unique items and may provide standardized feedback, assessment, and potential certification of LISA operators. Vento et al. (2019)15 and Reynolds et al. (2021)27 also identified core items that an operator should be able to undertake, but these reflected the individual opinions of the clinicians involved in formulating the recommendations. All their core items were included in the LISA-AT we propose and are now supported by international expert consensus. Various techniques for the LISA procedure exist,15 which may explain the lack of consensus achieved for various procedure-specific items like catheter type, Magill forceps, checking for gastric retention, and lung ultrasound, which were excluded from the final LISA-CUR and LISA-AT. These procedural factors clearly should be a focus of future research. Local protocols for each procedure step must exist before the LISA-CUR and LISA-AT can be adapted to the local setting and successfully implemented. Since this is first of a kind initiative, opinions may be inconsistent and further validation of the tool will help to refine it.

Development of the LISA-AT is the first step to enable mastery learning through standardized, continuous feedback and assessment until a predefined performance criterion is achieved.

However, there is currently no validity evidence to support the use of the LISA-AT and its ability to differentiate between operators with different levels of experience.21 To our knowledge, there is no pre-defined mastery learning level, and we do not know how much practice is needed during simulation-based LISA training to reach sufficient levels of mastery.9,28 Future studies need to validate the LISA-AT using Messick’s framework,29 which is the gold standard when evaluating validity as the American Educational Research Association recommended in their Standards for Education and Psychological Testing.30,31 Following validation, NICUs and neonatal simulation centers may standardize LISA training and assessment by implementing the LISA-CUR and LISA-AT. This should be a necessary step before the operator is promoted from training in the simulated environment to training in the clinical setting.

The Delphi research method is ideal for studying which items to include in the LISA-CUR and the LISA-AT when objective information is unavailable.32 We minimized the risk of conflict of interest and group pressure frequently associated with expert panels by anonymizing all responses prior to analyses and providing a unique link for each expert per round.32,33 This study was conducted following a detailed protocol (available from NB upon request) with no changes to the study design after the initiation of round 1, limiting individual members’ influence through all three rounds. Hence, only items achieving the predefined consensus cut-off were included in the final LISA-CUR and LISA-AT. Due to the online advertisement through the ESPR website, we are unaware of the total number of potential experts who might have seen the invitation. We are also unaware of the actual number of participants recruited by the online advertisement and the reasons for non-participation at each stage. However, we minimized the risk of bias from lack of response by achieving response rates of more than 80% of the initial responders in Delphi rounds two and three. Multiple NICUs contributed data from each country, minimizing the risk of bias arising from multiple responses from a single institution. No single country was represented by more than 15% of all the experts in our study. We chose a broad set of inclusion criteria for LISA experts to gather as many inputs as possible. Although this might have diluted the qualification of being an “expert,” baseline data showed that 97% of experts were board-certified neonatologists involved with the LISA procedure for several years. The experts involved in this study represented highly specialized NICUs from Western and non-Western countries. However, some NICUs may not have the resources to improve training and assessment for the LISA procedure, as this procedure is just one part of a resource-demanding intensive care strategy for preterm infants.

Conclusion

This Delphi process generated international consensus on a LISA curriculum (LISA-CUR) and a LISA assessment tool (LISA-AT). The LISA-CUR and LISA-AT are published with this study and may be used by LISA trainers, supervisors, and curriculum developers to guide their efforts.