key: cord-0718823-o9q3edpc
authors: Stentz, Michael J.; Wiepking, Matthew D.; Hodge, Kiley A.; Ramonell, Richard P.; Jabaley, Craig S.
title: Checklists Improve Team Performance During Simulated Extracorporeal Membrane Oxygenation Emergencies: A Randomized Trial
date: 2021-04-26
journal: Crit Care Explor
DOI: 10.1097/cce.0000000000000404
sha: 49e1fa546d4349f151d0250b5ebea8d28b5b7f5a
doc_id: 718823
cord_uid: o9q3edpc

OBJECTIVES: Extracorporeal membrane oxygenation–related complications are potentially catastrophic if not addressed quickly. Because complications are rare, high-fidelity simulation is recommended as part of the training regimen for extracorporeal membrane oxygenation specialists. We hypothesized that the use of standardized checklists would improve team performance during simulated extracorporeal membrane oxygenation emergencies. DESIGN: Randomized simulation-based trial. SETTING: A quaternary-care academic hospital with a regional extracorporeal membrane oxygenation referral program. SUBJECTS: Extracorporeal membrane oxygenation specialists and other healthcare providers. INTERVENTIONS: We designed six read-do checklists for use during extracorporeal membrane oxygenation emergencies using a modified Delphi process. Teams of two to three providers were randomized to receive the checklists or not. All teams then completed four simulated extracorporeal membrane oxygenation emergencies. MEASUREMENTS AND MAIN RESULTS: Simulation sessions were video-recorded, and the number of critical tasks performed and time-to-completion were compared between groups. A survey instrument was administered before and after simulations to assess participants’ attitudes toward the simulations and checklists. We recruited 36 subjects from a single institution, randomly assigned to 15 groups. The groups with checklists completed more critical tasks than participants in the control groups (90% vs 75%; p < 0.001). The groups with checklists performed a higher proportion of both nontechnical tasks (71% vs 44%; p < 0.001) and extracorporeal membrane oxygenation–specific technical tasks (94% vs 86%; p < 0.001). Both groups reported an increase in reported self-efficacy after the simulations (p = 0.003). After adjusting for multiple comparisons, none of the time-to-completion measures achieved statistical significance. CONCLUSIONS: The use of checklists resulted in better team performance during simulated extracorporeal membrane oxygenation emergencies. As extracorporeal membrane oxygenation use continues to expand, checklists may be an attractive low-cost intervention for centers looking to reduce errors and improve response to crisis situations.

www.ccejournal.org April 2021 • Volume 3 • Number 4 E xtracorporeal membrane oxygenation (ECMO) therapy is used to support critically ill patients who are failing traditional life support (1) (2) (3) (4) (5) . Unfortunately, ECMO therapy also involves significant risk, including mechanical failure of the ECMO circuit, dislodgement of the cannulae, and catastrophic bleeding secondary to coagulopathies and anticoagulants (6) (7) (8) (9) (10) . Management of patients on ECMO requires a nuanced understanding of physiology and patientmachine interactions to correctly diagnose and correct abnormalities. Because of the high-risk and complex care required by these patients, most ECMO centers use trained ECMO specialists to provide some or all of the bedside care to patients on ECMO (6, (11) (12) (13) .

The rapid expansion of ECMO use in the past decade and the ongoing severe acute respiratory syndrome coronavirus 2 pandemic (14) (15) (16) have created a need for more ECMO specialists, with an increasing number of those specialists practicing at centers with limited previous experience caring for ECMO patients (10, (17) (18) (19) . Because ECMO emergencies are rare but potentially fatal, high-fidelity simulation has been frequently used to provide training for such situations (20) (21) (22) (23) (24) .

The use of checklists has been shown to improve adherence to process measures, such as the proportion of steps completed or time-to-completion, in both clinical and simulated scenarios (25) (26) (27) . Although crisis resource management checklists have been implemented in many healthcare systems (28, 29) , no such well-established resource exists for ECMO emergencies.

We set out to create a set of read-do checklists (28, 30) for use during ECMO emergencies by any bedside provider to immediately stabilize a patient until the underlying problem is resolved or a specialized team arrives. We hypothesized that the use of these checklists during emergencies in a simulated clinical environment would increase the likelihood of prespecified critical tasks being performed, result in lower time-to-completion of critical tasks during simulated ECMO emergencies, and increase the reported self-efficacy of participants.

This study was carried out at a single large academic hospital in the United States (Emory University Hospital, Atlanta, GA) with a regional ECMO referral program established in 2014 (11) . At the time of this study, our ECMO center was performing approximately 60 ECMO cases annually, with a roughly even distribution of cases between veno-arterial and veno-venous ECMO. All ECMO patients are cared for in a single cardiothoracic surgical ICU. Our ECMO specialists are either registered nurses (RNs) or respiratory therapists (RTs) with experience in critical care who have completed additional class work and hands-on education in ECMO management. Additionally, we recruited volunteers from among the critical care fellows and advanced practice providers (APP) as their clinical schedules allowed. Informed consent was obtained from all participants prior to proceeding, and each specialist was given the option to opt-out of the investigation while still completing their mandatory annual recertification. All participants had previously completed ECMO-related "water drills, " in which participants perform technical tasks with a practice ECMO circuit (e.g., exchanging an oxygenator). Water drills do not typically involve simulated patients or clinical scenarios; all participants reported that they had not previously participated in ECMO-specific simulation training.

We obtained approval from the Emory University Institutional Review Board (IRB00106018) prior to recruiting any subjects. Results are reported according to the simulation-based research extensions of the Consolidated Standards of Reporting Trials guidelines (31) . Subjects completed simulated ECMO emergency scenarios as outlined below in teams of two or three between December 2018 and April 2019. Participants were grouped into teams to mimic routine clinical practice in which several individuals would respond to an ECMO emergency. After informed consent was obtained, teams were randomized to receive printed copies of relevant checklists (intervention group) or not (control group) based on computer-generated randomization blocks. Participants were also asked to complete a brief survey instrument before and after the simulations.

Local clinical experts were identified, and a modified Delphi method was used to create checklists for six emergency scenarios identified through clinical experience, prior research, and internal review of adverse events (20, 21) . The six scenarios chosen were air entrainment, console failure, major bleeding at the cannulation site, inadequate preload or "chugging, " hypoxemia, and inadvertent decannulation (32). Following consensus agreement, the checklists were finalized and then designed to fit on a single page, with clearly color-coded sections for "Signs, " "Immediate Action, " and "Next Steps. " The checklists used are shown in e-Appendix A (http://links.lww.com/CCX/A594).

We created a short survey instrument to measure subjects' perceived self-efficacy in the management of routine ECMO patients and life-threatening emergencies. Participants were asked to complete the same survey questions before and after completing the simulation scenarios. The postsimulation survey included questions regarding participants' attitudes toward the simulations; additionally, the intervention arm also received a short survey regarding the checklists.

From the six emergency scenarios for which checklists were created, four simulation scenarios were developed for testing (air entrainment, console failure, hypoxemia, inadvertent decannulation). We intentionally created fewer scenarios than checklists, so that participants in the intervention arm would not be able to deduce which scenario would come next. Written scripts for these scenarios were created, with expected critical actions from the subjects and prespecified events based on possible actions. Our ECMO program uses the CardioHelp system (Getinge, Gothenburg, Sweden), which includes a combined pump oxygenator and integrated air detectors on the inflow and outflow circuit limbs, and all scenarios were designed around these technical assumptions to mimic our clinical practice.

All scenarios used a low-fidelity mannequin developed by the authors specifically for ECMO simulation. The main body of the mannequin was made of cloth and concealed two 3/8′′ perfusion catheters running from the right thigh to a priming reservoir. Hoffman's clamps on the perfusion catheters could be used to modify resistance to flow. The circuit was also modified with a luer lock connector that could be used to introduce air into the inflow limb of the circuit. A CardioHelp Heart Lung Support set was primed with saline, and a 25F Bio-Medicus cannula (Medtronic, Dublin, Ireland) and an 18F Fem-Flex 2 cannula (Edwards Lifesciences, Irvine, CA) were attached to the inflow and outflow limbs of the circuit, respectively. These were then inserted into the perfusion tubing concealed within the mannequin, and the seals were made water-tight using adhesive film. The cannulae were then dressed with the standard transparent dressings used in our institution, and their depths marked according our ICU protocol. This allowed teams to inspect the cannulation site (right groin) and cannula positioning; other physical examination findings were not present due to the low-fidelity nature of the mannequin. Vital signs were displayed using a tablet computer running the SimMon software (Castle + Anderson ApS, Hillerød, Denmark), configured to appear similar to our standard ICU monitors. This app allows vital signs to be manipulated remotely, and scripted changes to the vital signs were preprogrammed for each scenario. The CardioHelp console displayed relevant ECMO variables, including flow, pump speed, and line pressures. Because venous oxygen saturation (Svo 2 ) could not be simulated directly on the CardioHelp console, this value was covered with tape on the console. A free-text number was added to the vitals display and labeled as Svo 2 ; this number could be manipulated by the instructors during simulations.

All subjects were given a standardized orientation to the simulator, including instructions regarding the vital signs and Svo 2 display, as well as explicit instructions that they could call for help or ask for other information if not present in the simulation scenario. Subjects in the intervention arm were given an orientation to the checklists, including an explanation of the formatting ("Signs, " "Immediate Actions, " and "Next Steps") and instructions that they were encouraged to use the checklists during the scenarios. Teams were given approximately 5 minutes to review the checklists prior to the first simulation scenario.

At the start of each scenario, the teams were given printed copies of the standardized hand-off scripts used for all ECMO patients at our center with information about the simulated patient filled in as appropriate. They were then asked to perform a standard ECMO circuit check and to go through their shiftchange hand-off routine as usual. As this was being done, the instructors (M.J.S., K.A.H.) would begin the appropriate clinical emergency scenario as indicated in the scenario scripts.

All simulation sessions were video-recorded, and only those with a complete video record were included in the analyses. For each scenario, the investigators identified six critical actions that teams should perform. All scenarios included two nontechnical tasks www.ccejournal.org April 2021 • Volume 3 • Number 4 (calling for help and addressing patient vitals) and four technical tasks specific to that ECMO emergency. One investigator (M.J.S.) reviewed these videos after the simulation sessions were complete to determine whether each critical task had been completed and the time-to-completion for each task. Because checklists could be seen in the videos, it was not possible to blind the coder to the assigned treatment group. After the appropriate endpoint was reached for each scenario, the instructor would indicate that the scenario had been successfully completed. Each scenario was followed by a debrief by the instructors (M.J.S., K.A.H.), based on the "Setting learning objectives, How did it go, Address concerns, Review learning points, Plan ahead" debriefing tool (33) . The feedback sessions were conversational and were not scripted. All feedback facilitators were also involved in the design of the study, so no additional training for other facilitators was required. This time was also used to answer any questions the subjects had and to allow time for each team member to have hands-on practice of technical skills (e.g., disconnecting the pump oxygenator from the console). There was no time limit to the feedback sessions. Two hours were allotted for each group to perform the four scenarios, and all groups finished within the allotted time without difficulty. All groups completed the four scenarios in the same order, with feedback sessions after each scenario. After the completion of the final simulation scenario and feedback session, all subjects were again given a survey assessing their perceived self-efficacy, their attitudes toward the simulations, and for the intervention group, their attitudes toward the checklists.

We compared the number of critical tasks completed by each arm of the study using Pearson's chi-square test. The times-to-completion were not normally distributed and were compared using the Mann-Whitney U test, with a Bonferroni correction for multiple comparisons. For groups that did not complete a required task, the time-to-completion was recorded as the total length of that simulation scenario. Self-efficacy was compared between groups using the Mann-Whitney U test. Statistics were performed in Microsoft Excel (Microsoft Corp., Redmond, WA) and R (R Foundation for Statistical Computing, Vienna, Austria).

We enrolled 36 subjects, with 18 in the intervention group and 18 in the control group. Despite random assignment of groups to intervention or control, there was a statistically significant difference in the composition of the two study arms, with more RN in the intervention group and more RT in the control group (Fisher test p = 0.033). There was no difference between intervention and control groups in the years of clinical critical care experience (3.5 [interquartile range (IQR), 2-8.5] vs 6 [IQR, ; p = 0.352). Subjects were grouped into eight teams in the intervention arm and seven teams in the control arm. Due to technical issues, a complete video record was not available for four of the 60 scenarios performed, and these were excluded from analysis (Fig. 1) .

Across all simulation scenarios with complete video recordings, the intervention group performed 168 of a possible 186 critical tasks, whereas the control group performed 112 of a possible 150 critical tasks (90% vs 75%; p < 0.001) (Fig. 2) . The groups with checklists were much more likely to complete the nontechnical critical tasks of calling for help (71% vs 44%; p < 0.001) and addressing the patient's vital signs (94% vs 60%; p < 0.001). The groups with checklists also completed significantly more of the technical ECMO-related critical tasks (94% vs 86%; p < 0.001).

Because of the unanticipated imbalance in the composition of the study arms, we performed a post hoc analysis comparing RT only groups, RN only groups, mixed composition groups, and APP/fellow only groups. When compared with other groups in the same treatment arm, group composition did not significantly affect the number of critical tasks completed. We also performed a post hoc analysis comparing groups with two members and three members. In the intervention arm, two-and three-member teams performed similarly; in the control arm, the three-member teams completed more critical tasks than the two-member teams (p = 0.037). These data are summarized in Table 1 .

After adjusting for multiple comparisons, none of the time-to-completion measures achieved statistical significance. The intervention group with checklists generally performed required critical tasks faster than the control group, with a few exceptions; these data are shown in Figure 3 .

In the intervention group, 16 of 18 subjects completed the postsimulation survey regarding the checklists. Of those responding, 15 of 16 subjects found the checklists to be "helpful" or "very helpful, " and all subjects reported that the checklists were "very easy to understand. " The majority of respondents (15/16) reported that they would be "likely" or "very likely" to use the checklists if they were available at the bedside. Both groups reported an increase in self-efficacy for dealing with life-threatening emergencies in ECMO patients (p = 0.003), and there were no significant differences between intervention or control groups in reported self-efficacy (Fig. 4) .

Narrative feedback regarding the checklists was mostly positive. Three respondents wrote in comments that the checklists should be made available at the bedside. Two respondents suggested that they should be reformatted or revised, either as a "structured checklist" or made "more succinct. " One respondent selfreflected that the aides may not be "referred to in a 'life-threatening' scenario. "

This study of process adherence during simulated ECMO emergency responses demonstrated more frequent performance of critical tasks with the use of checklists. These critical tasks included both scenario-specific technical tasks (e.g., checking sweep gas connections in a hypoxic patient) and general nontechnical tasks (e.g., calling for help, situational awareness of vital signs). The group with checklists reported overwhelmingly that the checklists were helpful and that they would be likely to use the checklists during a true emergency. To our knowledge, this is the first study to examine checklists specifically designed for the management of ECMO emergencies. Checklists have previously been created for emergent ECMO cannulation of pediatric patients (22, 23, 34) and have been used to evaluate teams participating in simulated ECMO emergencies (22) . Our study incorporated checklists designed for use at the bedside by ECMO specialists or other providers, and their usefulness was validated here using simulated scenarios. Given that these emergencies are rare and unpredictable, we did not evaluate whether the improved performance during simulations translates into better clinical care.

Checklists and other decision support tools have received growing attention in medicine over the past decade as a means to reduce avoidable medical errors. Checklists have been shown to improve clinical outcomes in a wide variety of situations (35, 36) although they may not be useful in every clinical situation (37, 38) . At the same time, some providers have resisted the implementation Post hoc analysis showed that three-subject teams in the control arm completed significantly more tasks than two-subject teams in the control arm ( a p = 0.037). All other comparisons were nonsignificant at α = 0.05.

of checklists for various reasons; for example, providers who perceive the checklists as redundant or unhelpful are less likely to use them (39) . Although most participants in our study reported that the checklists were helpful, a few participants thought the checklists should be restructured. One subject noted that they were less likely to use the checklists in a "'life-threatening' scenario, " suggesting that better checklist-specific training and thoughtful planning would be needed to successfully translate their implementation into routine clinical practice. Calling for help early, situational awareness, and the use of checklists are key aspects of crisis resource management (40) . In our simulations, use of checklists during simulations resulted in earlier and more frequent calls for help, as well as more frequent recognition of changes in the patient's vital signs. We suggest that this effect may be mediated by the correcting of common cognitive errors, such as overconfidence bias or fixation errors, although further study would be needed to test this hypothesis (41) .

The simulated emergencies used in this study required a combination of cognitive and physical interventions. For many of the cognitive interventions, such as working through the differential diagnosis for hypoxia, the group with checklists performed better than the control groups in completeness of their evaluation (such as ruling out a sweep line disconnection), while arriving at the "correct" answer only a few seconds behind the control group. For rote mechanical interventions, such as changing a pump oxygenator to a hand-crank or clamping the circuit following an inadvertent decannulation, the two groups were able to accomplish the task at similar proportions and in similar amounts of time. This finding is not surprising, as the skills required for these mechanical tasks are largely technical and were not described in detail in the checklists. However, this finding should be used to guide future work in ECMO safety, as checklists may be useful in some situations but ineffective in others. Additionally, our checklists were developed by content experts from our institution and are specific to our ECMO circuit configuration as well as the composition of our ECMO care team; they may not be universally applicable to other centers. Likewise, although our post hoc analysis showed a difference in performance between two-and three-member teams without checklists, the current study was not designed to examine interactions between provider experience or team size and performance; such factors should be considered when implementing these checklists clinically.

The use of checklists was not without trade-offs. The groups using the checklists took slightly longer to perform some tasks, such as restarting ECMO after a venous bubble was detected. Further research is needed to determine whether future iterations of these checklists, perhaps using a challenge-response format, would decrease the amount of time needed to perform the required tasks. This study was primarily intended as an initial validation of the checklists; further study would be needed when implementing the checklists into a clinical setting.

Subjects using the checklists during simulated ECMO emergencies were more likely to perform critical steps in a timely fashion and reported that the checklists were helpful. As ECMO use continues to expand, checklists may be an attractive low-cost intervention for centers looking to reduce errors and improve response to crisis situations.

CESAR trial collaboration: Efficacy and economic assessment of conventional ventilatory support versus extracorporeal membrane oxygenation for severe adult respiratory failure (CESAR): A multicentre randomised controlled trial

Extracorporeal membrane oxygenation for severe acute respiratory distress syndrome

Extracorporeal membrane oxygenation for 2009 influenza A (H1N1)-associated acute respiratory distress syndrome

Refractory cardiac arrest treated with mechanical CPR, hypothermia, ECMO and early reperfusion (the CHEER trial)

The 2CHEER study: (Mechanical CPR, hypothermia, ECMO and early re-perfusion) for refractory cardiac arrest

Systematic review and meta-analysis of complications and mortality of veno-venous extracorporeal membrane oxygenation for refractory acute respiratory distress syndrome

Review of ECMO (extra corporeal membrane oxygenation) support in critically ill adult patients

Extracorporeal life support registry report

ELSO Registry: Extracorporeal Life Support Organization Registry Report

ELSO member centers: Extracorporeal life support organization registry international report

Rapid development and implementation of an ECMO program

A cost-reducing extracorporeal membrane oxygenation (ECMO) program model: A single institution experience

An international survey: The role of specialist nurses in adult respiratory extracorporeal membrane oxygenation

Veno-venous extracorporeal membrane oxygenation support in COVID-19 respiratory distress syndrome: Initial experience

Extracorporeal membrane oxygenation for patients with COVID-19 in severe respiratory failure

Extracorporeal membrane oxygenation support in COVID-19: an international cohort study of the Extracorporeal Life Support Organization registry

Trends in extracorporeal membrane oxygenation growth in the United States

Extracorporeal membrane oxygenation (ECMO): Does it have a role in the treatment of severe COVID-19?

Association of hospital-level volume of extracorporeal membrane oxygenation cases and mortality. Analysis of the extracorporeal life support organization registry

The introduction of a high-fidelity simulation program for training pediatric critical care personnel reduces the times to manage extracorporeal membrane oxygenation emergencies and improves teamwork

Simulating extracorporeal membrane oxygenation emergencies to improve human performance. Part II: Assessment of technical and behavioral skills

Simulation-based assessment of ECMO clinical specialists

Implementation of an extracorporeal cardiopulmonary resuscitation simulation program reduces extracorporeal cardiopulmonary resuscitation times in real patients

Impact of an open-chest extracorporeal membrane oxygenation model for in situ simulated team training: A pilot study

A simulation-based trial of surgicalcrisis checklists

Crisis checklists for the operating room: Development and pilot testing

Checklists and multidisciplinary team performance during simulated obstetric hemorrhage

Emergency manual implementation in a large academic anesthesia practice: Strategy and improvement in performance on critical steps

Operating room crisis checklists and emergency manuals

Review article: Perioperative checklist methodologies

International Network for Simulation-based Pediatric Innovation, Research, and Education (INSPIRE) Reporting Guidelines Investigators: Reporting guidelines for health care simulation research: Extensions to the CONSORT and STROBE statements

Research guidelines for the Delphi survey technique

Operation debrief: A SHARP improvement in performance feedback in the operating room

Impact of simulation-based extracorporeal membrane oxygenation training in the simulation laboratory and clinical environment

An intervention to decrease catheter-related bloodstream infections in the ICU

Improving quality of care for maternal and newborn health: Prospective pilot study of the WHO safe childbirth checklist program

Association of checklist use in endotracheal intubation with clinically important www

Introduction of surgical safety checklists in Ontario, Canada

Barriers to staff adoption of a surgical safety checklist

Crisis management during anaesthesia: The development of an anaesthetic crisis management manual

Cognitive errors detected in anaesthesiology: A literature review and pilot study