key: cord-0953016-oueq78nl authors: Mavis, Stephanie C.; Kreofsky, Beth L.; Ouk, Melody Y.; Carey, William A.; Fang, Jennifer L. title: Training fellows in neonatal tele-resuscitation using a simulation-based mastery learning model() date: 2021-10-08 journal: Resusc Plus DOI: 10.1016/j.resplu.2021.100172 sha: 72e6a14adbd98984a80759c610b5e8714fa9d24e doc_id: 953016 cord_uid: oueq78nl BACKGROUND: Neonatal tele-resuscitation uses real-time, audio–video telemedicine to connect neonatologists with community hospital care teams during advanced neonatal resuscitations. While telemedicine continues to expand, best practices for training fellows in tele-resuscitation are not known. OBJECTIVE: We aimed to develop a neonatal tele-resuscitation curriculum using a simulation-based mastery learning model that provides neonatal-perinatal medicine (NPM) fellows with the knowledge, skills, and behaviors needed to lead tele-resuscitations. METHODS: Using technology-enhanced simulation education and a mastery learning model, we developed a longitudinal pilot tele-resuscitation curriculum. From 07/2018 to 03/2021, NPM fellows participated in the curriculum, which included individualized telemedicine learning, observing and leading simulated tele-resuscitations, and finally, performing clinical (non-simulated) tele-resuscitations. A performance assessment tool was developed to assess competency through eight questions mapped to the Accreditation Council for Graduate Medical Education (ACGME) core competencies, with responses on a 1 to 5 scale (1 = critical deficiencies; 5 = competence of an expert). RESULTS: Four NPM fellows participated in the curriculum, progressing through the curriculum at an individualized pace. Median scores on the three learning modules were 96–100%. Fellows participated in variable number of simulated tele-resuscitations based on when mastery was achieved (2–3 supervised simulations per fellow, 1–4 unsupervised simulations per fellow). In total, eighteen simulated tele-resuscitations (eight unsupervised, 10 supervised) and one clinical tele-resuscitation were conducted. Twenty-five performance assessments were completed. Assessment scores across the ACGME competencies were consistently high, with mean scores ranging from 4.2-4.6, with 4 equating to ‘ready for unsupervised practice’ and 5 equating to ‘competence of an expert’. CONCLUSIONS: As telemedicine use continues to expand, curricula that improve learners’ comfort with and proficiency in tele-resuscitation are essential. A simulation-based mastery learning model may be one approach that affords learners gradual exposure to and mastery of complex tele-resuscitation skills and behaviors. In response to the COVID-19 pandemic, the Accreditation Council for Graduate Medical Education (ACGME) implemented program requirements permitting residents and fellows to use telemedicine to care for patients 1 . Despite differences between virtual and in-person care 2,3 , limited evidence exists on how to effectively train fellows in telemedicine and assess their performance. Specialtyspecific milestones and competencies defining the knowledge, skills, and attitudes required to provide high quality telehealth care are in their infancy 4-6 . In the United States, neonatology is one of the top five specialty services in pediatric telehealth 7 q Prior poster presentation: Parts of this curriculum were presented at the oral presentation entitled "Teachers, Students, and How to Learn in an Acute Care Telemedicine Environment" at the 2019 National Telehealth Research Symposium, Chicago, IL, Sept 11-13, 2019. time, audio-video telemedicine service, connects remote neonatologists with community hospital care teams during advanced neonatal resuscitations. Neonatal tele-resuscitation improves the quality of resuscitation 8 and reduces patient transfers to the neonatal intensive care unit (NICU) 9, 10 . To successfully lead a team remotely, the consulting neonatologist must be knowledgeable in advanced neonatal resuscitation and have the necessary telemedicine technical and communication skills. Best practices for training neonatalperinatal fellows in tele-resuscitation and assessing their performance have not been described. To address these teaching and learning gaps, we developed and studied an innovative neonatal tele-resuscitation curriculum. We hypothesized that a simulation-based mastery learning model would allow trainees to achieve proficiency in leading remote neonatal resuscitations via telemedicine. Here we report our approach to curricular design, assessment tools used, and our preliminary outcomes. We developed a novel, proof-of-concept curriculum using our center's clinical neonatal tele-resuscitation program 11 that afforded neonatal-perinatal medicine (NPM) fellows increased exposure to and leadership of remote neonatal resuscitation. The study was conducted from July 2018 to March 2021, coincident with our implementation of a longitudinal neonatal tele-resuscitation curriculum for NPM fellows at Mayo Clinic. The fellowship program includes one fellow per year with a maximum of four fellows. Fellows attend high-risk deliveries and rotate at Mayo Clinic's 34-bed Level IV regional NICU and 24-bed Level III NICU 12 . Four Mayo Clinic Health System hospitals participated in the tele-resuscitation curriculum. These hospitals had Level I well newborn (n = 3) or Level II special care (n = 1) nurseries 12 with annual delivery volumes ranging from 250-700 live births/year. Nine NRP instructors formed the 'local NRP instructor' group and were familiar with the telemedicine technology used for neonatal resuscitation. The NRP Provider Courses included learners who were pediatricians, family physicians, nurses, and respiratory therapists completing either their initial or renewal training session. Using the simulation-based mastery learning model 13, 14 and instructional design principles of technology-enhanced simulation education 15 , we designed a three-year, longitudinal curriculum with the goal of training NPM fellows to successfully lead neonatal resuscitations using telemedicine. As such, simulation was integrated across the curriculum in a goal-directed, systematic manner. The process for simulated neonatal tele-resuscitations was the same as our process for actual clinical consultations 11 , including community hospitals requesting a consult via the organization's admission and transfer call center, the neonatologist or NPM fellow connecting to the community hospital team via telemedicine, and rapidly orienting the neonatologist/fellow to the resuscitation scenario before assuming the team leader role. The wireless proactively monitored telemedicine device in the community hospital had a dual camera system controllable by the neonatologist/NPM, a directional echo-cancelling microphone, and two speakers (Fig. 1A) . The neonatologist/NPM connected to the device from a workstation in the NICU using the provider access software. The software includes a 'multipresence' feature that permitted the fellow to experience the consult exactly as experienced by the neonatologist, including views of the mannequin and community hospital team, interpersonal communication, and control of the telemedicine cameras ( Fig. 1B) . At the end of the simulation, fellows participated in the group debrief that was led by the local NRP instructor. Participation in the debrief allowed for bidirectional feedback between the resuscitation team and the remote leader. The curricular approach is detailed in Fig. 2 . During their first year, fellows completed Neonatal Resuscitation Program (NRP) instructor training. They viewed three learning modules related to the unique features of neonatal tele-resuscitation, i.e., workflow considerations, professionalism and communication, and legal and compliance considerations. Fellows also completed telemedicine technical training that included operating the two cameras (e.g., panning and zooming) to obtain adequate view of the infant, monitors, and remote team members as well as manipulating the audio con- trols. Fellows had unlimited access to demonstration devices such that they could practice these skills at any time during training. Second-year fellows observed simulated tele-resuscitation consults, where the neonatologist guided a community hospital team through a complex neonatal resuscitation that was conducted during NRP training. After completing the observations, fellows assumed the role of expert consultant in two to three simulated teleresuscitation consults that were supervised by a neonatologist. After each simulation, fellows participated in a brief reflection on action and tailored, individualized learning with the Teleneonatology Medical Director (JLF). Fellows progressed to unsupervised simulated tele-resuscitation consultations as competency increased. This was determined both by achieving a mean score of 4 or more for each performance item during supervised consultations, as well as reported readiness for unsupervised consultation by the fellow. If fellows did not report comfort with leading remote resuscitation, they were offered additional opportunities for supervised consultation and feedback. Third-year fellows continued to perform unsupervised simulated consults. Using the same telemedicine platform, fellows who mastered simulated consults were permitted to perform clinical teleresuscitation consults supervised by a neonatologist using the multipresence feature. Because there were no published assessment tools for neonatal tele-resuscitation training, we developed instruments to evaluate the knowledge, skills, and behaviors that are critical to lead a remote, often unfamiliar, team through neonatal resuscitation. Content experts in neonatal tele-resuscitation (JLF) and health care delivery (JLF, BLK) developed written tests that were administered after fellows completed the learning modules. Experts in tele-resuscitation (JLF, BLK) and graduate medical education (GME; WAC) developed a performance assessment tool for observed simulated and clinical tele-resuscitation consults (Appendix 1). The tool assessed competency through eight questions mapped to the six ACGME core competencies, with responses on a 1 to 5 scale (1 = critical deficiencies; 5 = competence of an expert) that included descriptions for raters (i.e., the supervising neonatologists and local NRP instructors). The assessor could select "Unable to Assess" and provide individualized written feedback. Assessments were requested from both the supervising neonatologist and the local NRP instructor for all supervised consults (n = 10), from the local NRP instructor for the unsupervised consult (given no supervising neonatologist was present, n = 8), and from the supervising neonatologist for the clinical tele-resuscitation (as there was no NRP instructor at the delivery, n = 1), leading to 29 possible assessments. Descriptive statistics were performed on the written tests and performance assessments. All responses were reported as percentages, median (range), and mean (SD) as appropriate. Given the small sample size, no comparative analyses were done. Four NPM fellows participated in the neonatal tele-resuscitation curriculum. They successfully completed NRP instructor training, the three learning modules, and the telemedicine technical training. Median (range) scores on the modules were 96% (91-100%) for Module 1: Introduction to Teleneonatology; 100% for Module 2: Professionalism and 'Webside' Manner; and 96% (92-100%) for Module 3: Legal and Compliance Considerations. Fellows completed two to three supervised simulated consultations before achieving scores and reporting confidence to progress to unsupervised simulated consultations. Fellows performed one to four unsupervised simulated consults during the study period. Only one fellow progressed to the level of supervised clinical consultation during the pilot study, which occurred after four unsupervised simulation experiences. In total, the cohort of four fellows completed 18 simulated tele-resuscitations and one clinical tele-resuscitation. Performance assessments were requested for the 19 teleresuscitations. One assessment was sent to the wrong person, and three assessments were not completed (two by local NRP instructors during unsupervised consults, and one by the local NRP instructor during a supervised consult), resulting in 25 returned (86% of available) performance assessments. The 25 completed assessments provided a total of 190 rated responses and ten "unable to assess" responses to the eight questions (Table 1) . Mean scores across the six ACGME competencies ranged from 4.2 to 4.6, on a 1 to 5 rating scale, with 4 equating to 'ready for unsupervised practice' and 5 equating to 'competence of an expert'. Written feedback provided to the fellows included suggestions related to both neonatal resuscitation key behaviors and telemedicine-specific knowledge and skills. Selected examples are presented in Table 2 . This report provides early evidence that a curriculum utilizing a simulation-based mastery learning model allowed trainees to achieve proficiency in neonatal tele-resuscitation. Given the expansion of telemedicine services 16 , effective training for GME learners is essential. Our institution has a well-established regional, neonatal tele-resuscitation program that includes scheduled simulations with our community hospital sites 11 . These simulations provided a safe, supportive learning environment where fellows gained familiarity with the technology, processes, skills, and behaviors needed for teleresuscitation. This curriculum presents a longitudinal approach based on principles of simulation-based mastery learning models 15 and deliberate practice 17 . Design elements associated with improved long-term learning and transfer, including repetitive and distributive practice, cognitive interactivity, individualized learning, feedback, and clinical variation 15 , were incorporated into the training. Fellows were increasingly exposed to authentic clinical experiences using telemedicine, first by developing their skills as NRP instructors, then as remote consultants during NRP simulation, and ultimately as a remote consultant during actual neonatal resuscitations. Given the high standards needed for physicians to conduct teleresuscitation, we chose to incorporate multiple points of assessments from both neonatologists and experienced local NRP instructors over time. Written and performance assessment scores suggest that fellows comprehend key elements of tele-resuscitation, including the workflow and inter-professional communication needed to provide high quality care at a distance. By leading neonatal teleresuscitations, fellows experience the complex interplay between medical knowledge, communication skills, and trust affecting their ability to lead an interprofessional team. Feedback provided to the fellows after tele-resuscitation was highly relevant to the key behavioral skills emphasized in NRP training, including communicating effectively, maintaining professional behavior, using available information and resources wisely, allocating attention wisely, and clearly identifying a team leader. Feedback also reflected the need to develop skills specific to tele-resuscitation, such as agility with camera placement, voice intonation, and pacing of recommendations. Our pilot curriculum has limitations. First, general telemedicine competencies and validated assessment tools for GME learners are currently in their infancy 4 and do not yet exist for training fellows in tele-resuscitation. Core competencies and development of subsequent Entrustable Professional Activities of telehealth consultation will be necessary to guide future training and assessment. For this reason, we developed a pilot assessment tool guided by local experts in neonatal telemedicine and graduate medical education that will require further validity evidence in the future prior to widespread use. Our telemedicine use was limited only to acute settings, and our training may not generalize to non-emergent consultation. Additionally, during the pilot period, only one fellow progressed to the level of supervised clinical consultation, so further evaluation of fellow performance during clinical tele-resuscitation consults is needed. Lastly, feedback was collected from neonatologists and NRP instructors but not NRP students; this may provide an important perspective that helps NPM fellows improve communication and team-based performance. Future studies should assess the patient impact of learner-led tele-resuscitation. Our pilot data suggest that fellows who progress through such a curriculum are ready to perform clinical consultation, but further work is needed to ensure patient safety. While several studies on telemedicine training have reported learners' reactions 18, 19 , there is only one report on the impact of telemedicine training on patient outcomes that compared time to tPA between fellow-led and attending physician-led telestroke consultations. In this program, fellows progressed through telemedicine training through graduated exposure and entrustment until independently performing clinical telestroke consultation 20 -similar to our pilot tele-resuscitation curriculum 20 . Our tele-resuscitation curriculum has integrated higher assessment outcomes 21 , i.e., fellows' demonstration of skill in real and simulated practice, but could be strengthened by comparing patient outcome variables attained by fellows and attending neonatologists in our clinical tele-resuscitation program. Finally, general barriers to telemedicine training must be overcome in any telemedicine program, including the lack of reliable telemedicine infrastructure, difficulty identifying experts to serve as instructors, and the need for additional licensing and credentialing 15 . Our curricular model and assessment tools should be validated in additional learner populations utilizing telemedicine for acute or emergency care before widespread adoption within graduate medical education. As telemedicine use continues to expand, curricula that improve learners' comfort with and proficiency in tele-resuscitation are essential. A simulation-based mastery learning model may be one approach that affords learners gradual exposure to and mastery of complex tele-resuscitation knowledge, skills, and behaviors. The Mayo Clinic IRB deemed this study exempt as education research. All learners provided informed consent to use of their assessment data in this study. ACGME Response to the Coronavirus (COVID-19) The TeleHealth OSCE: Preparing Trainees to Use Telemedicine as a Tool for Transitions of Care Competency-Based Training for Entrustment in Telehealth Consultations AAMC. Telehealth Competencies Across the Learning Continuum. AAMC New and Emerging Areas in Medicine Series Lack of Telemedicine Training in Academic Medicine: Are We Preparing the Next Generation? Expert Consensus: Telehealth Skills for Health Care Professionals The Current Pediatric Telehealth Landscape The impact of telemedicine on the quality of newborn resuscitation: A retrospective study The Effect Of A Newborn Telehealth Program On Transfers Avoided: A Multiple-Baseline Study The Impact of Telemedicine on Transfer Rates of Newborns at Rural Community Hospitals Emergency Video Telemedicine Consultation for Newborn Resuscitations: The Mayo Clinic Experience Levels of Neonatal Care Simulation in healthcare education: A best evidence practical guide. AMEE Guide No. 82 A critical review of simulation-based mastery learning with translational outcomes Comparative effectiveness of instructional design features in simulation-based education: systematic review and meta-analysis Telemedicine: Pediatric Applications The Differential Influence of Experience, Practice, and Deliberate Practice on the Development of Superior Individual Performance of Experts Developing a Telemedicine Curriculum for a Family Medicine Residency Telehealth for the internal medicine resident: A 3-year longitudinal curriculum A call for formal telemedicine training during stroke fellowship The assessment of clinical skills/competence/ performance None. None. The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Jennifer L. Fang and William A. Carey have licensed intellectual property with and can earn royalties from Teladoc Health. Dr. Fang also holds stock in the company. Supplementary data to this article can be found online at https://doi. org/10.1016/j.resplu.2021.100172.