key: cord-1010127-7rvzkfwq authors: Cash, Rebecca E.; Leggio, William J.; Powell, Jonathan R.; McKenna, Kim D.; Rosenberger, Paul; Carhart, Elliot; Kramer, Adrienne; March, Juan A.; Panchal, Ashish R. title: Emergency medical services education research priorities during COVID‐19: A modified Delphi study date: 2021-08-21 journal: J Am Coll Emerg Physicians Open DOI: 10.1002/emp2.12543 sha: b8d117d0ef3dbb2c7ea3352a33143a2aff3c6dfe doc_id: 1010127 cord_uid: 7rvzkfwq OBJECTIVE: Our objective was to identify research priorities to understand the impact of COVID‐19 on initial emergency medical services (EMS) education. METHODS: We used a modified Delphi method with an expert panel (n = 15) of EMS stakeholders to develop consensus on the research priorities that are most important and feasible to understand the impact of the COVID‐19 pandemic on initial EMS education. Data were collected from August 2020 to February 2021 over 5 rounds (3 electronic surveys and 2 live virtual meetings). In Round 1, participants submitted research priorities over 9 specific areas. Responses were thematically analyzed to develop a list of research priorities reviewed in Round 2. In Round 3, participants rated the priorities by importance and feasibility, with a weighted score (2/3*importance+1/3*feasibility) used for preliminary prioritization. In Round 4, participants ranked the priorities. In Round 5, participants provided their agreement or disagreement with the group's consensus of the top 8 research priorities. RESULTS: During Rounds 1 and 2, 135 ideas were submitted by the panel, leading to a preliminary list of 27 research priorities after thematic analysis. The top 4 research priorities identified by the expert panel were prehospital internship access, impact of lack of field and clinical experience, student health and safety, and EMS education program availability and accessibility. Consensus was reached with 10/11 (91%) participants in Round 5 agreeing. CONCLUSIONS: The identified research priorities are an important first step to begin evaluating the EMS educational infrastructure, processes, and outcomes that were affected or threatened through the pandemic. The COVID-19 pandemic has affected the entire medical community, 1,2 and emergency medical services (EMS) are no exception. The need to understand the impact from the COVID-19 pandemic on the EMS education pipeline was recognized by national organizations and stakeholders in the EMS community. 3 In order to understand and address the impact, a national task force steering committee was formed by gathering representatives from 13 organizations involved in EMS education during the COVID-19 pandemic. The EMS workforce, and care they provide to ill and injured patients outside of the emergency department, is an integral part of the overall US emergency care system. Impacts from COVID-19 have affected EMS service call volume, [4] [5] [6] [7] increased EMS professional fatigue, and worsened well-being, 8, 9 and slowed the education pipeline. 3 Specifically, for the EMS education pipeline, it is believed that the majority of EMS education programs experienced at least some form of temporary closure, modifications to delivery of content and program requirements, and limited or loss of access to simulated education as well as live field and clinical opportunities. 3, [10] [11] [12] [13] More so, ability to test, certify, and recertify were delayed or extended with some states modifying the processes for EMS licensure. 3, 13 Workforce shortages and poorly trained personnel can have downstream effects for emergency physicians. Quantifying the impact of the COVID-19 pandemic on EMS education programs and developing mitigation strategies have been recognized as a priority, despite known challenges, such as resource sharing and standardized mechanisms for collecting data related to EMS initial education. 3, 10 Our objective was to identify research priorities to understand how the COVID-19 pandemic affected initial EMS education. We focused on initial EMS education for certification, rather than continued education after certification has been obtained, because the EMS education pipeline is directly related to workforce stability. We used a modified Delphi method with a panel of national EMS stakeholder organizations (Table 1) to develop consensus on the research priorities that were most important and feasible to understand the impact of the COVID-19 pandemic on initial EMS education. The Delphi methodology is a structured approach to consensus building with an expert panel that involves iterative rounds of data collection and feedback, typically in a face-to-face format. [14] [15] [16] As an extension of this approach, the modified Delphi methodology leverages asynchronous electronic communication. 15 The Delphi methodology has been widely used to develop research priorities in EMS and other healthcare settings. [17] [18] [19] [20] Participants for the expert panel were identified by national stakeholder organizations that formed the task force steering committee ( we asked participants to consider availability or access to existing data sources, the need for collecting data, and the cost or funding required to conduct the work. We calculated a mean importance and feasibility rating for each research priority. Using the calculated scores, we created an aggregated weighted importance score reflecting the combination of ratings for importance and feasibility, where: weighted importance score = 2 3 * mean importance We gave higher weight to importance ratings because some participants were more knowledgeable regarding importance of topics to EMS education rather than research feasibility. The top 12 priorities by weighted score were advanced to the Round 4 to determine consensus based on an a priori decision. The decision to use the top 12 priorities for final ranking was to enable participants to prioritize more effectively by limiting the cognitive burden. Round 4 was conducted via webinar in October 2020. We first presented a summary of the results from the previous rounds and the top 12 research priorities from Round 3. Participants were then asked to rank the research priorities in order from highest to lowest priority, considering both importance and feasibility in that ranking. A ranking score was determined by assigning points to each rank option, where a rank of 1 (most important) was equal to 12 points and a rank of 12 (least important) was equal 1 point. 21, 22 The sum of points obtained was calculated. Based on an a priori decision, the top 8 research priorities were then used to create a final prioritized list, rank ordered by total points, and the percentage of participants ranking each in the top 4 was calculated. The final round was conducted by electronic survey in February 2021. Participants were asked if they agreed with the top 8 prioritized list of research priorities. If a participant did not agree with the prioritization as decided by the group, the reason for dissent was elicited. We defined majority consensus as at least 75% of participants agreeing with the prioritized list of the top 8 research priorities. Because of an initially low response rate, a final unplanned reminder in February 2021 to nonresponding participants was required. A total of 15 participants were included from 12 national organizations (Table 1) . Not all participants chose to respond to the electronic surveys; however, participants were involved in the live meetings where data were also collected and thus included. In Round 1, we received 11 responses to the electronic survey (response rate = 73%). We received 12 responses each for Rounds 3 and 4 (response rate = 80%). In the final round, we received 11 responses (response rate = 73%). During the first round, 117 research priorities were submitted by the panel, ranging from 3 to 29 priorities in each of the 9 areas. After initial thematic analysis, there were 49 distinct research priorities over the 9 areas, which were then reduced to 23 final research priorities owinge to overlap across the topic areas. During The 27 research priorities identified by the panel were then rated by importance and feasibility in Round 3. The mean importance rating ranged from 2.42 to 3.75, and the mean feasibility rating ranged from 2.33 to 3.45. The final weighted importance score ranged from 2.42 to 3.56. Table 2 shows the ranking of research priorities by weighted importance score. Participants were provided the top 12 research priorities based on preliminary ranking by weighted importance during the live meeting and asked to rank order the list. Table 3 This study has several limitations. The participants were selected as representatives of the stakeholder organizations included in the larger national task force; however, we asked participants to respond as individuals rather than on behalf of the organizations they represented. In asking organizations to select their representative and study participants in this manner, certain perspectives may have been excluded. This study was limited to impact on initial EMS education in the United priorities, but we recognize that rankings of importance and feasibility were based on the expert panel's experience and opinions, which may not reflect the wider EMS community. Finally, the response rate to the final round was initially lower than expected, leading to an unplanned reminder to attempt to obtain participation from the full expert panel. The COVID-19 pandemic hindered access to essential EMS educational activities, such as laboratory hands-on learning and live patient encounters in hospital and field settings. [11] [12] [13] The impact of the pandemic on EMS education and subsequent effects on the workforce pipeline created concern for stakeholders and national organizations. The pandemic has accelerated an ongoing trend toward increasing distance learning and hybrid programs for initial and continuing education. 26, 28, 29 Before the pandemic, programs had also begun using simulation-based training to replace some of the traditional EMS education objectives of programs. 28, 29 Several of the research priorities identified by the panel address this trend to examine if these changes in educational methods have affected the entry-level competency of EMS clinicians. One interesting finding that was noted by the panel in the second live meeting was regarding the perceived importance and ranking of student versus faculty health and safety. Although student health and safety was ranked in the top8, faculty health and safety were not. There may have been less concern regarding faculty safety as many faculty members are not exposed to live patient encounters; faculty members may have benefits and paid time off (eg, from the Families First Coronavirus Response Act) if exposed to COVID-19, 30 whereas students may not; and because there is no consistent requirement that students hold health insurance in case they fall ill. At the beginning of the pandemic there was also a student-specific concern regarding shortages of personal protective equipment during clinical assignments. Despite the recognition by the panel of the importance of student health and safety, and the ongoing personal burdens for many caused by the pandemic, both student and faculty stress and anxiety were ranked lower as research priorities. These topics remain potential research areas to promote EMS education program and student resilience. Of note, there was 1 dissenting vote when final consensus was determined. This vote contended that the top 12 priorities had similarities that prevented them from reaching consensus. Although this vote is important and their perspective valued, it should not take away from 91% consensus being reached regarding these 12 research priorities. Future work in this area should recognize the potential for this occurring and take steps to ensure heterogeneity in themes. We Whether it is understanding the COVID-19 pandemic or ways to prepare, collegial research projects supported collaboratively by national stakeholders are encouraged. The authors would like to acknowledge the task force expert panel members who participated and provided their expertise and knowledge to this effort. Furthermore, we would like to acknowledge the hard work and flexibility of EMS programs across the country as well as the students who are dedicating themselves to the EMS profession. With limited access to clinical rotations, we should identify alternatives such as simulation lab, telemedicine, or field internship. We should understand if student demographics and volume has changed from pre-to post-pandemic. Has COVID impacted the ability to reach competency in the cognitive domain? As programs adapted to COVID, it is unclear if student outcomes differ pre-and post-pandemic, including impact on certification examination success, the role of distance learning and simulation, and program requirements for completion. Education programs should remain open and accessible to students. It is unclear how many are still operating, if class offerings or sizes differ (eg, only day vs day and night prior), and the impact of these changes. We should understand if faculty demographics, size, and structure have changed from pre-to post-pandemic. This includes availability of faculty to teach, education of faculty to teach on different platforms, shifting of lecturers to simulation instructors, and attrition of current educators. Faculty health and safety Faculty must be kept safe while performing in-person instruction. Their perceptions of risk are unclear. Faculty stress and anxiety a Understanding of faculty mental health concerns is necessary. COVID-related stresses can lead to increased mental health concerns that should be recognized and explored. (Continues) Future of EMS education after the pandemic The changes programs made in response to COVID have changed EMS education. How will future graduation rates be impacted? Will these changes continue beyond the pandemic? Hospital/ambulatory site access EMS programs rely on both hospital and ambulatory sites to meet continued competency. Programs need advocacy assistance in removing barriers (eg, liability, PPE, shortages of personnel and preceptors, and inherent value of EMS workforce) to keep these experiences available. How is and how much simulation is being used EMS education programs are increasingly using simulation technology in various ways. We need to understand the ways simulation is used, the simulation curriculum being developed, and impact on ability to interact with patients. It is unclear if there is an impact on student and patient outcomes if field and clinical experiences are limited. Impact of program changes on future employment The students who are completing programs adapted during the pandemic must still meet the needs of employers. Keeping EMS education accessible for all students Distance learning creates obstacles for students and educators, such as internet infrastructure and assuring access for students with different needs. Medical director approval of pandemic related changes is necessary. The extent to which medical directors are involved with educational changes and program advocacy is unclear. Pandemic-specific topics of education Specific topics are unique to COVID and should be considered when educating students. These include airway management, cardiac arrest management, handling of death and dying patients, and appropriate use of PPE. EMS programs rely on pre-hospital internship access to ensure competency. Programs need advocacy assistance in removing barriers (eg, liability, PPE, shortages of personnel and preceptors, and funding) to keep these experiences available. Program funding With the increase in alternative EMS education platforms, the impact on costs and funding models is unclear. Programs have made changes in response to the pandemic, including transitioning to online/distance learning, flipped classrooms, and independent study. The effectiveness and impact of these changes are unclear. Psychomotor competency outcomes during COVID Has COVID impacted the ability to reach competency in the psychomotor domain? As programs adapted to COVID, it is unclear if student outcomes differ preand post-pandemic, including the role of simulation and requirements for completion. Recruitment/enrollment We need to understand current demands, barriers to recruitment and enrollment, and needs for the EMS education pipeline. These include perceptions of risk, accommodating diversity in student populations in recruitment efforts, and detailing the value of EMS education. Programs are responsible to sometimes multiple regulatory bodies to provide evidence of student competency. How do programs continue to define competency in order to meet regulatory standards? Simulation accessibility All EMS education programs and students should have access to simulation. It is unclear if programs can afford simulation labs or have training to provide this type of education. Students must be kept safe during clinical and field internships. Students should be trained in proper use and have access to PPE. Students are also facing increased life stress. Student perception of competency a Confident student self-perception is important to morale building. Do students feel competent to practice having missed significant portions of in-person learning and practice? Understanding of student mental health concerns is necessary. COVID-related stresses can lead to increased mental health concerns that should be recognized and explored. It is unclear how simulation can or should replace live patient encounters. What are the differences between learner and patient outcomes with simulation versus real world learning? Can minimum entry-level competency be obtained with simulation alone? How does this compare to practices for physicians and nurses? Abbreviations: EMS, emergency medical services; PPE, personal protective equipment. a Added in Round 2. Hematology/oncology fellowship emergency restructuring in response to the COVID-19 pandemic-Henry Ford Hospital, Michigan The impact of the COVID-19 pandemic on final year medical students in the United Kingdom: a national survey Ambulance calls for substance-related issues before and after COVID-19 EMS responses and non-transports during the COVID-19 pandemic System impacts of the COVID-19 pandemic on New York City's emergency medical services The impact of COVID-19 on statewide EMS use for cardiac emergencies and stroke in Massachusetts Exploring stress coping strategies of frontline emergency health workers dealing Covid-19 in Pakistan: a qualitative inquiry extreme stress" as toll of Covid pandemic climbs, ambulance company CEOs say Approaches for initial education at all levels following COVID-19 pandemic Survey conducted for NHTSA shows impact of COVID-19 on paramedic education programs Survey: status of paramedic programs Committee on Accreditation of Educational Programs for the Emergency Medical Services Professions An experimental application of the DELPHI method to the use of experts The Delphi technique: a worthwhile research approach for nursing Research guidelines for the Delphi survey technique Priorities for prehospital evidence-based guideline development: a modified Delphi analysis Modified Delphi method derivation of the FAMILY (Family Assessment of Medical Interventions & Liaisons with the Young) EMS instrument Development and validation of content domains for paramedic prehospital performance assessment: a focus group and Delphi method approach pediatric research priorities in prehospital care Delphi analysis of science gaps in the Planning and Conducting Needs Assessments: A Practical Guide Recruitment and retention in rural and urban EMS: results from a national survey of local EMS directors Recruitment and retention of new emergency medical technician (EMT)-basics and paramedics Why do emergency medical services (EMS) professionals leave EMS? Investigating the efficacy of a hybrid EMT course EMS Workforce Planning & Development: Guidelines for State Adoption Simulation Use in Paramedic Education Research (SUPER): a descriptive study Mapping the use of simulation in prehospital care -a literature review AUTHOR BIOGRAPHY Rebecca E. Cash, PhD, MPH, is an Assistant Professor of Emergency Medicine at