key: cord-0870012-92y9iace
authors: Jauregui, J; Kessler, R; Villalón, N; House, J; Cole, M; Kman, N; Shandro, J
title: Medical Student Experiences of Applying Into EM During the COVID‐19 Pandemic: A Multi‐institutional Survey of EM Bound Medical Students
date: 2021-02-09
journal: AEM Educ Train
DOI: 10.1002/aet2.10587
sha: 1c572de5fc998575f8c313d388c7b8223951b519
doc_id: 870012
cord_uid: 92y9iace

BACKGROUND: The COVID‐19 pandemic added new and great uncertainty to the typical approach to applying into emergency medicine (EM) for medical students. There is little data on students' lived experience of applying during this time period. We performed a multi‐institutional survey of EM‐bound students' experiences with preparing to apply into EM during the pandemic. METHODS: This was a cross sectional survey design study of fourth year students preparing to apply into EM during the 2020‐2021 academic year. All self‐identified EM‐bound students at four participating institutions were recruited by email in June 2020 to participate in a 13‐item, 5‐point Likert scale survey. Univariate descriptive statistics, response rate and non‐response bias were calculated. RESULTS: Sixty‐seven of 125 eligible students responded for an overall response rate of 53.6%. Nonresponse bias for specific survey items ranged from 0.01 to 0.12. Students rated the importance of securing an EM rotation at their home institution the highest of any item (M = 4.81, SD = 0.68). Students indicated higher satisfaction with advice from their department of EM (M = 4.28, SD = 0.75) than from their school of medicine (M = 3.52 , SD = 0.89). Students indicated higher confidence in their home EM rotations’ ability to assure adequate PPE (M = 3.91, SD = 0.83) than an away rotation (M = 2.82, 1.09). Students reported feeling between moderately and quite stressed about applying into EM this year (M = 3.49, SD = 1.01), but reported the financial stress the lowest of any item (M = 1.46, SD = 0.84). Students rated it highly important that away rotations prioritize students from institutions without an EM residency (M = 4.51, SD = 0.93). CONCLUSION: Medical students applying into EM during the COVID‐19 pandemic felt confident in their EM advisors’ recommendations and their home institution’s PPE provision. Students with EM residency programs at their schools recognized the importance of away rotations being prioritized for students from schools without EM residency programs. Strong EM advising is important to students.

reported the financial stress the lowest of any item (M = 1.46, SD = 0.84). Students rated it highly important that away rotations prioritize students from institutions without an EM residency (M = 4.51, SD = 0.93).

Conclusion: Medical students applying into EM during the COVID-19 pandemic felt confident in their EM advisors' recommendations and their home institution's PPE provision. Students with EM residency programs at their schools recognized the importance of away rotations being prioritized for students from schools without EM residency programs. Strong EM advising is important to students.

The COVID-19 pandemic added new and great uncertainty to the seemingly ordered and typical approach to applying into emergency medicine (EM) for medical students. 1 One of the most significant disruptions from the pandemic was a sudden lack of access to away rotations for Accepted Article medical students applying into EM. Traditionally, performance on an away EM rotation has been very highly valued by EM residency program directors when evaluating an EM applicant's portfolio. 2 On May 11th, 2020, the Coalition for Physician Accountability, a group of national medical education organizations, recommended that away rotations be discouraged for the 2020-2021 academic year except in specific circumstances. 3 In addition, the CORD Advising Students Committee In Emergency Medicine published a consensus statement encouraging students to go on fewer, if any, away rotations. 4 Because of the disruption to the clinical learning environment and the ability to travel to away rotations, many institutions, including all four of our institutions, cancelled away rotations for their students. 5 Other than anecdotal reports, there is little data on students' experiences with preparing to apply into EM given changes due to the COVID-19 pandemic or how our advising interventions in response to the pandemic were impacting them.

As such, we performed a multi-institutional survey of EM-bound students' experiences with preparing to apply into EM during the pandemic.

This was a cross sectional survey design study. We considered all medical students self-reporting to their respective institutions as applying into EM from a purposeful sample of four institutions as eligible to participate in the survey. The University of Michigan (Michigan), OSU Wexner Medical Center (OSU), Oregon Health & Science University (OSHU) and The University of Washington (UW) all participated in the study. We surveyed students in June 2020, in the midst of the COVID-19 pandemic, social distancing measures, and at a time of national uncertainty about the future of medical students being permitted in the clinical environment.

We recruited participants by email. The initial survey invitation was sent out on June 1, 2020 with 2 reminders sent at 1 week intervals each. Data collection was completed by the end of June 2020. All participation was voluntary and there was no compensation for taking the survey. All responses were anonymized. The Human Subjects Division at all four institutions deemed the study to be exempt from IRB.

This article is protected by copyright. All rights reserved

We developed an original 13-item survey instrument guided by Messick's validity framework. 6 The content validity for the survey instrument was based on the CORD student advising guide and the timely Council of Residency Directors in EM (CORD) Advising Students Committee in EM (ASC-EM) consensus statement. 2, 4, 7 All authors are EM clerkship directors and iteratively contributed to the development of the survey. We intentionally avoided including more demographic questions in the survey to maintain strict anonymity of all participants. We performed cognitive, "think aloud" interviews with two senior medical students who had recently matched in EM and two rising fourth year medical students pursuing non-EM specialties for response process validity evidence and for clarity. We incorporated all feedback into the final instrument, which we determined via consensus of all authors (see appendix). We rated each item on a 1 to 5 point Likert scale ranging from "Not at all" to "Extremely," with response anchors intentionally worded to emphasize the construct being measured rather than simple agreement. 8

We collected and managed the study data using REDCap electronic data capture tools hosted at the University of Washington. REDCap (Research Electronic Data Capture) is a secure, webbased software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources.

We conducted data analysis using Microsoft Excel 2018 (Redmond, Washington) with which we calculated univariate descriptive statistics including the mean (M) and standard deviation (SD) of each item, a response rate according to the AAPOR definition of response rate 5 (RR5) and the non-response bias for each item to estimate the difference between those who responded and those who did not, using wave analysis. [9] [10] [11] [12] 

This article is protected by copyright. All rights reserved We electronically delivered the survey via email to 125 medical students applying into EM at our four institutions. Of those, 67 students responded to the survey for an overall response rate of 53.6%. Response rates for each participating institution ranged from 38.1%-70% (Table 1) .

The descriptive statistics and nonresponse bias for 11 of the 13 survey items are displayed in Table 2 . The nonresponse bias was minimal, ranging from 0.01 to 0.12 on the 5-point scale. The other two items inquired about whether students perceived that they were still permitted to do away rotations. Of all respondents, only one student reported that their institution was allowing them to do away rotations despite their institution's policy against it. The same student also reported that they were "Not at all Confident" that they would secure an away rotation despite their perceptions they were allowed to do so.

EM-bound medical students rated the importance of securing an EM rotation at their home institution prior to submitting residency applications this year the highest (M = 4.81, SD = 0.68).

Students also indicated that they were more satisfied with the advice from their department of EM (M = 4.28, SD = 0.75) than from their respective schools of medicine (M = 3.52 , SD = 0.89). Not surprisingly, students reported feeling between moderately and quite stressed about applying into EM this year (M = 3.49, SD = 1.01) and at least moderately concerned that they needed to do an away rotation at a particular institution to get an interview there (M = 3.27, SD = 1.14). However, they also reported that the rationale for not permitting away rotations was quite clear to them (M = 4.34, SD = 0.90) and that they were moderately to quite confident in their advisors plan to support them through the process of applying this year (M = 3.68 , SD = 0.98).

Students rated the financial stress they were experiencing the lowest of any item (M = 1.46, SD = 0.84). In addition, they indicated that they were much more confident in their home EM rotations' ability to assure them adequate PPE (M = 3.91, SD = 0.83) than an away EM rotations' ability to do so (M = 2.82, 1.09). It was highly important to students that away rotations prioritize spots for students at institutions without an EM residency program (M = 4.51, SD = 0.93).

These results illustrate the stressors that medical students experienced while preparing to apply in EM during the COVID-19 pandemic and illuminate areas where we can focus our support for students in response. They underscore the importance of effective advising that has been

This article is protected by copyright. All rights reserved previously demonstrated in the literature and add to the paucity of data we have on students' experience with advising. 13 Students identified that their EM departmental advising inspired more confidence than the general career advising from their medical schools. These results highlight the importance of having engaged and involved specialty-specific advising, in addition to the more generalized residency application advising from a school of medicine. Although students applying in all specialties faced similar challenges with lack of away rotations and upended schedules, our EMbound students placed a higher value on the career advising from EM faculty than the generalized school of medicine advising. These findings may have implications for medical students at institutions without an EM residency program to provide directed career advising, where students may face even more challenges than usual during a time when specialty-specific career advising and support is viewed as critical by EM-bound students. Furthermore, students from an institution without an EM residency program are much more likely to be from historically excluded in medicine backgrounds. 14 The different advising and EM experiential opportunities that students have access to at these institutions may further exacerbate existing inequities in our field.

Our results also highlight that students feel between moderately and quite stressed, but not extremely so. We imagine that the clear EM national consensus guidelines and collaborative leadership from EM specialty societies also contributed to a sense of security and community in the EM applicant group. 15 It would be interesting to compare our results to students applying into other specialties, particularly those which did not release clear national guidelines to guide and support programs and applicants during this time.

Students expressed confidence in their advisors' plans and in the clarity from their advisors for the rationale of not permitting away rotations this year. Supporting this, they strongly understood the importance of rotating at their home institution in EM prior to applying to residency.

Students recognize that an away rotation is important to getting an interview and nonetheless still agree with the rationale of the no away rotation policy. To us, this demonstrates that our EM

This article is protected by copyright. All rights reserved bound students understand the seriousness of the pandemic and importance of shared sacrifice. It also reinforces the importance of strong and timely leadership through consensus guidelines.

Predictably, and positively, students expressed very low financial stress from aways this year-further supporting the fact that they understand the policies and rationale for no away rotations.

The low financial stress is of benefit during a time of great stress and it will likely also be an unintended benefit of having remote interviewing for residency this year. Given the well documented financial burden of applying to residency, it would be wise to consider ways that we can maintain some of the decreased financial stress of applying that has resulted from this years' upheaval. 16 In addition, further research evaluating the effectiveness of virtual interviewing in light of the financial burden of traveling to traditional in-person interviews would also help us better understand ways we may be able to address financial inequalities in applying into EM.

Additionally, students were more confident that their home EM rotation would be able to provide them adequate PPE than an away EM rotation (figures 3 and 4). Students seem to be more apprehensive about the safety of away rotations and more comfortable with the safety of their home institution. It is not surprising to be more comfortable with what is familiar. However, it also illustrates the challenges of variance among local and regional specific policies when students are expected to move between different hospital systems nationally.

Finally, students reported that it was highly important that away rotations prioritize spots for students without home EM residency programs. This highlights a sense of shared community and generosity among EM-bound students, who recognize that the limited available away rotations should go to students most in need of a core EM rotation. It is clear that students have strong confidence in their EM advisors, even in a time of great uncertainty, and it follows that our EM educational community has an opportunity to evaluate all medical students' access to EM specific advising including the need to prioritize away rotations for those students at institutions without EM residency programs. Given that there are over 80 medical schools without EM residency programs, we would suggest implementing practical, nationally collaborative processes such as creating more formal partnerships between schools without an EM residency program and other geographically proximate institutions with EM residency programs. Other

This article is protected by copyright. All rights reserved ideas include increasing virtual advising, either from these formal partnerships, or through professional society sponsored national advising teams. 17

This study is limited by the nature of it being a convenience sample of medical students from 4 institutions rather than a sample of the entire population of students applying into EM in the 2020-2021 academic year. These 4 institutions also have departments of emergency medicine with established EM residency programs, which may make it difficult to generalize the results to all, especially to students at schools without EM residency programs. Future work in this area should certainly expand to a wide range of institutions to include the important perspectives of a broader range of medical students. Furthermore, while the nonresponse bias was minimal and the response rate was adequate, our results may not be fully representative of all the students we recruited. We also recruited students who self-identified as pursuing EM as a career in June 2020, which may have changed over time and a small percentage of responses may no longer be from students still applying into EM.

This multi-institutional study of 4th year medical students applying into EM during the COVID-19 pandemic demonstrates that the communication and rationale for no away rotations this year was clear, and that students felt confident in their EM advisors' recommendations and their home institution's ability to provide them adequate PPE. Students with EM residency programs at their schools recognized the importance of away rotations being prioritized for students from schools without EM residency programs. During a time of uncertainty, clear and consensus-based advising likely made this experience less stressful for students. Most importantly, it reinforces the importance of strong EM advising to students. 

This article is protected by copyright. All rights reserved Figure 4 : Student Responses to the Survey Item, How confident are you that any away EM rotation will e to assure you adequate PPE?" Extreme Moderatel

COVID-19: A Medical Student's Perspective. AEM Education and Training

student-Accepted Article This article is protected by copyright. All rights reserved resources/2019studentadvisingguidefinal-web-version

AAMC Guidance on Medical Students' Participation in Direct Patient Contact Activities. 2020. Available at

Consensus Statement Regarding SLOEs and Away Rotations from the CORD Advising Students Committee in Emergency Medicine. 2020. Available at

Advising Medical Students During COVID-19: The Case for a Single Emergency Medicine Rotation for All

Validity: on meaningful interpretation of assessment data

Addressing Challenges in Obtaining Emergency Medicine Away Rotations and Standardized Letters of Evaluation Due

Developing questionnaires for educational research: AMEE Guide No

Surveys of Health Professions Trainees: Prevalence, Response Rates, and Predictive Factors to Guide Researchers

Standard definitions: Final dispositions of case codes and outcome rates for surveys. aapor.org. American Association for Public Opinion Research

Improving response rates and evaluating nonresponse bias in surveys: AMEE Guide No. 102

Accepted Article This article is protected by copyright. All rights reserved 12. Brown C, Watson W. Techniques for improving survey response rates in medical education

Mentoring during Medical School and Match Outcome among Emergency Medicine Residents

-2-21 Residency Application Process for US Medical Students Planning Careers in Emergency Medicine in the Main Residency Match

Time to Reevaluate the Away Rotation: Improving Return on Investment for Students and Schools

COVID-19: A Driver for Disruptive Innovation of the Emergency Medicine Residency Application Process

This article is protected by copyright. All rights reserved