key: cord-0770982-5xv8pmto
authors: Lund, Sarah; MacArthur, Taleen; Keune, Jonie; Enger, Teresa; Martinez-Jorge, Jorys; McKenzie, Travis; Nelson, Megan; Olson, Angela; Que, Florencia; Stulak, John; Rivera, Mariela
title: Showcasing a General Surgery Residency Program During the COVID-19 Pandemic
date: 2021-05-06
journal: J Surg Educ
DOI: 10.1016/j.jsurg.2021.04.018
sha: d41269cd4dcaaa5f6f82551275c0f2f79b994266
doc_id: 770982
cord_uid: 5xv8pmto

OBJECTIVE: The virtual interview season has challenged general surgery residency programs to recruit applicants through the loss of visiting clerkships, tours, and time with residents. Webinars, increased informal resident and faculty sessions, and live-narrated video tours are potential solutions. This study aimed to assess the effectiveness of these elements in virtually showcasing a residency program during the virtual interview season. DESIGN/SETTING: Prospective applicants to one general surgery residency program (Mayo Clinic in Rochester, Minnesota) were invited to attend six webinars: Program Overview, Simulation Education, Diversity, Resident Life, Mingle with Residents, and Last-Minute Q&A. An anonymous survey was sent to all registered participants of the webinars. Interviewees participated in a preinterview social hour with resident and faculty and a live-narrated video tour of our facilities during their interview. A second anonymous survey was sent to all interviewees. PARTICIPANTS: Webinars – 33% of 159 unique registrants surveyed participated. Interviews – 46% of 109 interviewees surveyed participated. RESULTS: Average satisfaction with the webinars was 9.4/10. Overall, 98% of attendees felt that the webinars gave them a “feel” for the program. Attendees found the Last-Minute Q&A webinar and Program Overview to be most useful. For resident-led webinars, 100% of attendees felt that themed break-out rooms were effective. Average satisfaction with the interviews was 4.4/5. Interviewees rated access to faculty and residents highly (4.4/5 and 4.5/5, respectively). 98% of interviewees found the live-narrated video tour helpful. CONCLUSIONS: A webinar curriculum can be effective in virtual residency recruitment, as prospective applicants developed a good understanding of the resident program after participating. Further, live-narrated video tours and purposefully incorporating several avenues for informal conversations with residents and faculty can successfully address applicant concerns about virtual interviews.

OBJECTIVE: The virtual interview season has challenged general surgery residency programs to recruit applicants through the loss of visiting clerkships, tours, and time with residents. Webinars, increased informal resident and faculty sessions, and live-narrated video tours are potential solutions. This study aimed to assess the effectiveness of these elements in virtually showcasing a residency program during the virtual interview season. DESIGN/SETTING: Prospective applicants to one general surgery residency program (Mayo Clinic in Rochester, Minnesota) were invited to attend six webinars: Program Overview, Simulation Education, Diversity, Resident Life, Mingle with Residents, and Last-Minute Q&A. An anonymous survey was sent to all registered participants of the webinars. Interviewees participated in a preinterview social hour with resident and faculty and a live-narrated video tour of our facilities during their interview. A second anonymous survey was sent to all interviewees. PARTICIPANTS: Webinars À 33% of 159 unique registrants surveyed participated. Interviews À 46% of 109 interviewees surveyed participated.

RESULTS: Average satisfaction with the webinars was 9.4/10. Overall, 98% of attendees felt that the webinars gave them a "feel" for the program. Attendees found the Last-Minute Q&A webinar and Program Overview to be most useful. For resident-led webinars, 100% of attendees felt that themed break-out rooms were effective. Average satisfaction with the interviews was 4.4/5. Interviewees rated access to faculty and residents highly (4.4/ 5 and 4.5/5, respectively). 98% of interviewees found the live-narrated video tour helpful. 

In March 2020 the Association of American Medical Colleges issued a statement encouraging all residency, medical school, and fellowship interviews to transition to a virtual format until the resolution of the novel coronavirus 2019 (COVID-19) pandemic. 1 This, along with the general shift to virtual interactions, primed residency programs across the nation to prepare for a virtual interview season.

Applicants typically use interview days to determine compatibility based on both subjective opinions and objective details about residency programs. Many of the subjective opinions that candidates form come from spending 1 or 2 days talking to current residents, preinterview social events, interviewing with faculty, and touring facilities and program locations. With virtual interviews, decisions about compatibility will be made solely based on the interview sessions, eliminating many avenues for a future resident to determine their feel for a program. Several editorials propose that the advantages to virtual interviews include minimal disruption to medical student's rotations and cost-savings, potentially as much as $6000 per applicant. [2] [3] [4] On the other hand, editorials theorize that virtual interviews result in the loss of informal interactions between applicants and program leadership, faculty, and residents that enable applicants to evaluate a program for an adequate fit. 2, 3 Furthermore, virtual interviews do not allow applicants to view training facilities (hospitals/clinics and simulation centers) or experience the city a training program resides within. 2, 3 Several fellowship programs underwent virtual interviews in the spring of 2020. One editorial published on the experience of surgical oncology virtual fellowship interviews at one program described their interview day process: using break-out rooms for interviews and a main room for social interactions. 5 A survey of the experiences of applicants and faculty after these virtual interviews reported improvement in the perception of the flow of the interview schedule as compared to prior inperson interview days, however, identified areas of concern including inability to experience the local city and hospital environment. 4 Furthermore, a survey of breast surgery fellows after experiencing virtual interviews in the spring of 2020 showed that both applicants and faculty overwhelming felt that the experience was seamless but still expressed a preference for live interviews (69%). 6 In contrast, a recent nationwide survey of Otolaryngology prospective applicants reported that 37% of applicants felt that the program would not have enough information about them based on virtual interviews to make an informed decision and 62% of applicants did not think they would have enough information from virtual interviews to inform their rank list. 7 Furthermore, another nationwide survey of Urology prospective applicants reported that 64% of applicants felt resident informal interactions were the most important component of the interview day and 81% of applicants felt these informal interactions could not effectively be duplicated virtually. 8 In this survey, an additional 87% of applicants felt a city visit could not be accomplished virtually. 7 Several editorials have recommended that programs address the disadvantages to virtual interviews head-on through creative multimedia and interactive virtual solutions. [9] [10] [11] Based on these concerns about limited informal interactions between applicants and programs prior to the interview cycle, our general surgery residency program implemented a webinar curriculum for prospective applicants. To address concerns reported in the literature of limited informal interactions with faculty and residents during the interview day, interviewees were invited to an informal meet-and-greet with surgical faculty and current residents the evening before their interview and spent time in break-out rooms throughout the interview day with current residents. Finally, to provide a sense of the surrounding city and hospital facilities, interviewed applicants were given a live narrated, video tour by current residents during their interview day.

After implementing these components to our recruitment before and during virtual interviews, we aimed to assess the effectiveness of these virtual recruitment methods in showcasing our general surgery residency program.

Prospective applicants to Mayo Clinic in Rochester, Minnesota's general surgery residency program in 2020-2021 cycle were invited via social media and email to attend a webinar curriculum in August to October of 2020. Four webinars were held as a 1-to 1.5-hour long live video conference led by program leadership, in which program leadership gave a brief (15-20 minute) presentation regarding a topic specific to our general surgery resident program. An informal question and answer session followed. The topics for these webinars are Program Overview, Simulation Education, Diversity, and Last-Minute Question & Answer (Q&A) with Program Leadership. Each session started with a 20-to 30-minute presentation on the webinar topic led by a faculty leader in that area. These presentations were followed by a 45 minute to 1 hour session of informal question and answer with numerous faculty and resident panelists.

Two additional webinars (Mingle with Residents and Resident Life) allowed prospective applicants to interact with current general surgery residents to ask questions and gain insight about the program. At these resident-led webinars, break-out rooms with selected residents were available for prospective applicants to join in 15-minute rotations throughout 1.5-hour webinar. Each residentled webinar had four themed break-out rooms, using the themes: Clinical Experience, Residency Tracks (Preliminary, Cardiothoracic Integrated, Vascular Integrated), Life in Minnesota/Residency Culture, and Education & Research Experience. After the last webinar, all registered attendees were sent an anonymous, optional online survey about their webinar experience.

Virtual interviews at Mayo Clinic in Rochester, Minnesota included several components. The night before the interviews, interviewees attended a 1.5-hour resident and faculty social. Interviewees were rotated randomly through five 15-minute break-out rooms. Ice-breaker questions (i.e., What is your favorite vacation destination?) were used to facilitate conversation. Every interviewee spent one rotation in a break-out room with program leadership, one rotation in a break-out room with only residents, then three rotations in break-out rooms that contained a mix of faculty and residents.

During the virtual interview day, interviewees rotated through two 20-minute traditional interviews with surgical faculty and senior residents, a 20-minute virtual simulated skills mini-multiple interview that assessed technical and non-technical skills through six synchronously and asynchronously assessed short stations (described in a previous study), 12 a 60-minute informal question and answer video-call with current residents, and a 60-minute live-narrated video tour. The video tour included clinical (floor, outpatient, and inpatient) and nonclinical (cafeteria, gym, etc.) areas of the hospital and pertinent areas of the surrounding city. Several current residents live-narrated this tour for interviewees during a video call on the interview day, pausing the video to insert anecdotes and answer interviewees' questions. All components were conducted over a virtual platform, using the break-out room functionality to seamlessly rotate residents through sessions. After the conclusion of the interview day, interviewees were sent an anonymous, voluntary online survey about their virtual interview experience at our institution. This study was deemed exempt by the Mayo Clinic Institutional Review Board.

We had 159 unique registrants for all webinars, with several participants attending more than one webinar. Approximately 100-120 prospective applicants registered for each webinar, with approximately 50-80 attendees at each session. Of the 159 registrants, 33% (53) participated in the survey. Participants attended a variety of webinar sessions, with the largest number attending the Last-Minute Q&A (33 participants) and the Program Overview (29 participants) sessions (Fig. 1) . The majority of participants felt that the webinars should be offered in-person in future years (62%), but 34% of participants felt a virtual component to the webinars should continue (Fig. 2) . Participants felt that the Program Overview and Last-Minute Q&A sessions gave them the best subjective feel for the residency program (Fig. 3) . Almost all of the participants (98%) felt they developed a subjective feel for the residency program after the webinars and 72.5% of participants developed a unique program understanding from attending the webinars (Fig. 2) . When asked what the participants felt they learned solely from participating in the webinar curriculum, participants responded: "details about rotations," "diversity of the program," "atmosphere of training," "what the program values in their residents," "character and personality of faculty and residents which I could never have learned otherwise," and "learning about resident's relationships with their PD and faculty." Overall satisfaction with the webinars was, on average, 9.2 out of 10 (Table 1) . With respect to the resident-only webinar sessions, 100% of participants felt the break-out room format was effective and 97% of participant felt the themes used were effective (Fig. 4) . Overall satisfaction with the resident-led sessions was, on average, 9.3 out of 10 (Table 1) . Participants commented that "it was helpful to see how residents interact with one another" and that residents were "transparent" and their "answers felt honest and real." When asked what topics or additional themes participants would have found useful, participants responded: "IMG room," "virtual tour," and "day in the life of a resident in Rochester." Participants suggested that "longer break-out room time," "more residents from different backgrounds (IMGs)," and "increasing the number of residents that participate" would improve resident-led webinars.

We interviewed 54 applicants for categorical positions and 55 applicants for preliminary positions. Of those 109 interviewees, 46% (50 interviewees) responded to the survey. Overall satisfaction with the interview day was, on average, 4.4 out of 5 and interviewees rated their understanding of the program after the interview day at, on average, 4.6 out of 5 (Table 1) . Interviewees rated their access to faculty and residents highly, at 4.5 and 4.5 out of 5 on average respectively. Interviewees rated their ability to convey themselves highly, at 4.2 out of 5 on average (Table 1) . Only 11 survey participants (22%) experienced technical difficulties, of which 18% (2 interviewees) felt their technical difficulties compromised the integrity of their interview ( Table 1 ). The technical difficulties most commonly encountered involved video recording of technical skills (suturing, knot tying) to upload for asynchronous assessment (8 participants).

While the majority (64%) of participants would prefer inperson residency interviews, 36% of participants would prefer some virtual component to resident interviews in the future (Fig. 5) .

With respect to the live narrated video tour, 98% of participants found the tour helpful (Fig. 5 ). Participants commented that they "absolutely loved the live resident voiceover facility tour," "it was completely beneficial and insightful," "this was fantastic," the tour "exceeded expectations," and "I appreciated having a resident there to commentate and pause when applicants had questions." Participants felt the video tour could have been improved by including a section on housing, a sped-up video showing a day in the life of a surgery resident, and more time and residents dedicated to this session. 

The abrupt transition to a virtual interview cycle in 2020-2021 due to the COVID-19 pandemic challenged general surgery programs to recruit applicants without showcasing their programs in person. By considering pre-existing literature about potential concerns, we were able to successfully implement a webinar curriculum as virtual recruitment prior to the interview cycle. Prospective applicants were highly satisfied with our webinars. Their comments on the aspects of the program they could have only learned during the webinar curriculum highlights the need for events reaching out to prospective applicants when in-person rotations or visits are not possible. Interestingly, while the two resident-led webinar sessions were highly rated in terms of satisfaction, prospective applicants felt that these sessions were least able to give them a subjective feel for our residency program. Instead, prospective applicants developed that feel for our program most through sessions led directly by our Program Director (Program Overview and Last-Minute Q&A with Program Leadership). This information, paired with prospective applicant suggestions of more time and more residents for the resident-led sessions, can provide useful avenues for change to improve the utility of these resident-led sessions in the coming interview cycle. Further, this finding encourages us and other programs to ensure that prospective applicants have time to hear directly from and ask questions to program leadership throughout residency recruitment.

Improving applicant ability to develop a subjective feel for our program was one of our main objectives during residency recruitment. We noted that published editorials and surveys brought up concerns that applicants would not be able to explore this feel for a program well in a virtual format. 7, 8, 10 Luckily, by tackling this problem head-on with a webinar curriculum and multiple aspects of the interview day, prospective applicants and interviewees reported that they were able to develop a good understanding of the program. The feedback that prospective applicants came away from the webinar curriculum feeling that they understood the character and culture of our program well, and that they were given insight into how program leadership interacts with residents, speaks to the value of these sessions.

When designing our interview day, we addressed concerns from previous survey studies of prospective applicants that informal interactions with residents and city/hospital tours could not be accomplished virtually. 8 We created numerous opportunities for interviewees to interact informally with both residents and faculty throughout the interview experience, through the preinterview social hour and during the interview day. Likely because of these experiences, we saw that interviewees rated thorough access to faculty and residents highly. Additionally, we designed a live narrated video tour of our facilities and surrounding town, which interviewees rated highly as exceeding expectations.

Virtual interview cycles are fraught with recruitment challenges, as the lack of visiting clerkships and in-person interviews eliminates avenues for residency applicants to determine their fit with a program. We demonstrate a successful way to virtually showcase a general surgery residency program, through a structured webinar recruitment curriculum, a live-narrated virtual facility and city tour, and by highlighting resident and faculty informal interactions during the interview day.

Conducting interviews during the coronavirus pandemic 2020

Video interviewing: a review and recommendations for implementation in the era of COVID-19 and beyond

Otolaryngology residency interviews in a socially distanced world: strategies to recruit and assess applicants. Otolaryngol Head Neck Surg

How has COVID-19 affected the costs of the surgical fellowship interview process?

Virtual interviews for the complex general surgical oncology fellowship: the Dana-Farber/Partners experience

Virtual surgical fellowship recruitment during COVID-19 and its implications for resident/fellow recruitment in the future

Otolaryngology match 2020-21: survey of prospective applicants in the setting of COVID-19

Urology residency applications in the COVID-19 era

Recruitment and networking with social media for the otolaryngology match in the COVID-19 pandemic

Implications of COVID-19 on the general surgery match

COVID-19 and the radiology match: a residency program's survival guide to the virtual interview season

A simulation-based selection process for trying to identify medical students who will become outstanding general surgery residents