key: cord-0915241-prge3lhh
authors: Labiner, Hanna E.; Anderson, Cristan E.; Maloney Patel, Nell
title: Virtual Recruitment in Surgical Residency Programs
date: 2021-10-04
journal: Curr Surg Rep
DOI: 10.1007/s40137-021-00302-9
sha: 7dd742d37935de899164a785e612aeb741bc3fd1
doc_id: 915241
cord_uid: prge3lhh

PURPOSE OF REVIEW: The Covid-19 pandemic forced residency programs to drastically change their interview processes and adopt virtual interviewing for the 2020–2021 match cycle. RECENT FINDINGS: While virtual interviewing decreased cost and increased convenience for applicants and programs involved in the match, it also introduced several potential disadvantages. Maximizing technological capabilities was an area of utmost concern at the start of the interview cycle, and multiple medical education organizations quickly recommended ways to move to virtual process, and to prevent and troubleshoot technical problems. However, other issues were less straightforward, such as how to address new sources of bias introduced by virtual interviewing, and how to ensure that programs and applicants could make informed decisions about their rank lists after only limited virtual interactions. Additionally, the increased convenience of interviewing raised concerns that students would accept more interviews, disrupting the established calculus programs used to determine how many interviews to offer per spot available. SUMMARY: In this review, we examine the benefits and disadvantages of virtual interviewing, review recommendations from the current literature on how to improve the process, and discuss what we learned from our own experience at an academic general surgery residency program over the course of this unprecedented interview season.

Until March 2020, most medical students and physicians had limited experience with video calls in a professional capacity. The Covid-19 pandemic, however, catapulted the match process into the age of virtual communication, as limiting face-to-face interaction and interstate travel became paramount. In May 2020, the Coalition for Physician Accountability, a cross-organizational group consisting of multiple medical education organizations, including the Association of American Medical Colleges (AAMC), American Medical Association (AMA), Accreditation Council for Graduate Medical Education (ACGME), and others, strongly recommended virtual interviewing for the 2021 Match cycle [1] . In response, residency programs across the country pivoted to create a recruitment season that embraced technology.

There has been much speculation as to the potential benefits and disadvantages of virtual interviews for surgical residency programs and applicants. Areas of particular interest have included technological concerns, financial and scheduling advantages, effects on diversity initiatives, and the impact on match results. In this article, we examine the general expectations and concerns that programs and applicants expressed heading into this unusual match season, expound on how our general surgery residency program adapted to virtual interviewing, and discuss further This article is part of the Topical collection on Surgical Education.

& Nell Maloney Patel malonene@rwjms.rutgers.edu avenues of research to improve the virtual interview process moving forward.

While virtual interviews are relatively new to the field of medicine, the business world has been conducting them for years, and has published advice for interviewees on how to succeed in a virtual setting [2] . Using these recommendations as a backbone, program directors, residents, and medical education organizations compiled lists of tips to help programs and applicants make the most of their interviews [3] [4] [5] [6] . Some common suggestions are for candidates to dress appropriately for the formality of the interview, maintain the camera at eye level, use adequate lighting, keep the phone or computer plugged into the charger throughout the interview, turn off device notifications to avoid being interrupted, ensure a reliable internet connection, and keep the background neutral and clutterfree [2] [3] [4] [5] [6] .

When choosing a platform on which to host our program's virtual interviews, it became clear that we would need the ability to use breakout rooms to accommodate having a Q&A session with our residents in addition to the multiple simultaneous interview rooms. We felt breakout rooms were vital to the success of our interview days to foster informal, spontaneous interactions to allow the candidates to assess the culture of the program. We ultimately settled on using our university subscription for Zoom TM . There are many important factors to consider when choosing a platform. A secure platform is necessary not only to keep hackers out of the interview session, but also to ensure applicants do not unintentionally interrupt each other's interviews. Ensuring the interview session is password protected and using a virtual waiting room can prevent intrusion from hackers and other interviewees [4] . It is also helpful for both interviewers and interviewees to be receive a notification 2-5 min prior to the end of each interview to ensure applicants have adequate time to get their questions answered. We found it beneficial to have the interview automatically end at the allotted time in order to stay on schedule. Understanding the benefits and limitation of each platform is essential in constructing an effective virtual interview day.

Adequate technological support and maintaining communication with both interviewers and interviewees are paramount for running a successful virtual interview. Multiple sources suggest emailing applicants and faculty in advance with information regarding the platform being used to host the interview, the schedule for the day, and the contact information of the program coordinator in case of last-minute technological problems [4, 7] . Applicants should also be advised to test out the platform ahead of time and to familiarize themselves with it [7] .

Prior to the start of our interview season, our residency held a test day where medical students and preliminary residents from our institution were interviewed by our core faculty. The students, residents, and faculty were asked to provide feedback about our virtual interview setup, and our faculty in turn provided feedback to the interviewees about their backgrounds and professional appearance. Our program chose to hold a virtual meeting with faculty the night before interviews to help them test the software and to go over the schedule for the following day. We also provided faculty with the phone numbers of their interviewees in case they got disconnected during their interviews. We did make adjustments throughout the interview season based on feedback from faculty, residents, and applicants. We found this flexibility to be an essential part of our success, and our interview days went very smoothly.

Although the Zoom TM platform worked well for the formal interview sessions, we have discussed that in future we would like to use more creative online platforms to improve the interview process. Our program had beta tested an online virtual world called Sophya TM [8] for previous resident events. This platform, much like massive multiplayer online games, allows each person to create an online avatar and to travel around a virtual space while interacting with the avatars of other users; in this case, residents and faculty. In the Sophya TM platform, walking close to another avatar or group of avatars will bring up a video chat window for each person to allow one-on-one or group conversation. It is also possible to create locked private video chats within the group, which would be conducive to an interview environment. This platform would allow applicants to easily interact with multiple different people and create their own breakout rooms. To our knowledge, this has not yet been tried, but it could improve the applicant's interview day experience by making interactions feel more natural.

Even before the interview season commenced, there was particular concern that the awkwardness of virtual communication would significantly limit the effectiveness of pre-interview social events. Conversation in virtual social events may die down quickly, so it was recommended that programs set an end time in advance to help keep applicants engaged and prevent extended lulls in conversation [4, 9] . Other recommendations for a successful virtual social included maintaining a good ratio of applicants to trainees, providing a dress code, and letting applicants know ahead of time whether they could eat and drink during the event. Breakout rooms were also found to be crucial, as they allow applicants to interact with residents more easily, and also observe how residents interact with each other [4, 9] . Many recommended using room themes to improve the flow of discourse within these breakout rooms. Themes are pre-set topics that can help drive the conversation and prevent applicants from having to hear the same question answered multiple times in different breakout rooms. Good examples of themes include ''what is it like to live or commute in the area'' and ''what do the residents do in their free time'' [4, 9] . We found switching the rooms roughly every 15 min helped keep the conversation flowing while still providing enough time for applicants to have their questions answered. Additionally, after receiving feedback from applicants that they wanted more opportunities to get to know each other, we began implementing a 30-min applicant-only session prior to the start of the social, which was very well received.

Because virtual interview days were generally shorter than they would have been in person [10] , we split our days into separate morning and afternoon sessions. All applicants joined us initially for grand rounds and a presentation from the program director, after which the afternoon group was able to log off until their session began later. This dual session received positive feedback from applicants and faculty.

Some programs recommended decreasing the length of each individual interview due to decreased attention span in a virtual setting [11] , however, others warned that longer interviews were needed for applicants to adequately assess the program [12] . After trialing 30-min and 20-min interviews, we ultimately settled on each candidate having three 25-min interviews with a paired faculty member and resident, as well as shorter ''speed dating'' style interviews with the Program Director, Associate Program Director, and Chairperson. We found that this approach allowed adequate time for interviewers and applicants to assess each other without running out of topics of conversation.

Between interviews, our applicants were put in a breakout room with 2-5 residents who were available to answer questions. The resident room was not compulsory, and applicants were invited to turn off their camera and take a break between their interviews if they preferred. We received feedback from multiple applicants that having the resident room available was helpful and gave them great insight into our program's culture.

Interview season is infamous for being a financial drain on medical students, who generally have significant levels of debt and no income. One of the most obvious benefits of virtual interviews is that no money needs to be spent on travel, accommodations, or food. Most surveys show virtual interviews save applicants roughly $500-800 per program [12] [13] [14] [15] [16] [17] . Applicants also miss fewer clinical days when they do not need to travel, which is particularly important for non-medical student applicants, such as preliminary residents interviewing for categorical positions [14, 15] . Therefore, virtual interviews are not only financially beneficial to applicants, but also prevent disruptions to their learning and may level the playing field for nonstudent applicants. These benefits are not confined only to applicants. Programs also benefit financially from virtual interviews. Whereas our program usually spends approximately $10,000-15,000 on event space and food for interviews, this year we spent $0. Additionally, because virtual interview days are less time consuming, there may be a decrease in lost productivity by surgical faculty [10, 14, 16] .

One of the major concerns going into the 2020-2021 interview season was that the decreased cost and increased ease of interviewing, as well as fears that the unusualness of this application cycle would affect candidates' abilities to match, would lead to applicants applying to and interviewing at significantly more programs as backups [18, 19] . Preliminary data from the AAMC confirm that surgical programs received an average of 1,045.68 applications for the 2021 match cycle, increased from 887.57 in 2020 [20] . This led to concerns that the most competitive candidates could potentially fill up interview spots at an increased number of programs, thus causing other qualified but slightly less competitive candidates to be left without interviews. Programs therefore had legitimate concerns that by interviewing these highly competitive candidates they could end up with unmatched spots [21] . Fortunately, preliminary data from a subset of over 60 surgery residency programs showed that these fears appear to be unfounded, as they did not find an increase in the number of interviews received by the top candidates [22] . General Surgery programs also had to be aware of the increase in applicants for integrated programs in Vascular Surgery, Plastic Surgery, and Cardiothoracic Surgery. Many of these applicants applied to General Surgery Programs as a backup, Curr Surg Rep (2021) 9:25 Page 3 of 7 25 presumably with the General Surgery Programs ending up lower on their rank lists. The dual applicants further confounded the potential for not filling the categorical spots in a General Surgery Program. As part of the National Resident Matching Program (NRMP) rules, programs are not allowed to ask candidates to disclose when they are applying to different tracks, which meant that programs had to be discerning and holistic in their approach when deciding which candidates to offer an interview [23] . Luckily, despite initial concerns, NRMP data showed that categorical general surgery programs did not seem to have a more difficult time matching applicants this year than previously (99.7% both years). Applicants to categorical general surgery programs did have a slightly lower match rate this year (53.7% compared to 56.4% in 2020), but this decrease is likely a continuation of recent trends [17, 24] .

Although it did not appear to affect program match rates, potential solutions to the problem of applicants accepting an increasing number of interviews have been discussed at length in the recent literature. Popular suggestions include limiting the number of interviews applicants are allowed to accept and introducing preference signaling [19] . Preference signaling is a process in which applicants indicate to a pre-specified number of programs that they are particularly interested in that program. Usually, signaling is done naturally (i.e., choosing to do an away rotation, or attending a second look). With constraints in place limiting applicants' abilities to naturally signal their interest, an artificial signaling system could help programs ensure they are interviewing and ranking applicants who are genuinely interested [9, 11, 25] . A third suggestion is for programs to simply interview more people [11] , however, most programs agree this would be an unnecessary hassle with minimal reward, particularly, if the other safeguards are put in place [9] .

With increasing understanding of the importance of diversity in medicine, there has been speculation as to how changes to the interview process would affect socioeconomic, gender, and racial diversity in hiring. While some hoped that the changes would act as an equalizer, others quickly pointed out that the remote interview process could exacerbate established biases and introduce new ones.

Although there is not yet information on match results by gender, a survey of ENT applicants in 2020 suggested that females may be more negatively affected by virtual interviewing than males. In this survey, 70.0% of female applicants felt less confident about their ability to match because of the switch to virtual interviewing, compared to only 47.1% of male applicants [18] . To continue to promote gender equity, it is imperative that future research seek to determine the reasons behind this difference and find ways to address them.

Another concern expressed by applicants was that programs would be more likely to favor those who already had a connection to the program. Although faculty were hopeful that the uniqueness of this season would spark a more holistic review of each candidate, most applicants tended to suspect that programs would instead place more weight on test scores, pedigree, degree type, and connections [18, 19] . Fortunately, cursory data from the NRMP showed that degree type did not appear to carry more weight than in the previous years, as Doctor of Osteopathic Medicine (DO) students actually had a slightly increased rate of matching into a categorical general surgery position compared to last year (62.13% from 59.4%). Additionally, graduate applicants appear to have had a slightly higher match rate this year than previously. This may support that programs did in fact perform more holistic reviews of candidates. However, concerns that international medical graduates (IMGs) would particularly suffer in this cycle did appear to be supported by preliminary NRMP match data, which showed IMGs fared slightly worse than in 2020 [17, 24] . It was postulated that IMGs were disproportionately affected by the limited capacity for away rotations, which are usually essential for networking and helping them establish themselves as competitive candidates [9] .

One way virtual interviews may promote diversity in hiring is that the decreased cost of interviewing could help level the playing field for applicants without stable finances. In the past, these students would have been unable to attend as many interviews as their peers, who may be getting financial support from family [11] . There is, however, concern that students who are unable to pay for highspeed internet or expensive camera and audio equipment will make a poorer impression on a virtual interview, an issue that they would not have in person [26] . Additionally, applicants with children, particularly single parents of young children, are at a significant disadvantage interviewing virtually during a pandemic. It is generally recommended that applicants who cannot control background noise perform their interviews from the medical school library [26] . However, with remote learning and pandemicrelated issues in obtaining childcare, asking an applicant to separate themselves from their children may not be straightforward. Instead of putting the onus on applicants to protect themselves from discrimination, programs should encourage interviewers to be cognizant of their own biases and should support the applicants who would be affected by them. Of note, there is a paucity of data published on the effects of virtual interviews on ableist hiring biases, but this is an important topic that will require further research. Similarly, it is of vital importance that future studies investigate how virtual interviews impact racial hiring biases in order to improve equitability.

One of the most daunting aspects of virtual interviewing is the difficulty of making an informed decision based on only a few hours of video calls. For applicants, resident morale and the culture of the program are two of the most important factors for determining rank list order [27] . Many are therefore concerned they will not be able to adequately assess programs without having the opportunity to interact with residents and faculty [14, 15, 18] . Because of this, virtual social events with trainees are considered an absolute necessity during a virtual interview season [4, 9, 11] .

Over the course of an in-person interview day, applicants interact with multiple different faculty members and residents in a less formal manner between interviews as well as at lunch. During a virtual interview day, however, applicants may only interact with the faculty and residents who formally interview them. With a decreased quantity of interactions, applicants are more likely to assume that a single interaction reflects more widely on the program as a whole, a phenomenon known as signal distortion [7] . Signal distortion may also affect how programs evaluate interviewees, particularly because of the high priority placed on interpersonal skills. The NRMP Program Director Survey from 2020 found that program directors cited interpersonal skills, interactions with faculty during interview and visit, interactions with housestaff during interview and visit, and feedback from current residents among the top 6 most important factors in ranking applicants [28] . Therefore, each interaction between a resident or faculty member and an applicant will carry more weight than it would during an in-person interview day, which could drastically affect how applicants rank programs and vice versa.

Additionally, programs are concerned about their ability to be able to adequately portray themselves to applicants [11] . To combat this, it is recommended that programs have an updated program website and active social media presence, as this helps applicants better understand a program's culture and whether it would be a good fit for them [4, 11, 29] . Savvy programs were even able to use social media analytics to measure the impact of their program on platforms like Twitter TM and Instagram TM . Programs that utilized creative videos and hashtags to improve their visibility in the virtual world found that these initiatives were received positively [30] . Applicants also overwhelmingly agree that programs should have a virtual Q&A session with the program director and provide virtual tours of the facility and surrounding area to help them better assess culture and fit [7, 11, 18, 31] .

Interestingly, some of our preliminary residents who interviewed virtually for categorical positions said they felt virtual interviews were sufficient for adequately assessing a program. This may be because they already had a strong understanding of how a program should function, so could focus on assessing only the aspects that mattered most to them. In general, our preliminary residents said they drastically preferred the virtual interviews to in-person interviews because of decreased cost and travel time. Therefore, in the future, this group in particular may benefit from being offered the ability to interview virtually.

The Covid-19 pandemic has necessitated many changes to both the match process and surgical education over the past year, and the effects of these changes on resident recruitment remain to be seen. Some suspect that, as a result, there may be increased attrition over the next few years. Because of the difficulties in assessing program culture over a virtual platform, there may be an increased number of residents who find that their new program is not a good fit for them [9] . Given that applicants feel social media is an important factor in understanding a program's culture [11, 29] , it will be interesting to see whether there was an application bias towards programs with a more sophisticated online presence, and whether applicants who matched these programs feel like the program was adequately portrayed.

Additionally, there is a concern that curriculum changes necessitated by Covid-19 may leave graduating medical students ill-prepared for their intern year, thereby increasing burnout. Away rotations in 2020 were largely postponed except in extenuating circumstances [1], which may have limited the number and breadth of surgical rotations students were able to experience. A survey of surgical residents found that those who did not do surgical subinternships or take overnight call as medical students were more likely to report feeling underprepared at the start of residency. Even more concerning, this was significantly associated with increased risk of burnout [32] . This suggests that the difficulties of the 2021 match cohort are not over. Incoming residents may need more support and guidance than in the previous years, and their faculty and co-residents should be encouraged to assist them and be vigilant for signs of burnout. 

While virtual interviewing was done out of necessity this year, there appear to be many benefits, particularly, with regard to overall convenience and decreased financial strain. While there are many potential disadvantages to virtual interviews, none of them appear insurmountable. Now that the infrastructure is in place, programs should consider how best to use virtual interviewing in the future. Many different possibilities have been proposed, such as offering virtual interviews to applicants already familiar with the program (i.e., from an away rotation) or who have already completed an intern year at another program. It has also been suggested that programs could offer a round of preliminary interviews virtually before inviting a subset of applicants for in-person interviews to help keep costs down [12] . However, more research is needed on systemic bias in virtual interviewing to mitigate inequalities within the hiring processes prior to making any permanent changes. The 2020-2021 interview season was a challenging one for all involved, but the lessons learned over the past year have the potential to revolutionize future residency hiring practices.

Funding There are no sources of funding.

Data availability Not Applicable.

Code availability Not applicable.

Conflict of interest There are no conflicts of interest.

Consent to participate Not applicable.

Ethical approval Not applicable.

have been highlighted as: • Of importance •• Of major importance 1. A. Acgme, Ama, Aogme, Aamc, Cmss/Opda, Ecfmg, Nrmp. Final Report And Recommendations For Medical Education Institutions Of Lcme-Accredited

9 tips for mastering your next virtual interview, In business insights

The plastic surgery residency interview revisited: virtual interviews and beyond

One of the first compilations of interviewee experiences with virtual interviewing for surgical fellowship programs. Provides recommendations to programs and to applicants on how to portray themselves well over a virtual medium

Virtual interviews: tips for interviewers

Virtual interviews: applicant preparation guide

A review of the literature on communication challenges with virtual interviewing and suggestions for mitigating them

Virtual interviews as part of residency recruitment

Otolaryngology residency interviews In A socially distanced world: strategies to recruit and assess applicants

Virtual surgery residency selection: strategies for programs and candidates

Novel Implementation of virtual interviews for otolaryngology resident selection: reflections relevant to the covid-19 era. Oto Open

Live online video interviews dramatically improve the plastic surgery residency application process

The perception and costs of the interview process for plastic surgery residency programs: can the process be streamlined?

Virtual interviews for the independent plastic surgery match: a modern convenience or a modern misrepresentation?

Using skype as an alternative for residency selection interviews

Advanced Data Tables 2021 Main Residency Match

Otolaryngology Match 2020-21: survey of prospective applicants in the setting of covid-19

Potential implications of covid-19 for the 2020-2021 residency application cycle

Eras Statistics Preliminary Data

The case for capping residency interviews

Resident Recruitment Crisis? Data Says, Not So Much

Match Participation Agreement For Applicants And Programs 2021 Main Residency Match

Results And Data 2020 Main Residency Match

Residency match during the covid-19 pandemic: the clear and present danger of the remote interview

Virtual interviewing tips for the otolaryngology match cycle during the covid-19 pandemic. Oto Open

Applicants' self-reported priorities in selecting a residency program

Nrmp, Results of the 2020 Nrmp Program Director Survey. 2020, Nrmp Data Resease and Research Committee

Recruitment and networking with social media for the otolaryngology match in the covid-19 pandemic. Otolaryngol Head Neck Surg

When residency interviews turned virtual: improve program visibility and virtual interview process

Improving the residency program virtual open house experience: a survey of urology applicants

A National mixed-methods evaluation of preparedness for general surgery residency and the association with resident burnout

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations