key: cord-0856167-68ubz6jx
authors: Hamade, Nour; Bhavsar-Burke, Indira; Jansson-Knodell, Claire; Wani, Sachin; Patel, Swati G.; Ehrlich, Adam C.; Paine, Elizabeth; Hosseini-Carroll, Pegah; Menard-Katcher, Paul; Fayad, Nabil
title: Virtual Gastroenterology Fellowship Recruitment During COVID-19 and Its Implications for the Future
date: 2021-05-04
journal: Dig Dis Sci
DOI: 10.1007/s10620-021-07014-1
sha: 01244ee0613e39091b8aaaad9edc3f66cead1a0f
doc_id: 856167
cord_uid: 68ubz6jx

BACKGROUND AND AIMS: Amid the COVID-19 pandemic, medical education organizations endorsed a virtual recruitment format, representing a stark change from traditional in-person interviews. We aimed to identify the attitudes and perceptions of Gastroenterology Fellowship Program Directors (PDs) and applicants regarding the virtual interview experience and the role of virtual interviews (VI) in the future. METHODS: We designed separate surveys targeting PDs and applicants using the Qualtrics software. At the end of the interview season, we e-mailed both survey links to all PDs and requested that they forward the applicant survey to their interviewed candidates. Surveys were voluntary and anonymous. Descriptive statistics were used to analyze the data with results presented as percentages. RESULTS: A total of 29.7% of PDs completed the survey. Compared to traditional interviews, VI were viewed by 46.5% of PDs to be very suboptimal or suboptimal. Yet, 69.1% envisioned a role for VI in the future. A total of 14.2% of applicants completed the survey. Compared to traditional interviews, VI were viewed by 42.3% of applicants to be very suboptimal or suboptimal. However, 61.8% saw a future role for VI. While both applicants and PDs reported that establishing an interpersonal connection was a disadvantage with VI, applicants placed more emphasis on this need for connection (p = 0.001). CONCLUSION: Overall, PDs and applicants report mixed views with regard to VI but anticipate that it may continue to have a future role. VI may augment future recruitment cycles with care taken to not disadvantage applicants, who rely heavily on the interview process to create personal connections with programs. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s10620-021-07014-1.

The emergence of the coronavirus disease-19 has necessitated significant changes in the healthcare system and medical education. Due to the travel restrictions and social distancing recommendations related to the COVID-19 pandemic [1] , the usual interview process was not feasible. The Association of American Medical Colleges (AAMC) endorsed virtual formats [1] , which are notably different from in-person formats used in prior years. Anecdotally, this presented a challenge for programs. There is an established role for virtual learning [2] and there may similarly be a role for virtual interviews (VI) in the future. While several studies were published recently on this topic in different specialties, most had a small number of participants, with some showing conflicting results with regard to the preference of participants as to in-person versus VI, and the effect this has had on the applicants' and programs' ranking process [3] [4] [5] [6] . In this study, we gathered the perceptions and evaluations from program directors (PD) and applicants who participated in the virtual Gastroenterology fellowship match, with the aim to inform the role and best practices of VI in future Gastroenterology recruitment seasons.

This study was a cross-sectional, nationwide survey targeting Gastroenterology program directors and fellowship applicants. We constructed two separate online surveys, one for program directors and one for applicants, using the webbased survey platform, Qualtrics (Seattle, WA).

At the end of the fellowship interview season, both surveys were distributed by e-mail to all program directors via the American Gastroenterological Association listserv of program directors [7] . Program directors were in turn asked to forward the applicant survey link to all the candidates who interviewed at their programs, with assistance from their program coordinators. Surveys were first distributed on October 1, 2020 and subsequent reminder e-mails were sent weekly. Five reminders were sent through the end of the study period on November 7, 2020, when surveys were closed to any further responses.

After the final match results were announced on December 2nd, 2020, a brief follow-up survey was sent by e-mail to PDs to gather additional information about these results. A single reminder e-mail was sent the following week on December 7, 2020.

The study was granted exemption by the Indiana University Institutional Review Board. Participants did not receive any financial compensation for completion of the surveys with participation being entirely voluntary. All survey results were anonymous to not influence rank lists or match results. Given the continuously evolving situation during the COVID-19 pandemic, the many competing demands of fellowship applicants and program directors, and the overlap of the time frame of fellowship interviews relative to the COVID-19 pandemic, we set our a priori expected response rate at 20%, as we anticipated these factors would restrict participation.

The program director survey consisted of 21 questions. These addressed basic information about the program (location, academic affiliation, number of positions available), logistical information on the 2021 Electronic Residency Application System (ERAS) season (number of interviews offered, number of interview days, number of applicants interviewed, etc.), impressions about the VI season (advantages, disadvantages), and opinions regarding the potential role of VI in subsequent interview seasons (supplementary material-program director survey). Questions were included regarding the prior interview season (2020 ERAS application cycle), to both serve as a control and to better understand any differences between the virtual and traditional interview formats. For clarification, the 2021 ERAS season marks Gastroenterology fellowship positions to start in July 2021, whereas the 2020 ERAS season represented the match that resulted in fellows who started in July 2020.

The applicant survey consisted of 17 questions. These questions covered demographic information (age, gender, race, geographical location), logistical questions about the application process (number of programs applied to, number of interviews offered and accepted), impressions about the VI format (advantages, disadvantages), and finally opinions on the future of VI in subsequent application cycles (supplementary material-applicant survey).

In both surveys, whenever applicable, a five-point Likert scale was used to ask participants to rank their answers from 1 to 5 (1-strongly disagree, 2-somewhat disagree, 3-neither agree nor disagree, 4-somewhat agree, 5-strongly agree). A consent form preceded each survey and continuation to engage in survey participation served as informed consent.

To ascertain the face and content validity of both surveys, pilot testing was performed to query the appropriateness and relevance of the questions, as well as the time needed to complete each survey. For the PDs survey, a subgroup of co-authors on this manuscript-including three program directors-(S.W., S.P., A.E, E.P.,N.F.), pretested and edited the survey to ensure that questions addressed pertinent aspects of the interview process and covered the outcomes being measured. For the applicant survey, a group of five residents applying to the fellowship match of other specialties were asked to pretest the survey and provide feedback on the questions as they relate to the aim of the paper. Each survey took less than 10 min to complete by the pilot groups.

The brief follow-up survey sent to PDs included two questions to collect data on the proportion of positions at each institution that matched internal candidates from the residency program at the same institution this year, compared to that proportion in the 2020 ERAS application cycle. This was intended to further assess whether VI affected the rate of matching applicants from within or outside the candidates' residency program institutions. For the follow-up survey, a similar consent form and process were used as for the initial surveys.

For the PDs' and applicants' surveys, to ensure that respondents do not submit multiple survey responses, we utilized a feature of the Qualtrics platform that limits each participant to one response only.

Data were summarized using descriptive statistics. Continuous variables were reported as mean ± standard deviation (SD). Absolute (count) and relative (percentage) frequencies were given for survey responses. Continuous variables were analyzed by a 2-sample t-test when appropriate, while qualitative variables were compared using a Chi-square test.

In addition to the above analysis, we performed a comparative analysis of pertinent corresponding questions between the applicant and program director surveys. This included analysis of responses related to how VI compared to in-person interviews, to the role of VI in the future, and to the perception of certain disadvantages of VI. Comparison of these data was conducted using a two-sample t test and Chi-square test.

Furthermore, to assess how representative the program director respondents were of US Gastroenterology programs, we ran a comparative analysis to general demographic data of US programs extracted from the FREIDA™ Residency Program Database [8] . A p-value of < 0.05 was considered statistically significant. Statistical analysis was performed using IBM SPSS version 27 (IBM, Armonk, NY).

The response rate of program directors was 29.7%, with 60 PDs completing the survey out of the 202 Gastroenterology fellowship program directors throughout the USA. This fulfilled the a priori expected response rate, with 21.3% completing the survey in its entirety. Most respondents worked in tertiary-care or academic-affiliated institutions (n = 55, 91.7%). The majority of program directors were from programs located in the Northeast (n = 21, 35.0%) with relatively equal distribution among other regions. The average number of fellowship positions offered per program was 3.7 (SD 1.5). Baseline characteristics from the PD survey are included in Table 1. VI were offered by 100% of programs whose PDs participated in the survey. One-on-one interviews were the most commonly used format and accounted for 74.1% of VI. Most programs used Zoom® as their interview platform (67.4%). Electronic promotional materials were distributed by 76.7% of program director respondents to showcase their training programs.

There was no statistically significant difference between the average number of interviews offered (p = 0.901), the average number of applicants interviewed (p = 0.251), or the average number of interview days offered (p = 0.424) between the 2020 and 2021 ERAS application cycles. There was a statistically significant difference for the programs in the number of applications received, with more programs receiving 400 + applications in the 2021 season as compared to the 2020 season (p < 0.0001). These data are presented in Table 2 . The post-Match PD survey also showed that the proportion of matched internal applicants in each program was not significantly different between the 2020 (40.1%, SD 24.9) and 2021 (36.2%, SD 24.8) ERAS application cycles (p = 0.446).

The top three advantages of VI reported by program directors were conservation of funds (83.3%), accommodation of applicants (83.3%), and ability to conduct interviews after hours or from home (66.7%). The most common drawbacks were inability to give a tour of the institution (97.6%), inability of applicants to explore the city (90.5%), and inability to make a personal connection with the applicant (85.7%) ( Table 3) .

Compared to traditional interviews, VI were viewed to be very suboptimal or suboptimal by 46.5% of respondents. More than half of PDs (54.8%) envisioned a continued role for VI in the future, with 76.9% of those believing that VI will take on a supplementary role to traditional interviews either for applicants who cannot attend face-to-face interviews or for those who are invited for a second-look interview.

The response rate of applicants was 14.2% (126 respondents out of 885 applicants, based on the National Resident Matching Program 2020 Match results statistics-appointment year 2021) [9] , with 104 candidates completing the entire survey. All 100% of respondents participated in VI. The mean age of applicants was 29.4 years (SD 1.6). Most respondents identified as White (37.0%) or Asian (39.4%), and 83.2% of respondents identified as US Medical Graduates. Applicants were evenly distributed across the different geographic regions in the nation. The average number of programs that each applicant applied to was 74.4 (SD 48.6). The average number of interviews offered was14.2 (SD 6.9). The average number of interviews that applicants attended was 11.6 (SD 4.1) ( Table 4) .

Most applicants had previous web-based experiences (n = 74, 70.5%), with 95.2% having a readily available computer with a good quality camera, and high-speed Internet access and 93.3% having access to an appropriate environment with a professional background in which to conduct their interviews.

Applicants reported the major advantages of VI compared to traditional interviews to be the cost savings (100.0%), as well as the ability to interview at more programs either because of less time off work needed (91.3%) or less conflict in scheduling different interviews (76.7%). Applicants reported that the main disadvantages of VI were the inability to visit and learn about the city where the program is located (96.1%), inability to observe faculty and fellows interact (92.2%), and inability to take a physical tour of the program facilities (84.5%). Notably, 34.3% cited technical difficulties with using video technology ( Table 5) .

As compared to in-person interviews, VI were viewed to be very suboptimal or suboptimal by 42.3% of applicant respondents. Most applicants (61.8%) who completed the survey envisioned a continued role for VI, with 79.8% of those believing that VI will take on a supplementary role to traditional interviews in the future.

There was no statistically significant difference in the responses of applicants and PDs with regard to their assessment of the VI process compared to the traditional interview format (p = 0.36). Both applicants (50.5%) and PDs (64.3%) believe that VI will not fully replace in-person interviews during future application cycles (p = 0.48). However, both applicants and PDs felt that VI could play a supplementary role to in-person interviews in the future, with no statistically significant difference between both groups There was no statistically significant difference in the perceptions of applicants and PDs regarding several disadvantages of VI, such as complexity of scheduling (p = 0.14), technical challenges (p = 0.73), ability to interact with fellows (p = 0.7), or ability to tour the city where the program is located (p = 0.14). While both applicants and PDs did report that establishing an interpersonal connection was a disadvantage of VI, applicants placed more emphasis on this being a disadvantage as compared to PDs (p = 0.001) ( Table 6 ).

The COVID-19 pandemic has forced major change in the Gastroenterology fellowship interview process this year, transforming it into an entirely virtual experience. This was a significant change from previous years in which the recruitment season had been conducted primarily in-person. As such, Gastroenterology programs were faced with many unknowns in adopting a new VI platform. While this was an unprecedented change for Gastroenterology programs, several other US residency and fellowship programs had adopted and published on virtual interviewing, both prior to [10] [11] [12] [13] [14] and during the COVID 19 pandemic [3, 4, 15, 16] . While these studies showed that VI have reduced costs for applicants and residency programs [11, 15, 17] and found favorable faculty satisfaction [3, 4, 12, 18] , applicants' perceptions varied. Some studies reported that applicants perceived VI as less effective than in-person interviews in letting them accurately express themselves, or tour the facility and/or city, and thus made them less comfortable in ranking the program [3, 11, 19] . However, these negative perceptions were mitigated by other studies, which provided applicants access to a video tour of the medical center and surrounding city, online information regarding the program, optional post-interview departmental tours, and virtual sessions with current residents [3, 4, 20] .

Our study is the first and largest of its kind to gather the opinions and impressions of Gastroenterology fellowship program directors and applicants regarding VI in a real-world setting. We were able to meet our a priori set threshold for response rate and gather data from over 20% of PDs across the nation. While we did fall short in meeting that threshold with interviewees, we were still able to gather responses from over 120 applicants, making this one of the largest surveys aimed at gathering data on a VI experience for Gastroenterology fellowship recruitment.

Not surprisingly, applicants and program directors agreed on many of the advantages of VI such as decreased cost and more convenient scheduling. In addition, the majority of applicants had previous web-based experience and access to an appropriate environment and video technology to conduct the interviews, indicating that adoption of this format is accessible to most applicants. Interestingly, while there was no major difference in terms of average number of applicants interviewed during this year's recruitment process compared to last year, programs received a larger number of applications. This increase indicates that the virtual platform encouraged applicants to apply to more programs as had been predicted [21] , but did not encourage programs to interview more candidates. Thus, while our data showed that 67% of applicants applied to ≥ 60 programs, 60% of applicants were offered ≤ 14 interviews. The average number of interviews that applicants attended was 11.6 (SD 4.1), with attendance rate of 81.7% or four out of five interviews offered indicating that applicants were likely to attend most interviews they were offered. Another concern had been that external candidates were less likely to rank programs they have not visited or that programs were less likely to rank an applicant they had never met face-to-face; however, our data showed that there was no difference in the number of internal candidates matched between the last two application cycles. However, there were disadvantages to VI, mostly related to barriers to social interactions between applicants and interviewers, as well as the inability of applicants to physically tour the facilities of the program. These were still viewed as disadvantages despite most programs reporting that they had expanded their online and social media presence by creating videos of their facilities to enhance the applicants' exposure to the program. These barriers seemed to have affected applicants disproportionately more than programs, as applicants felt that they could not express their unique personal traits sufficiently over a web-based interface. While this finding has not been echoed by other similar studies performed in the surgical specialty interview match [3, 4] , this may potentially be explained by the observation that applicants participating in a highly competitive subspecialty match such as Gastroenterology have been shown to perceive more pressure during the interview process compared to programs [22, 23] .

Despite these discrepancies, a large percentage of both applicants and PDs felt that there will be a continued role for VI in the future, corroborating findings in other studies [3, 4] . In our study, most respondents reported that they felt the role for VI will be more of a supplementary one to in-person interviews. This option may be considered when geographic location and/or financial situations limit the ability of candidates to be present for in-person interviews.

It seems reasonable to suggest that VI will be used in future fellowship recruitment seasons, though the extent of their utility remains unknown. We were able to identify areas that will require further optimization to improve the experience of those participating in VI. Given that at least a third of both applicants and PDs reported technical difficulties, it will be prudent to optimize the technology and test it before interview day. Programs should have troubleshooting options provided ahead of time; they should have a technology assistant or program coordinator available on interview day who can be contacted directly when problems arise [24] . Exposure to the programs' facilities and cities was also reported disadvantages, and as such, programs should produce video tours of their facilities and offer program overview material available online to applicants [25, 26] . Beyond the pandemic, when travel is less of a restraint, programs can also offer optional in-person tours after interviews for those applicants who are particularly interested.

Perhaps the most important issue to consider regarding the disadvantages of VI include applicants' reported inability to express their unique personality traits or have proper interactions with fellows in the program [27] . To help ameliorate this, we recommend residency programs offer virtual mock interviews for fellowship applicants prior to the interview season to help both residents and faculty practice VI skills beforehand and get comfortable with expressing themselves in a virtual space. Attention to eye contact, hand gesturing, appropriate lighting, and facial expressions are all key components that applicants can practice to help them relate better to faculty interviewers [28] . In order to address applicants' concerns regarding limited interaction with current fellows, programs should consider offering a private meeting for applicants to interact virtually with current fellows, scheduled before or during the interview day.

It is important to note that this study included several limitations. Most of the program directors who answered the survey were from tertiary-care or academic-affiliated institutions, which may have introduced selection bias, as our analysis showed that this is not entirely representative of US Gastroenterology programs (p = 0.04). Our sample also contained a larger proportion of programs with 4 or more fellowship positions as compared to the general distribution of US Gastroenterology programs, which may have also biased our results (p = 0.01) ( Table 1) . A notable limitation was the low response rate to the surveys which limited the representativeness of our data and conclusions. This was more pronounced for the applicant survey which may have been due in part to the method of dissemination, as it relied on the programs sending out e-mails to their interviewees. Thus, we could not control whether these surveys, or any of the reminders, were indeed sent out by the programs. Additionally, a potential confounding factor might have been that applicants who were sent the survey were from programs whose director was motivated to respond to the survey, and since the PDs who responded to the survey were not necessarily a representative sample, this may have also influenced the representativeness of the applicants as well.

In conclusion, it is unclear at this point whether VI will remain a future necessity due to the ongoing pandemic or whether they may become the new normal even after COVID-19. Our survey indicates that there will likely be a role for VI in the future, with both applicants and program directors agreeing in this area. Certain shortcomings will have to be overcome before this platform can approximate the full experience of an in-person interaction. Further research exploring the specific techniques utilized on interview day will be meaningful in understanding the actual VI setup. Ultimately, whether videoconferencing will take on a supplementary role or a main role in the future of recruitment interviews remains to be seen.

The online version contains supplementary material available at https:// doi. org/ 10. 1007/ s10620-021-07014-1.

Author's contribution NH contributed to study concept and design; drafting of surveys; acquisition of data; analysis and interpretation of data; and drafting of the manuscript. IB-B contributed to critical review of surveys; statistical analysis; interpretation of data; drafting of the manuscript; and critical review of the manuscript. CJ-K contributed to analysis and interpretation of data; drafting of the manuscript; and critical review of the manuscript. SW contributed to critical review of surveys and critical revision of the manuscript for important intellectual content. SP contributed to critical review of surveys and critical revision of the manuscript for important intellectual content. ACE contributed to critical revision of the manuscript for important intellectual content. EP contributed to critical revision of the manuscript for important intellectual content. PH-C contributed to critical revision of the manuscript for important intellectual content. PM-K contributed to critical revision of the manuscript for important intellectual content. NF contributed to study design; critical review of surveys; critical revision of the manuscript for important intellectual content; and study supervision.

Funding Not applicable.

Conflict of interest None.

Association of American Medical Colleges. Conducting interviews during the coronavirus pandemic

Adapting a GI fellowship to a pandemic: novel approaches to accommodating a novel virus

Virtual surgical fellowship recruitment during COVID-19 and its implications for resident/fellow recruitment in the future

Program and candidate experience with virtual interviews for the 2020 Complex General Surgical Oncology interview season during the COVID pandemic

Graduate medical education virtual interviews and recruitment in the era of COVID-19

Planning virtual residency interviews as a result of COVID-19: insight from residency applicants and physicians conducting interviews

American gastroenterological association

Match Results Statistics: Medical Specialties Matching Program-Appointment Year 2021

Interview from anywhere: feasibility and utility of web-based videoconference interviews in the gastroenterology fellowship selection process

Randomized evaluation of a web based interview process for urology resident selection

Residency interview video conferencing

Feasibility and usability of tele-interview for medical residency interview

Live online video interviews dramatically improve the plastic surgery residency application process

How has COVID-19 affected the costs of the surgical fellowship interview process?

Virtual interviews for the complex general surgical oncology fellowship: the dana-farber/ partners experience

Using skype as an alternative for residency selection interviews

Videoconference interviews for an adult reconstruction fellowship: lessons learned

Radiology resident recruitment: A study of the impact of web-based information and interview day activities

Comparison of web-based and face-to-face interviews for application to an anesthesiology training program: a pilot study

Potential implications of COVID-19 for the 2020-2021 residency application cycle

Planning for Virtual Interviews: Residency Recruitment During a Pandemic

Increased average number of medical publications per interviewee from 2009 to 2018: a study of 100 interviewees to an academic gastroenterology fellowship program

Video interviewing: a review and recommendations for implementation in the era of COVID-19 and beyond

The new virtual reality: how COVID-19 will affect the gastroenterology and hepatology fellowship match

Use and utility of Web-based residency program information: a survey of residency applicants

Evaluating factors and resources affecting ranking of diagnostic radiology residency programs by medical students in 2016-2017

Videoconference interviewing: tips for success