key: cord-0776644-cxryy2gl
authors: Taparra, Kekoa; Ebner, Daniel K; Cruz, Denise De La; Holliday, Emma B
title: The Impact of COVID-19 on Radiation Oncology Residency Applicant Away Rotations, Interviews, and Rank Lists: A Comparison Between the 2020 Match and 2021 Match
date: 2021-10-29
journal: Adv Radiat Oncol
DOI: 10.1016/j.adro.2021.100842
sha: be0a5179d4b72bc23f3263a16de260391ac135b0
doc_id: 776644
cord_uid: cxryy2gl

BACKGROUND: The COVID-19 pandemic modified the Residency Match process for fourth-year medical students. In-person away rotations were discouraged, interviews were virtual, and traditional factors used to rank programs were absent. Here, we compare survey results administered to both the 2020 and 2021 Match applicants to assess the influence of the pandemic on the RO Match process. METHODS: An Institutional Review Board approved prospective cross-sectional study was conducted. The 2020 and 2021 RO Match applicants at a large RO program were invited to participate. Descriptive summary statistics were assessed. RESULTS: The 2020 and 2021 Matches each had seventy-six applicants complete the survey with response rates of 54% and 57%, respectively. The two groups were predominantly white, cisgender male, single, and without children. While 11% of 2020 applicants did not complete away rotations, 45% of 2021 applicants did not. For 2021 Match applicants, 65% of away rotations were performed virtually while 51% were not for medical school credit. 84% of applicants were satisfied with virtual interviews and 72% felt cost savings were worth not having in-person interviews. While 49% of Match 2020 applicants spent >$5,000 in interview costs, 0% of the Match 2021 applicants did so, with 45% spending <$100. Post-interview communications from programs increased during the pandemic from 36% to 42% in 2020 Match and 2021 Match, respectively. While program culture was the most common factor influencing 2021 Match applicants program rankings, half of applicants did not gain a sense of program culture during virtual interviews. CONCLUSIONS: We find 2021 Match applicants completed fewer away rotations, were satisfied with virtual interviews/reduced costs, and did not gain a sense of program culture through virtual rotations/interviews despite it being the most important ranking factor reported. This study supports further exploration of virtual away rotations and virtual interviews moving forward beyond the pandemic.

important ranking factor reported. This study supports further exploration of virtual away rotations and virtual interviews moving forward beyond the pandemic.

Due to the coronavirus disease 2019 (COVID-19) pandemic, the Association of American Medical Colleges discouraged clinical rotations at outside institutions ("away rotations") while recommending online interviews (1) .

Traditionally, in-person away rotations served as opportunities for applicants to learn about other residency programs. Moreover, residency program directors report away rotation evaluations as the most important metric for offering interviews (2) .

Interviews are traditionally in-person and impact programs' Match rank order lists (3, 4) . Few studies have evaluated online/virtual residency interviews, though some data suggest they may enhance the applicant pool by reducing travel costs, scheduling conflicts, and educational interruptions (5) . Gamesmanship through letters-ofinterest and "rank-to-match" letters complicate the Match process, with post-interview communication providing some applicants unfair advantages (6) .

The drastic changes to the Radiation Oncology (RO) Match process offer a window both for study of equitable access to virtual away rotations as well as the opportunity afforded by online interviews. This study evaluates COVID-19-related changes in the RO Match Process including away rotations, interviews, and program ranking through survey of the 2020 and 2021 Match applicants.

An Institutional Review Board approved prospective cross-sectional study was conducted by surveying RO applicants during the 2019-2020 (Match 2020) and 2020-2021 (Match 2021) cycles. The applicants to a single large, accredited RO residency program were surveyed following Match Day by email. Consent was obtained electronically, participation was voluntary, and responses were anonymous.

Participants were sent a non-validated online survey as previously reported (7), distributed using the REDCap Survey tool (2013, Vanderbilt University). Questions sent to both groups focused on applicant demographics, RO rotations, interviews, factors impacting applying or ranking programs, and post-interview communications (Supplemental Table 1 ). New questions were added to the 2021 survey pertaining to perspectives and satisfaction with virtual away rotations and online interviews.

Descriptive summary statistics were tabulated using chi-square and Fisher's exact tests. All tests were two-tailed with a significant P-value threshold of 0.05. All statistical analyses were conducted using R v4.0.3 in RStudio v1.3.1093 (8) .

Fifty-seven percent (76 of 133) of 2021 Match applicants completed the survey. There were no significant demographic differences between the 2020 and 2021 Match cohorts ( Table 1) . The 2021 survey respondents were primarily white (52%), cisgender male (60%), single (39%), without children (88%), and from the Northeast or South regions. Two-thirds of applicants reported having a radiation oncology program in their medical school.

For 2021 Match survey respondents, 45% did not complete any away rotations compared with 11% of 2020 applicants (Figure 1) . Among 2020 and 2021 Match survey respondents who did not participate in away rotations, 75% and 73%, respectively, expressed desire to do so and 50% and 52%, respectively, felt not completing an away rotation impacted their rank. Most away rotations were performed virtually (65%) and were not for medical school credit (51%; Table 2 ). One-in-five applicants did not gain a sense of culture via virtual away rotations.

Factors impacting which programs applicants applied to were statistically unchanged: general reputation (88%), location (86%) and mentor recommendation (66%) remained the most influential factors (Supplemental Table 2) (7). Factors impacting 2021 Match applicant ranking of programs also included Twitter (11%), Student Doctor Network (7%), and the anonymous Google Spreadsheet (32%).

The 2021 Match survey respondents applied, offered, and accepted a median of 20 (interquartile range [IQR], 13-29), 16 (IQR, [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] , and 14 (IQR, [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] interviews, respectively. Figure 2 summarizes attitudes towards virtual interviews for the 2021 cohort due to the COVID19 pandemic. 84% of applicants were satisfied with virtual interviews and 72% felt the cost savings were worth it. Only 51% reported gaining a sense of program culture from virtual interviews. Interview costs were significantly reduced for 2021 applicants, with 49% of 2020 Match versus 0% of 2021 Match applicants spending >$5,000 in interview costs. 45% of 2021 Match applicants spent

Post-interview communications by programs to survey respondents increased from 36% in 2020 to 42% in 2021 (Supplemental Table 3 ); no other significant differences were noted, with 58% of 2021 applicants reported inappropriate questioning, 33% reported post-interview "rank-to-match" correspondence, 79% reported writing a "top choice" letter of interest, 25% reported submitting more than one letter of interest, and 38% reported mentorassisted communications 7 (Supplemental Table 3 ). Most factors impacting how survey respondents rank programs remained unchanged (Figure 4 ; Supplemental Table 2 ), although department facilities and technology impact increased from 36% to 58% (p=.009) and department website impact increased from 13% to 36%

(p=.002). For 2021 Match, 33% also reported virtual away rotations and 63% reported resident achievements as factors impacting how they ranked programs.

This RO Match Survey compares perspectives on away rotations, interviews, and ranking practices of the pre-COVID-19 2020 Match and 2021 Match survey respondents. Due to COVID-19 restrictions, many 2021 Match applicants did not complete away rotations. Virtual away rotations were created to provide an alternative, but many applicants did not receive any school credit. Applicants expressed high levels of satisfaction with virtual interviews, which were less costly. Program tours and department websites had an increased impact on how applicants ranked programs during COVID-19. While the interpretation is limited by the response rate of 57%, this is the first paper to prospectively compare RO residency applicant perspectives on away rotations, interviews, and ranking practices before and during COVID-19.

Away Away rotation costs are significant posing a barrier for students with limited financial resources.

Underrepresented in medicine (UIM) students are 40% less likely than non-Hispanic white students to attend medical schools with RO departments and thus depend on away rotations for RO exposure and mentorship (10) .

Novel programs demonstrate the potential to increase diversity within RO through increasing accessibility of virtual RO rotations (11) . Away rotations may also be challenging for applicants with children or other familial responsibilities. Normalizing meaningful alternatives to the traditional in-person away rotations may increase equity among applicants. Most virtual away rotations during Match 2021 did not provide applicants medical school credit, likely due to their rapid development. We recommend programs continue offering virtual away rotations while working with medical schools to provide credit for these experiences.

COVID-19-associated travel restrictions necessitated virtual interviews. Particularly for RO residency applicants, who are completing both internship and residency interviews, average pre-COVID-19 in-person interview costs are upwards of $8,000 (12) . While it has been hypothesized that virtual interviews could mitigate these costs (13), the extent was largely unknown. During the 2020 Match 49% of applicants spent >$5,000 on interviews (2% spent <$1,000), while during the 2021 Match no applicants spent >$5,000 on interviews (45% spent <$100). Despite program culture being the top factor impacting RO program ranking by applicants, half of applicants failed to gain a sense of program culture during virtual interviews. Nonetheless, the majority of 2021 applicants were satisfied with virtual interviews and expressed the cost savings were worth the tradeoffs of virtual interviews.

Post-interview communications are pervasive in RO (6, 7, 14, 15) . These correspondences occur bidirectionally with applicants sending letters of interest to programs and programs informing applicants they are "ranked to 

Prior to the pandemic, away rotations, informal conversations with residents, and program culture observations were the three most significant factors impacting an applicant's ranking of programs (7); these remain unchanged.

However, two factors significantly increased: program websites and the departmental facilities, technology, and machines. The 2021 applicants likely used online information to better understand programs. For example, the "anonymous Google Spreadsheet" is used across residency specialties used by applicants to share information about program culture and general impressions, which may ultimately influence how applicants perceive programs. Whether this leads to similar levels of program fit and satisfaction is unknown.

Respondents represented applicants to one large RO residency program, potentially limiting the generalizability of the study; however, our cohort includes the majority of applicants in the 2020 and 2021 Match (17). The data interpretation is limited by a 56% response rate. Response bias is also possible given the response rate and survey dissemination timing. The delayed survey distribution time may impact recall bias, although the survey followed Match Day to allow respondents to answer questions honestly without worry of impacting their Match.

The COVID-19 pandemic introduced new considerations to the RO Match process. The 2021 Match survey respondents completed fewer away rotations (mostly virtual, many without credit), were satisfied with virtual interviews and associated costs savings, reported decreased familiarity with program culture, and engaged in more post-interview communications. This study provides support for exploring virtual away rotations and interviews beyond the pandemic.

Number of completed away rotations by Match 2020 and Match 2021 applicants.

Match 2021 applicant perspectives on virtual interviews. Binary categories were created with responses for either 1) very much or somewhat and 2) neutral, not much, or not at all.

Interview cost comparison between the 2020 Match and the 2021 Match.

Factors that impact how applicants rank programs during the 2020 Match and 2021 Match. 

National Resident Matching Program. Data Release and Research Committee: Results of the 2014 NRMP Program Director Survey

Qualities of residency applicants: comparison of otolaryngology program criteria with applicant expectations

How we select our residents--a survey of selection criteria in general surgery residents

Feasibility and Usability of Tele-interview for Medical Residency Interview

The Game Continues: Seeking Clarity in the Radiation Oncology Match Advances in Radiation Oncology

R: A language and environment for statistical computing

Patterns and Perceptions of "Away" Rotations Among Radiation Oncology Residency Applicants

Diversity Based on Race, Ethnicity, and Sex, of the US Radiation Oncology Physician Workforce

Improving Diversity and Inclusion in the Post-Coronavirus Disease

Era Through a Radiation Oncology Intensive Shadowing Experience (RISE)

A Burnout Reduction and Wellness Strategy: Personal Financial Health for the Medical Trainee and Early Career Radiation Oncologist

Resident Considerations for Virtual Interviews in Radiation Oncology: Perspectives From the Sunshine State

Integrity of the National Resident Matching Program for Radiation Oncology: National Survey of Applicant Experiences

Trends in Radiation Oncology Residency Applicant Interview Experiences and Post-Interview Communication

Responses regarding the number, credits received, and observations of program culture among 2021 Match applicants who did away rotations. A total of 44 radiation oncology rotations were for credit and 30 away rotations were not for credit. Abbreviations: AR = away rotations. Totals may not equal to 100 due to rounding.