key: cord-0691013-vsueiyat
authors: Aiyer, Amiethab A.; Granger, Caroline J.; McCormick, Kyle L.; Cipriano, Cara A.; Kaplan, Jonathan R.; Varacallo, Matthew A.; Dodds, Seth D.; Levine, William N.
title: The Impact of COVID-19 on the Orthopaedic Surgery Residency Application Process
date: 2020-05-28
journal: J Am Acad Orthop Surg
DOI: 10.5435/jaaos-d-20-00557
sha: 44c2425ac50491d3afa719edb1b07ae0f0ea05a4
doc_id: 691013
cord_uid: vsueiyat

Over recent months, coronavirus disease 2019 (COVID-19) has swept the world as a global pandemic, largely changing the practice of medicine because it was previously known. Physician trainees have not been immune to these changes—uncertainty during this time is undeniable for medical students at all levels of training. Of particular importance is the potential impact of COVID-19 on the upcoming residency application process for rising fourth-year students; a further source of added complexity in light of the newly integrated allopathic and osteopathic match in the 2020 to 2021 cycle. Owing to the impact COVID-19 could have on the residency match, insight regarding inevitable alterations to the application process and how medical students can adapt is in high demand. Furthermore, it is very possible that programs will inquire about how applicants spent their time while not in the hospital because of COVID-19, and applicants should be prepared to provide a meaningful answer. Although competitive at a basal level, the complexity of COVID-19 now presents an unforeseen, superimposed development in the quest to match. In this article, we aim to discuss and provide potential strategies for navigating the impact of COVID-19 on the residency application process for orthopaedic surgery.

O ver recent months, the novel coronavirus, causing coronavirus disease 2019 (COVID- 19) , has swept the world as a global pandemic, largely changing the practice of medicine because it was previously known. Physician trainees have not been immune to these changes, from fourth-year medical students graduating early to assist with the increased medical burden to delays in the start of the semester for matriculating students. Uncertainty during this time is undeniable for medical students at all levels of training. Of particular importance is the potential impact of COVID-19 on the upcoming residency application process for rising fourth-year students. Owing to the impact this could have on the residency match, insight regarding inevitable alterations to the application process and how medical students can adapt is in high demand.

Matching into a position within an Accreditation Council for Graduate Medical Education (ACGME)accredited orthopaedic surgery residency program is a highly competitive process, with almost all available positions being filled annually. 1 The average orthopaedic applicant in 2018 submitted 85.7 applications, compared with 54.1 in 2008; this increase has been attributed to both specialty competitiveness and ease of application submission conferred by the Electronic Residency Application Service (ERAS). 2 Although the applicant cohort is highly accomplished, orthopaedic surgery claims the lowest match rate of fourth-year medical students when compared with all other medical specialties, at just 75%. This competitiveness is related to increases in the number of applicants with minimal change to the number of available training spots. 2, 3 Higher match success on an individual basis has been associated with Alpha Omega Alpha (AOA) membership status, USMLE step 1 score AQ : 3 , third-year clerkship grades, USMLE step 2 Clinical Knowledge score, research productivity, and clinical performance. 1, 2, [4] [5] [6] [7] Applying to orthopaedic surgery residency is not only daunting because of the competitive nature of the match but also the substantial expense. The average applicant spends roughly $7,000 on the interview process, not including financial expenses related to multiple "away rotations" done at outside institutions (per applicant an average of 2.4 rotations are done and cost an additional $2,799, on average). 8 Most students borrow money to pay for these expenses. 2, 8, 9 These reported findings are a testament to the challenge of the orthopaedic residency application process in the United States. A successful match requires high-level performance in all categories, strategic planning, financial support, and substantial optimism.

In addition to being competitive and expensive, the application process is also continuously evolving. The recent merging of the Accreditation Council for Graduate Medical Education (traditionally for allopathic medical graduates) and the American Osteopathic Association (traditionally for osteopathic medical graduates) represents an additional consideration for the upcoming 2020 to 2021 cycle; the impact of this on the application process is not completely understood. 4 However, the complexity of COVID-19 now presents an unforeseen, superimposed development in the quest to match. In this article, we aim to discuss and provide potential strategies for navigating the impact of COVID-19 on the residency application process for orthopaedic surgery.

How Does COVID-19 Impact the Role of Away Rotations?

Weeks after COVID-19 began directly affecting the United States, curricular changes were implemented within medical schools across the nation. On March 17, 2020, the AAMC released a statement: "The AAMC strongly supports medical schools pausing all student clinical rotations, effective immediately, until at least March 31." 10 A unified response was noted across institutions as US medical schools began transitioning lectures to an online platform and withdrawing third-and fourth-year students from in-hospital clerkships. The suspension of clinical rotations has continued through May. These measures lead to several implications for the upcoming residency application cycle. For example, there will be a dramatic reduction of home and away orthopaedic rotations for medical students to demonstrate their clinical skills, effort, and teamwork. Medical schools will also be pressed to accommodate missed clinical time for their own mandatory fourth-year rotations. The potential for students to do home orthopaedic surgery rotations later this cycle still exists; however, on May 11, 2020, the AAMC announced their discouragement of any away rotations and a delay of the timeline for the application process. 11,12 Specifically, ERAS determined that for this cycle, residency programs will gain access to applications and Medical Student Performance Evaluations (MSPE) will be released to programs on Wednesday, October 21, 2020, delayed from the previously set date of September 15 for the opening of ERAS and October 1 for the MSPE. 12 Although speculative, this may have implications for the timeline of the residency match cycle as well. Given that away rotations will not occur this year, this will yield an unprecedented and a drastic change from the average 2.4 away rotations per orthopaedic applicant of recent years. 2 Away rotations have been important for the orthopaedic surgery application for a variety of reasons. First, the ability of an applicant to complete at least two away rotations has, in the past, correlated with increased likelihood that they will successfully match. More than 50% of matched applicants end up matching at either their home program or one of the programs at which they completed an away rotation. 2, 13 Furthermore, applicants who completed at least two away rotations were 10% more likely to match than those who completed one or zero, with diminishing returns after successful completion of two. 2 Second, participation in away rotations provides applicants with insight regarding their "best fit" program. Anecdotally, applicants who visit different programs may make more educated rank decisions, increasing Match satisfaction. Although literature suggests that the away rotation structure may not coincide with goals of undergraduate medical education, away rotations certainly play an even more important role for students who do not have an orthopaedic program at their home institutions. 9 For these students in particular, away rotations help build an orthopaedic community and network to which they do not otherwise have access.

Historically, the purposes of away rotations are generally fourfold. If an applicant does well on a rotation, they can make a lasting impression that may improve their rank. 13 Away rotations allow applicants to demonstrate they are a team player, hard worker, and encompass character traits that will translate into a successful future resident. Away rotations may also provide an applicant with the opportunity to overcome a "weak" portion of their application. For example, an applicant with a lower-than-average USMLE step 1 score may be able to exemplify traits, such as grit and conscientiousness, which have been identified as predictors of positive resident performance. 7, 14 By dedicating a month to a specific away rotation, applicants display notable interest in that program. This dedication and commitment has the potential to give the student a competitive edge, especially given residency programs are sifting through over 100 applications for each residency position. Finally, on an away rotation, students can also obtain letters of recommendation from their newly acquired orthopaedic surgery mentors. Letters written by away rotation orthopaedic surgeons describe how students do within their specialty and provide additional characterization of an applicant in addition to their home program faculty.

The AAMC's decision to discourage away rotations for the 2020 to 2021 application cycle is cited to be for multifactorial reasons, all leading to the inaccessibility of away rotations for the entire applicant pool. 11 For the sake of fairness across all applicants, this consensus agreement is beneficial. If some institutions were to offer away rotations and some not, this would have created an otherwise avoidable incongruence of opportunity across the applicant cohort. As a result, an all-or-nothing approach to away rotations for this application cycle is of the greatest benefit to the applicant pool to maintain a homogenous playing field.

Because away rotations will not occur this year, we recommend that applicants focus on clinical experiences in orthopaedics at their home institutions, understanding that these opportunities will also be limited by changes related to COVID-19. Students without a home program have been taken into account by the AAMC recommendation and may benefit from connecting with local orthopaedic surgeons, either community or academically based, to gain valuable experience. 11 Thus, high-quality letters of recommendation from community-based orthopaedic surgeons, or surgeons and physicians outside the orthopaedic specialty, may be much more common this year.

Despite the stated importance of away rotations, the orthopaedic surgery residency application contains many other factors worth highlighting. Given orthopaedic surgery rotations will be reduced in 2020, applicants should focus on optimizing other elements of their applications that have not already been completed. These parameters include writing the personal statement, studying for USMLE step 2 CK (if it has not been taken), research productivity (completing ongoing research projects, book chapters, and/or review articles), identifying mentors who can write strong letters of recommendation (LOR, including the American Orthopaedic Association [AOA AQ : 4 ] Standard Letters of Recommendation [SLOR]), and strongly advocate for the student whenever possible. In addition, students should consider taking advantage of volunteer/leadership opportunities related to the COVID-19 pandemic.

With this information in mind, how can students best strengthen their applications, and how can programs differentiate between them? What factors can applicants control, and how can they best use their time during the suspension of clinical responsibility?

USMLE scores (USMLE step 1 and USMLE step 2 CK) have been continually cited by surveyed program directors as notable measures used to stratify applicants, and preparation for these examinations can be a point of focus for applicants during this time. To highlight their importance, step 1 was shown to correlate with resident performance on the ABOS part 1 examination in one study (n = 181 residents) 15 and step 2, similarly, in another (n = 60 residents). 16 In past years, students with suboptimal step 1 scores have pursued away rotations to positively impress two or three particular programs, thus ensuring their application gets considered despite the initial screen. With less firsthand data from away rotations this year, programs may place increased emphasis on the USMLE step 2 CK score for all applicants, especially in anticipation of step 1 becoming a pass/fail examination in 2022. Students with lower step 1 scores are strongly encouraged to take the USMLE step 2 CK examination before October 2020 to include this additional data point on their application, although including a step 2 CK score in the application, granted the score is strong, will be of benefit to all applicants.

Other quantitative academic measures such as clerkship grades and AOA status carry importance and should be focused on; however, their potential limitations are becoming increasingly scrutinized. Recent research has revealed notable bias in standardized testing and clerkship grades. 17, 18 Some medical schools are even removing AOA status as a distinguishing concept because of associated bias. 19 As a result, other qualitative factors are becoming more important in the application process.

Extracurricular endeavors represent an opportunity for students to display dedication and productivity during the COVID-19 crisis. Although direct clinical research has been paused, applicants can seek opportunities to assist with retrospective projects, finishing studies, review articles, or book chapters. Students may also consider contributing to volunteer efforts to help the hospital or broader community, depending on location and need, examples include sewing masks or creating other personal protective equipment, participating in telehealth or COVID-19 hotlines, acting as a scribe, or assisting with medical school education. It is very possible that programs will inquire about how applicants spent their time while not in the hospital because of COVID-19, and students should be prepared to answer these questions in a meaningful way. As such, pursuing meaningful activities will allow students to demonstrate their initiative, dedication, and underlying values.

Defining research experience is subjective, but the quality may be more important than the quantity of research contributions. For example, leading a project from conception to completion and being able to speak insightfully about this process may be more valuable than engaging in multiple projects on a more superficial level. In addition, although research in orthopaedics may help students learn about the specialty, scientific contributions in any field are considered valuable. Regardless of the specific subject matter, research is a means for applicants to exhibit intellectual curiosity, dedication to scientific investigation, critical analysis skills, leadership capabilities, perse-verance, and ability to work within a team. 6 Contrarily, applicants should be aware that the number of publications as a medical student has not necessarily been shown to correlate with global evaluation scores in residency or ABOS part I scoring. 16 In addition to spending time on research, applicants can reflect desirable qualities through other experiences. Volunteer work may demonstrate empathy, a passion for community outreach, or dedication to systems improvement. Previous careers can lead to diverse skills (eg, business, engineering, teaching, and leadership) and perspectives that would bring richness to any training program. Importantly, students who have been employed out of necessity to support themselves or their families should be acknowledged for this effort and dedication.

LORs and the SLOR are cited as important factors of the application and allow programs to better understand how physicians with whom they have worked in a clinical and/or research setting perceives the applicant. LORs may be more notable this year because students will have reduced opportunities to demonstrate their demeanor and clinical performance. 20 Residency applications typically include three to four LORs and/or SLOR. Students may have limited clinical opportunities from which to request letters of recommendation at home because of reductions in elective patient care. With no away rotations this cycle, more LORs and SLORs will need to be completed by home faculty. Applicants could also consider including letters from nonorthopaedic surgeons in their application (eg, an attending physician from the general surgery clerkship with whom the medical student had a positive experience). These letters should come from individuals who can speak to an applicant's capabilities as a future physician and surgeon, as well as their personal qualities. Although a letter from a well-respected orthopaedic surgeon may carry considerable influence, a letter from a wellestablished physician outside of orthopaedics can be used to as a testament to character and leadership qualities. Research mentors may also contribute influential letters; they can add longitudinal insight regarding skills, dedication, and commitment. In contrast to previous years, reliance on nonorthopaedic letters of recommendations may be more critical during this application cycle.

Finally, applicants may use this time to refine their personal statement and organize a list of extracurriculars, leadership experiences, and hobbies which provides applicants with the opportunity to discuss other facets of their lives that define them as individuals. The section of the application allows students to provide insight into their values, background, and any particular challenges they may have faced in life. The potential impact of the personal statement should not be underestimated. Although it can be an opportunity for discussion during interviews, it can also have a decidedly negative impact on an application when the writing goes off point.

To navigate this, students should seek honest feedback from their mentors about the content and style their personal statement. It is often valuable to seek additional outside perspective from various individuals (nonfamily/ nonorthopaedics) because their feedback can add to the strength of your personal statement.

In addition to changes applicants must consider, programs may need to adapt to the current situation as well. First, programs will need to consider how to best to accommodate students who have lost educational time secondary to COVID-19. The AAMC's guidelines also included a recommendation to use universal video interviews, in lieu of the traditional in-COVID-19'S Impact on the Orthopaedic Applicant person interview, for all programs including students' home programs. 11 There is already literature support for the use of this platform in orthopaedics. 21 However, video interviews place a strain on both the student and program regarding the ability to identify relative strengths and interest in each other. It can also diminish the ability to showcase the resources available and physical spaces in which training takes place. These issues must be taken into account to optimize, to the best of programs' abilities, an equal playing field for all applicants and to ensure a truthful presentation of the program to applicants over a virtual platform.

Orthopaedic residency programs have not yet defined a formal position on how the COVID-19 will affect the application process. Leaders in Emergency Medicine, Dermatology, Neurosurgery (other specialties for which the application process is heavily dependent on away rotations), and Obstetrics and Gynecology have recently released guidelines for the upcoming application cycle, summarized in detail in ½T1   Tables  1-4 . [22] [23] [24] [25] The newest published recommendations (on May 11, 2020) from the Coalition for Physician 

Residency program leadership should consider reducing their typical number of Standardized Letters of Evaluation (SLOE) needed to review an application to one SLOE (or less) to account for students who cannot obtain a SLOE at their home institution. Programs should be willing to accept alternative letters of recommendations (LOR) to act as surrogates for their typical SLOEs requirements 2. Encourage programs to give weight to alternative (non-SLOE) letters

Letter writers who are not EM physicians should be made aware of the importance of ensuring their letters address knowledge, skills, and behaviors typically seen in the "qualifications for EM" section of SLOEs. 

In the event that a student is both able to travel from his/her home institution and to secure an available clerkship position at an institution accepting visitors, that a student should not do more than one visiting rotation (for those with a home EM program), and not more than two, maximum for those without a home EM rotation. Table 2 Summary of the SNS Policy on External Medical Student Rotations During the COVID-19 Pandemic

All external medical student rotations in neurological surgery will be deferred in 2020 Each student should have at least 8 weeks (2 rotations) of neurological surgery experience at their home institution. For students without a home program, these 8 weeks should be completed at the closest ACGME-accredited program. Letters of recommendation should come from the student's home institution, including at least two (2) from neurosurgical faculty members and one (1) from a general surgery faculty member (ie, a program director, clerkship director, or surgical faculty mentor). Letter templates will be provided at the SNS website Letters of recommendation from external rotations will be looked upon unfavorably (with the exception of those students without a home program) Direction regarding interviews will be forthcoming and determined in close alignment with ERAS and the NRMP Accountability's Work Group on Medical Students in the Class of 2021 and the AAMC will affect students applying to all specialties, including orthopaedic surgery, and are summarized in ½T5 Table 5 . The

Advising Students Committee in Emergency Medicine has requested that students complete no more than one away rotation for those students with a home EM program and no more than two away rotations for those without a home Emergency Medicine (EM) program (Table 1) . 23 They have also suggested programs be more flexible with nonspecialtyspecific LORs required from Table 3 APGO and CREOG Recommendations During the COVID-19 Pandemic

Guidelines for the application cycle:

Limit away rotations for the 2020-2021 academic year to those students who cannot acquire those experiences locally Be flexible in the number of specialty specific LORs required Encourage residency programs to develop alternate and innovative means of conveying information about their residents, their residency program, and location to applicants Tentative Timeline:

Adjust the ob-gyn application submission deadline to October 16, 2020, or the MSPE letters release date (if modified), whichever is later Adjust interview offer dates to November 3 and November 10 Adjust interviews to begin no earlier than November 10 Adjust student status final determination to December 23 Consider offering earlier interview dates to candidates who can travel by automobile, given the variable COVID-19 peaks and associated travel restrictions Consider providing the opportunity for video interviewing, especially for those students who may not be able to travel because of local COVID-19 infection rates. Table 4 Summary of the Consensus Statement on the Upcoming Dermatology Residency Application Cycle Regarding COVID-19

Research: "We understand that projects have been halted or delayed secondary to the COVID-19 pandemic and will note students' previous and ongoing participation in research and academic projects. Efforts that students have put forth are valuable, irrespective of whether they culminated in published work" Volunteer/service/other experiences:

Traditional experiences and opportunities have been altered or made impossible because of the COVID-19 pandemic.

Opportunities associated with change in institutional practice related to COVID-19 may be variable across institutions. Previous and current volunteer experiences will be reviewed in this context. Away rotations:

Away rotations should not be perceived as required or necessary for matching into dermatology residency. If you have specific interest in any program, visit the program website to determine policy change at the program level. Research year:

It makes sense to continue plans for pursuing a research year if students were already planning to do so before the COVID-19 pandemic; however, no reason exists to choose this path simply because of COVID-19 related changes to one's application. USMLE step 2:

Students may have planned to take the USMLE step 2 examination but are now unable to do so because of lack of availability of testing centers. Refer to program websites to determine whether step 1 score cutoffs are used and/or whether step 2 scores are recommended or required.

applicants. The Society of Neurologic Surgeons released a policy on April 29, 2020, announcing "all external medical student rotations in neurological surgery will be deferred in 2020," and because applicant experience in neurological surgery rotations is important before committing to the specialty, "each student should have at least 8 weeks (2 rotations) of neurological surgery experience at their home institution" ( ½T2 Table 2 ). 24 The Association of Processors of Gynecology and Obstetrics and the American College of Obstetricians and Gynecologists released a joint statement that they would "address this inequality by limiting away rotations happening late in the year to those students who cannot acquire experiences locally" ( ½T3 Table 3 ). 22 Table 4 ). 25 These society committees have also recommended programs be "flexible in the number of specialty-specific LORs required" with neurosurgical residency leadership specifying that LORS should "come from the student's home institution, including at least two from neurosurgical faculty and one from general surgery leadership. [22] [23] [24] [25] Ultimately, the AAMC "recommends that for the 2020 to 2021 academic year, away rotations be discouraged" except under special circumstances. They additionally recommend the use of online interviews and virtual visits for all applicants including local students, a delayed opening of ERAS and release of the MSPE to residency programs, and general communications ( Table 5) . 11 Delayed release of the MSPE or "Dean's Letter" will facilitate the completion of student core clerkships, of which evaluative summary comments are included on the MSPE. These "sensible" solutions may have other impacts later in the Table 5 Summarized AAMC Final Report and Recommendations for Medical Education Institutions

For the 2020-2021 academic year, away rotations are discouraged, except under the following circumstances: (1) learners who have a specialty interest and do not have access to a clinical experience with a residency program in that specialty in their school's system and (2) learners for whom an away rotation is required for graduation or accreditation requirements. *Individuals meeting these exceptions should limit the number of away rotations as much as possible and consider geographically proximate programs when appropriate.

All programs should commit to online interviews and virtual visits for all applicants, including local students, rather than inperson interviews for the entire cycle. The medical education community should create a robust digital learning environment and set of tools that will yield the best experiences for programs and applicants. Recommendation 3-The ERAS Opening for Programs and the Overall Residency Timeline ERAS opening for residency programs should be delayed and the MSPE release should be delayed and opening and release should happen on the same day. *ERAS announced that this date is October 21, 2020. 12

Specialty organizations should work with the individual programs to develop and communicate to applicants and schools clear, consistent plans and practice around both away rotations and interviews as soon as possible. Medical schools should develop clear, consistent policies around any limitations of students' participation in away rotations and in acceptance of visiting students, and the schools should communicate these as soon as possible. With a goal of decreasing stress and increasing a sense of fairness, we suggest programs and schools commit to a consistent policy for the entire upcoming residency application and selection cycle. Both programs and schools should include statements about COVID-19-related training, testing, and quarantine requirements for any away rotations that are allowed.

year, for example, pushing Match Day closer to graduation and closer to the beginning of internship. The situation will continue to change, but for the sake of being able to move forward, it becomes increasingly important for orthopaedic chairs, program directors, and the orthopaedic community to agree on a plan of action.

Although the strategies described above may provide the most benefit for the most applicants, implementation will be challenged by individual motivations and anxieties associated with the residency application process. Strong mentoring and advising is critical this cycle. 26 We recommend that students seek honest feedback about their relative competitiveness and that medical educators, deans, residency directors, and program chairs practice thoughtful transparency to guide students through the application process. This approach will likely decrease the time, effort, and expense associated with the application process without decreasing the match rate. 27 These considerations are important for both students and residency programs at all times, but especially in the context of the COVID-19 pandemic, when all of these resources are especially precious.

The effects of COVID-19 will inevitably lead to changes in the orthopaedic residency application process that will affect all students and the residency programs. With decreased opportunities for learning and exposure in clinical settings, including the removal of away rotations, applicants may be able to adjust by seeking alternative forms of defining themselves. These may include scheduling the USMLE step 2 CK before applications are submitted, conducting research remotely, or volunteering in their communities. Although the pandemic has brought new and potentially stressful considerations to the residency application process, it is important to maintain perspective and remember that we are all navigating these times together. The COVID-19 crisis has placed an incredible burden on healthcare systems, economies, and individual lives around the world. This challenge and uncertainty may also present opportunities for students to define themselves not only as applicants in orthopaedic surgery but as individuals overcoming a healthcare crisis.

Critical assessment of the contemporary orthopaedic surgery residency application process

Matching in orthopaedic surgery

Academic characteristics of orthopaedic surgery residency applicants from 2007 to 2014

Unifying the orthopaedic surgery residency application process under a single accreditation system: A primer

Factors affecting orthopedic residency selection: A cross-sectional survey

The effect of applicant publication volume on the orthopaedic residency match

A comparison of matched and unmatched orthopaedic surgery residency applicants from 2006 to 2014: Data from the National Resident Matching Program

The economic burden of orthopedic surgery residency interviews on applicants

Orthopaedic surgery residency: Perspectives of applicants and program directors on medical student away rotations

AAMC: Final Report and Recommendations for Medical Education Institutions of LCME-Accredited

The cost of getting into orthopedic residency: Analysis of applicant demographics, expenditures, and the value of away rotations

Objective predictors of grit, self-control, and conscientiousness in orthopaedic surgery residency applicants

Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination?: A multicenter study

Does residency selection criteria predict performance in orthopaedic surgery residency?

Core clerkship grading: The illusion of objectivity

Clinical performance evaluations of third-year medical students and association with student and evaluator gender

Racial disparities in medical student membership in the Alpha Omega Alpha honor society

Videoconference interviews for an adult reconstruction fellowship: Lessons learned

APGO and CREOG Residency Application Response to COVID-19

CORD: Consensus Statement Regarding SLOEs and Away Rotations From the CORD Advising Students Committee in Emergency Medicine

Surgeons SoN: Society of Neurological Surgeons Policy on External Medical Student Rotations During the COVID-19 pandemic

Consensus Statement on Upcoming Dermatology Residency Application Cycle With Respect to COVID-19 Pandemic

Factors influencing the number of applications submitted per applicant to orthopedic residency programs

The orthopaedic surgery residency application process: An analysis of the applicant experience