key: cord-0918348-3wot3f8e
authors: Vongsachang, Hursuong; Fliotsos, Michael J.; Lorch, Alice C.; Singman, Eric L.; Woreta, Fasika A.; Justin, Grant A.
title: The impact of COVID-19 on ophthalmology resident surgical experience: a retrospective cross-sectional analysis
date: 2022-03-04
journal: BMC Med Educ
DOI: 10.1186/s12909-022-03205-0
sha: 9010591cdb04e01f9b9d68f6cd7e63cad2a58e1e
doc_id: 918348
cord_uid: 3wot3f8e

BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic caused significant disruption to in-office and surgical procedures in the field of ophthalmology. The magnitude of the impact of the pandemic on surgical training among ophthalmology residents is not known. This study aims to quantify changes in average case logs among United States (U.S.) ophthalmology residency graduates prior to and during the COVID-19 pandemic. METHODS: Retrospective, cross-sectional analysis of aggregate, national data on case logs of U.S. ophthalmology residency graduates from 2012 to 2020. The yearly percent change in the average number of procedures performed in the Accreditation Council for Graduate Medical Education (ACGME) ophthalmology resident case logs were analyzed using linear regression on log-transformed dependent variables. The average percent change from 2019 to 2020 was compared to the average yearly percent change from 2012 to 2019 for procedures performed as the primary surgeon, and primary surgeon and surgical assistant (S + A), as well as procedures for which there are ACGME minimum graduating numbers. RESULTS: Across all procedures and roles, average case logs in 2020 were lower than the averages in 2019. While average total cases logged as primary surgeon increased yearly by 3.2% (95% CI: 2.7, 3.8%, p < 0.001) from 2012 to 2019, total primary surgeon case logs decreased by 11.2% from 2019 to 2020. Cataract (-22.0%) and keratorefractive (-21.1%) surgery experienced the greatest percent decrease in average primary surgeon cases logged from 2019 to 2020. Average total cases logged as S + A experienced an average yearly increase by 1.2% (95% CI: 0.9,1.6%, p < 0.001) prior to 2020, but decreased by 9.6% from 2019 to 2020. For ACGME minimum requirements, similar changes were observed. Specifically, the average case logs in YAG, SLT, filtering (glaucoma), and intravitreal injections had been increasing significantly prior to 2020 (p < 0.05 for all) but decreased in 2020. CONCLUSIONS: These findings demonstrate the vulnerability of ophthalmology residency programs to a significant interruption in surgical volume. There is a critical need for development of competency-based, rather than volume-based, requirements to evaluate readiness for independent practice. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12909-022-03205-0.

The coronavirus disease 2019 (COVID-19) pandemic caused significant disruptions to in-office and surgical procedures in ophthalmology. Following guidance urging provision of only urgent or emergent care from March to Open Access *Correspondence: hvongsa2@jhmi.edu 1 Wilmer Eye Institute, Johns Hopkins University School of Medicine, 600 N. Wolfe St, MD 21287 Baltimore, USA Full list of author information is available at the end of the article July 2020 in the United States (U.S.), total encounters in ophthalmology fell nearly 80% [1] [2] [3] . Residency programs had to consider methods to supplement resident learning during this time, especially considering that these shutdowns occurred in the months prior to residency graduation in June 2020 [4] [5] [6] . Residency is a crucial time for residents to gain surgical skills in preparation for fellowship and/or career as a practicing ophthalmologist, primarily through guided exposure to diverse cases performed in the operating room. As the impact of the pandemic on ophthalmic surgical training nationally has not been demonstrated, this study aimed to quantify changes in surgical training among ophthalmology residents in the U.S. prior to and during the pandemic.

We accessed aggregate data on average case logs performed as the primary surgeon, and both primary surgeon and surgical assistant (S + A) roles among graduating ophthalmology residents nationwide from 2012 to 2020, published by the Accreditation Council for Graduate Medical Education (ACGME) [7] . Graduating residents are those in the final year of their four-year ophthalmology residency. Residents record cases performed as either the primary surgeon or surgical assistant into an online ACGME case log system. Each academic year, the ACGME produces anonymized, aggregated datasets with descriptive statistics on cases logged for each procedure across all residents nationally. We downloaded spreadsheets containing this publicly available data from the ACGME website by year. S + A data were included in this study to capture the operative experience of some procedures, such as vitreoretinal and refractive surgeries, that are primarily performed in the surgical assistant role. Cases logged for procedures with minimum ACGME requirements were also included and analyzed separately for more granular detail on specific procedures required for graduation.

For 2012-2019 data, the yearly percent change in the average cases performed by graduating ophthalmology residents per procedure was analyzed using linear regression models on log-transformed response variables with robust variance. This analysis was conducted for procedures performed as surgeon, S + A, and those designated as minimum ACGME graduation requirements. For each procedure, the average yearly percent change from 2012 to 2019 was compared to the percent change in average case logs between 2019 and 2020. Analyses were conducted on Stata SE/15.1 (StataCorp, College Station, Texas) with statistical significance set at p < 0.05. This study abides by the tenets of the Declaration of Helsinki and received exemption from the Johns Hopkins Institutional Review Board.

Across all procedures and roles, average case logs in 2020 were lower than the averages in 2019. While average total cases logged as primary surgeon increased yearly by 3.2% (95% CI: 2.7, 3.8%, p < 0.001) from 2012 to 2019 (Table 1) , total primary surgeon case logs decreased by 11.2% from 2019 to 2020. In the primary surgeon category, cataract, keratorefractive, and retina vitreous surgery experienced the greatest percent decreases in average cases logged from 2019 to 2020, at -22%, -21.1%, and − 15.6%, respectively ( Fig. 1 A) . Average primary surgeon cases in cataracts, glaucoma, and other retinal surgeries were generally increasing prior to 2020 (p < 0.001 for all) but declined in 2020. While average total laser cases were decreasing prior to 2020 (-1.9%, 95% CI: -3.0, -0.8%, p = 0.006), the decline was more pronounced in 2019-2020 at -8.4%. Cases logged as S + A were evaluated as well, of which the results are included in Fig. 1B and Table S1 .

Similar patterns can be seen in the procedural categories for which ACGME requires a minimum number for graduation (Table 2 ). Average case logs in YAG capsulotomy, selective laser trabeculoplasty (SLT), filtering (glaucoma), and intravitreal injections had been increasing prior to 2020 (p < 0.05 for all) but decreased in 2020 ( Fig. 1 C) . In terms of absolute case volume, the numbers recorded in 2020 were comparable to recent years for cataract surgery (2014) However, for laser peripheral iridotomy (LPI), panretinal photocoagulation (PRP), pterygium, keratorefractive surgery, strabismus, total oculoplastics and orbit procedures, and chalazion, the 2020 case logs were the lowest reported numbers since 2012.

Overall, among graduating residents, reported procedural numbers across all categories decreased significantly in 2020. In an international survey conducted in early May 2020, the vast majority of ophthalmology residents and fellows reported a > 75% decrease in surgical training [8] . The deviation from expected trends in case logs as demonstrated in this study and the reports of severe disruption in surgical volume in the months prior to graduation suggest that the overall decreases of 11.2% and 9.6% in primary surgeon and S + A procedures, respectively, among graduating ophthalmology residents in 2020 were strongly influenced by COVID-19. High percentage decreases in average case logs were observed in both high volume (e.g. cataract) and low volume (e.g. Table 1 Average Logged Cases across Graduating Residents by Graduation Year, Surgeon Role. P-value is for the average yearly % change prior to 2020 12.5 (7) 13.3 (7) 13.1 (8) 13.4 (8) 13.8 (9) 15.8 (12) 16.3 (13) 14 10.4 (7) 10.4 (7) 11 (7) 10.5 (7) 10.5 (7) 10.1 (6) 9.5 (6) 1. Oculoplastics and Orbit -Ptosis 6.6 (8)

6.5 (7) 6.5 (7) 6.7 (6) 6.9 (7) 7.3 (7) 7.6 (7)

6.9 (6) 6.7 (7) 1 keratorefractive surgery, retinal vitreous) procedures. The relative loss of dozens of cases for high volume procedures or 1-2 cases for rarer procedures may amount to valuable resident training experience lost. It is reassuring, however, that for certain procedures, the surgical numbers have not decreased below that of recent years.

Nonetheless, a critical limitation of using minimum surgical numbers to assess competency is that it is difficult to ascertain what threshold suffices as adequate experience, as the number required for competency may vary by trainee. Historically, programs have graduated residents who have had on average, lower case logs than residents who graduated during the COVID-19 pandemic. In fact, in 2020, the ACGME suspended minimum case log requirements and requested program directors evaluate competency for graduation [9] . Developing and implementing competency-based assessment to help adequately gauge resident preparation, as well as alternative training strategies (e.g. virtual reality simulators) [4] [5] [6] to provide residents with sufficient experience in a time with decreased surgical volume was a key task of residency program directors with the 2020 graduating classes [8] .

The need for competency-based assessment and methods to supplement surgical training is not limited to the 2020 graduating class [9] [10] [11] . The surgical experience of upcoming graduating classes has been and continues to be impacted by the COVID-19 pandemic, but other interruptions in patient volume, such as loss of a high-volume surgeon or training site, can be equally disruptive to resident surgical and clinical education. This notion of shifting from minimum required numbers to competency-based requirements will likely continue to expand in graduate medical education, as studies have shown that residents achieve competency at different numbers [12] . While the ACGME has outlined levels of achievement of six competency milestones on a 1-5 scale (of which surgical skills falls under the Patient Care competency) [13] , residency program directors have reported a need for more guidance and resources in implementing competency assessment even prior to the pandemic [14] . Our findings demonstrating the global decrease in surgical volume for residents during the pandemic only further emphasize that continued efforts to establish feasible, valid, and reliable assessment tools and surgical simulation for the variety of procedures that residents are expected to perform are needed [15] . Earlier, increased surgical exposure with the transition from traditional (one year of internship prior to three years of ophthalmology residency) to integrated four-year ophthalmology residency programs may also facilitate the establishment of competency-based assessment. Ultimately, with standardized tools widely incorporated into residency training, rigorous competency-based assessment can help identify areas of improvement for residents and allow for confidence in patient outcomes for residents deemed ready for independent practice [16] . In addition, competency-based assessment will allow for better evaluation of the effect of unexpected interruptions in surgical volume on surgical competency. This study has several limitations. Most importantly, we are not able to assess any loss of competency due to the decreased volume of cases during the COVID-19 pandemic. In addition, while we present changes in resident case logs from 2019 to 2020, we are unable to attribute the decrease in resident case logs to COVID-19 disruptions alone. However, we were able to compare the changes experienced between 2019 and 2020 to the general trend in surgical volume in prior years. Additionally, the analysis is limited to case logs of graduating residents. While these senior residents may have had the most disruption in surgical volume in their final year, we cannot yet quantify changes to the surgical experience of more junior trainees during the pandemic. We also do not assess the operative experience of fellows, many of whom are in one-year programs. Further, while we used an official nationwide dataset, this data relies on accurate logging of cases by residents. Additionally, this study analyzed aggregated data, and thus, these results may not apply to the surgical experience of individual residents or residency programs. Finally, future work will need to evaluate the impact of lower surgical volume on resident surgical skills and patient outcomes.

In conclusion, this study provides evidence that cases logged by residents greatly decreased during the time of COVID-19, however for many procedures with minimum requirements, the surgical numbers were not lower than that of recent years. The COVID-19 pandemic highlighted the vulnerability of ophthalmology residency programs to a significant interruption in surgical volume and may serve as an impetus for moving towards competency-based, rather than volume-based, assessment. Developing competency-based rather than volume-based requirements in graduate medical education may be valuable in assessing readiness for practice, particularly during periods with interruptions in surgical volume.

Abbreviations COVID-19: coronavirus disease 2019; U.S.: United States; S + A: primary surgeon and surgical assistant (S + A); ACGME: Accreditation Council for Graduate Medical Education.

Important coronavirus updates for ophthalmologists

Recommendations for urgent and nonurgent patient care

Natio nal-Patie nt-and-Proce dure-Volume-Track er-and-Report_ Weekly-Update

Incorporating a virtual curriculum into ophthalmology education in the coronavirus disease-2019 era

Finding Focus in Crisis: Resident-Driven Graduate Medical Education at a Military Training Facility during the COVID-19 Pandemic

Academic Ophthalmology during and after the COVID-19 Pandemic

The Accreditation Council for Graduate Medical Education. Case Log Graduate Statistics National Report

Reshaping ophthalmology training after COVID-19 pandemic

COVID-19: Special Communication to Surgical Program Directors, including Case Log Guidance

Verifying Surgical Competence: Our Fiduciary Responsibility

Ophthalmology Resident Surgical Competence: A Survey of Program Directors

Relationship of procedural numbers with meaningful procedural autonomy in general surgery residents

The Accreditation Council for Graduate Medical Education. Ophthalmology Milestones

A national program director survey of the shift to competency-based education in ophthalmology

Competency-Based Education Assessment Tools for Laser Capsulotomy, Peripheral Iridotomy, and Selective Laser Trabeculoplasty

Association Between Surgeon Technical Skills and Patient Outcomes

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Not applicable.

The view(s) expressed herein are those of the author(s) and do not reflect the official policy or position of the U.S. Army Medical Department, the U.S. Army Office of the Surgeon General, the Department of the Air Force, the Department of the Army, Department of Defense, the Uniformed Services University of the Health Sciences or any other agency of the U.S. Government. Mention of trade names, commercial products, or organizations does not imply endorsement by the U.S. Government.

All authors had significant contributions to the manuscript deserving authorship and reviewed the final version of this manuscript. 3 

The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s12909-022-03205-0.

Authors' contributions HV analyzed and interpreted the data and drafted the manuscript. MJF, ACL, and ELS contributed to revising the manuscript. GAJ and FAW acquired the data, conceived of the study and study design, and revised the manuscript. All authors read and approved the final manuscript.

Hursuong Vongsachang was supported by Grant TL1 TR003100, National Center for Advancing Translational Sciences, National Institutes of Health. The funding body was not involved in the design of the study; the collection, analysis, and interpretation of data; nor the writing of the manuscript.

The datasets generated and/or analysed during the current study are available in the Accreditation Council for Graduate Medical Education repository, https:// apps. acgme. org/ ads/ Public/ Repor ts/ Report/ 25.

This study abides by the tenets of the Declaration of Helsinki and received exemption from the Johns Hopkins Institutional Review Board (IRB00181755).

Not applicable.

The authors declare that they have no competing interests.