key: cord-0778137-3covgyp0
authors: Cruz, Alan Paniagua; Skolarus, Ted A.; Ambani, Sapan N.; Hafez, Khaled; Kraft, Kate H.
title: Aligning Urology Residency Training With Real-World Workforce Needs
date: 2020-10-09
journal: J Surg Educ
DOI: 10.1016/j.jsurg.2020.09.018
sha: e5f37b93813b45cde3ba97534ed92d1c7569d97a
doc_id: 778137
cord_uid: 3covgyp0

OBJECTIVE: Research suggests recently graduated urology residents do not feel ready for independent practice. We conducted a study to determine if Accreditation Council for Graduate Medical Education (ACGME) minimum case requirements, resident case logs, and graduating resident perceived readiness for practice are aligned with the procedural demand and needs of the current urology workforce. DESIGN: Correlative study comparing the association between (1) workforce demand and ACGME case requirements, and (2) workforce demand and perceived resident competency. Three distinct datasets were used; (1) the 2017 Medicare Part B National Summary Data File; (2) the 2017 National Data Report published by the ACGME; and (3) a graduating resident survey from Okhunov et al. SETTING: N/A. PARTICIPANTS: N/A. RESULTS: In 2017, there were a total of 6,784,696 urologic cases performed through Medicare. We found nonsignificant positive associations between resident case logs (rho = 0.16, p = 0.5784), ACGME minimum procedure requirements (rho = 0.42, p = 0.1255), and Medicare procedural demand. Our 15 index procedures accounted for 21.1% (n = 1,431,775) of all Medicare cases, with a median number of 7706 procedures. Endopyelotomy was the least common procedure (n = 98), while cystoscopy was the most common (n = 980,623). Medicare case volume was positively correlated with graduating residents’ procedural confidence (r = 0.86, p < 0.0001). We identified four categories with varied alignment of training and demand: (1) high volume and high confidence, (2) high volume and low confidence, (3) low volume and high confidence, and (4) low volume and low confidence. CONCLUSIONS: Optimizing urology residency training is time-sensitive and important. Using national Medicare data coupled with recently graduated urology resident survey results, we provide a guiding framework for improving the alignment of training with workforce demand. Informed by these results, we recommend altering training requirements to reflect these needs.

OBJECTIVE: Research suggests recently graduated urology residents do not feel ready for independent practice. We conducted a study to determine if Accreditation Council for Graduate Medical Education (ACGME) minimum case requirements, resident case logs, and graduating resident perceived readiness for practice are aligned with the procedural demand and needs of the current urology workforce. DESIGN: Correlative study comparing the association between (1) workforce demand and ACGME case requirements, and (2) workforce demand and perceived resident competency. Three distinct datasets were used; 

In 2017, there were a total of 6,784,696 urologic cases performed through Medicare. We found nonsignificant positive associations between resident case logs (rho = 0.16, p = 0.5784), ACGME minimum procedure requirements (rho = 0.42, p = 0.1255), and Medicare procedural demand. Our 15 index procedures accounted for 21.1% (n = 1,431,775) of all Medicare cases, with a median number of 7706 procedures. Endopyelotomy was the least common procedure (n = 98), while cystoscopy was the most common (n = 980,623). Medicare case volume was positively correlated with graduating residents' procedural confidence (r = 0.86, p < 0.0001). We identified four categories with varied alignment of training and demand: (1) high volume and high confidence, (2) high volume and low confidence, (3) low volume and high confidence, and (4) low volume and low confidence. CONCLUSIONS: Optimizing urology residency training is time-sensitive and important. Using national Medicare data coupled with recently graduated urology resident survey results, we provide a guiding framework for improving the alignment of training with workforce demand. Informed by these results, we recommend altering training requirements to reflect these needs. ( J Surg Ed 000:1À8. Ó 2020 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)

There is increasing concern that surgical residents are not competent to enter independent practice by the time they complete their residency training. 1-3 Survey data from recent urology residency graduates suggests similar trends. In a study by Okhunov et al., for example, 61% of recent residency graduates reported not being able to independently perform a robot assisted laparoscopic prostatectomy. 4 These findings are especially concerning in the face of an anticipated workforce deficit, driven by projections showing an alarming shortage of urologists over the coming decades [5] [6] [7] and an aging patient population that has increased urologic need. 8 While determination of surgical competency is ultimately at the discretion of the residency program director, the Accreditation Council for Graduate Medical Education (ACGME) provides recommendations in the form of minimum case requirements to help guide this decision. In a recent longitudinal case log study, Silvestre et al. found that urology residents consistently reported more cases than the minimum requirement. 9 Nonetheless, it appears that although residents are meeting case requirements, residency programs are training urologists who may not be ready for practice. One aspect that has yet to be explored is whether reported case logs and resident perceived readiness for practice are aligned with the procedural demand of the current workforce. It goes without saying that, upon graduation, residents should be most comfortable and competent with the procedures they will encounter in independent practice.

For these reasons, we conducted a correlative study to determine whether resident case logs from the ACGME and perceived readiness for surgical practice among graduating urology trainees are aligned with procedural demand and current workforce needs. Not only will our findings offer insight on the current state of urology residency training, but they may also provide a framework to optimize training in order to meet the increasing demands of a diminishing workforce.

A correlative study design was used to evaluate the association between (1) workforce demand and perceived resident competency, and (2) workforce demand and ACGME case requirements. For this analysis, 3 distinct data sets were used, including: (1) the 2017 Medicare Part B National Summary Data File; (2) the 2017 National Data Report published by the ACGME; and (3) a graduating resident survey from Okhunov et al. 4 Medicare Part B National Summary Data File To determine current procedural demand, we used the Part B National Summary Data File, which is a summary of allowed services, charges, and payment amounts billed to Medicare during a fiscal year. 10 We selected this dataset for several reasons: (1) it is publicly available through the Centers for Medicare & Medicaid Services website; (2) a large proportion of the United States population is covered under Medicare (17.4% in 2017) 11 ; and (3) since Medicare insurance covers patients 65 and older, procedural demand, as determined by this dataset, will likely be the most representative of future urologic need given our aging patient population.

Graduating resident case logs were then determined by using the 2017 National Data Report published by the ACGME. 12 This report contains information on the current number of training programs, total graduating residents, and descriptive statistics on the national procedural experience, including recommended minimum case requirements for certification. Urology resident case logs were correlated with workforce demand by using common Current Procedural Terminology (CPT) codes provided by the ACGME. 13 Pediatric urology procedure categories were excluded from our analysis as these are not billed through Medicare.

Finally, survey data from Okunov et al. 4 were used to determine if graduating resident perceived readiness for practice was correlated with urology workforce demand. In this study, a 45-item survey was administered to residents in their chief year or within five years of graduation. Survey topics included residency program characteristics, research and clinical training, postgraduate training decision, and perceived surgical proficiency in various urological procedures. Fifteen index procedures were chosen by 2 authors (APC and KHK) based on a spectrum of proficiency as determined by absolute confidence level in Okhunov et al. Table 4 . CPT codes used for the 15 index procedures are present in Supplementary Table 1 . Relevant CPT codes were added to determine total number of cases performed for an index procedure. For example, for a percutaneous nephrolithotomy (PCNL), we added cases billed for procedure codes 50080 and 50081 to determine total number of PCNL cases in 2017.

Descriptive statistics and nonparametric Spearman correlation coefficients were used to characterize the relationship between graduating resident case logs, perceived resident proficiency, and urology workforce case volume. Correlations with a p value less than 0.05 were defined as statistically significant. Case numbers in figures were log-transformed to improve data interpretation. All analyses were conducted using SAS (version 9.4, SAS Institute, Cary, North Carolina). This study was deemed exempt from review by the University of Michigan Institutional Review Board.

In 2017, there were a total of 297 graduating urology residents and 6,784,696 urologic procedures performed through Medicare. We found that the most common ACGME case log category billed to Medicare was "transrectal ultrasound/prostate biopsy" with a total of 307,525 cases, while the least commonly billed category was "Male urethra" with 850 cases ( Table 1 ). The case log category with the lowest number of median cases per resident was "male urethra" with 16, while the highest was "ureteroscopy" with 225 cases. Furthermore, there was significant interresident case log variability by procedure categories. For example, the number of "ureteroscopy" cases performed by a single resident ranged from 84 to 764 (Supplementary Fig. 1 ). There were three procedure categories (i.e., urodynamics, shock wave lithotripsy, and female) where the resident with minimum number of logged cases did not meet recommended ACGME procedure requirements. Finally, we found that Medicare case volume had nonsignificant positive correlations with recommended ACGME minimum case requirements (rho = 0.42, p = 0.1255; Fig. 1 ) and median resident case logs by procedure category (rho = 0.16, p = 0.5784; Fig. 2 ).

Our 15 index procedures accounted for 21.1% (n = 1,431,775) of total Medicare cases. The median number of cases for the index procedures was 7706, with a range of 84-980,623 (endopyelotomy and cystoscopy, respectively; Table 2 ). Graduating resident proficiency for the chosen index procedures ranged from 0% for endopyelotomy and 100% for transurethral resection of bladder tumor. A graphical representation of case volume by resident proficiency is present in Fig. 3 ; procedure volume above the median and procedure proficiency above 50% were classified as "high volume" and "high proficiency," respectively. Conversely, procedure volumes below the median were classified as "low volume" and proficiency scores below 50% were described as "low proficiency." We found that there was a significant positive correlation between reported graduating resident proficiency and Medicare case volume (r = 0.86, p < 0.0001). For example, cystoscopy had a resident proficiency of 91% and almost 1 million cases in Medicare, whereas urethroplasty had a proficiency of 26% and only 1304 total cases.

Amidst an anticipated workforce shortage, aligning urologic training to achieve resident competency with current workforce demand is time-sensitive and important. To our knowledge, this is the first study to assess whether urologic residency training is aligned with realworld procedural demand. Using national Medicare data, we found that recommended ACGME minimum case requirements and median resident case logs have nonsignificant positive associations with procedures performed by the current urology workforce. We further coupled this national Medicare dataset with self-reported resident proficiency to provide a guiding framework for improving the alignment of training with workforce demand. Previous research suggests that minimum case requirements set by the ACGME are not reflective of the learning curve required to achieve competence in these procedures. 14 The present study adds to these concerns by showing that current minimum case requirements and median resident case logs are not significantly associated with the needs of the workforce. If our goal is to develop urologists that are surgically competent for independent practice, it appears that one possible solution would be to adjust minimum case requirements, both by increasing and decreasing, to better reflect current urologic procedural demand. Although changes in surgical volume do not necessarily equate to changes in competence, there remains a strong association between the two. 15 Moreover, considering that residents are performing less cases due to an 80-hour work week restriction, 16 adjusting case requirements is an attractive option, as it may lead to better utilization of a resident's limited time.

In an attempt to better optimize residency training to meet workforce demand, we created a guiding framework using Medicare data and self-reported resident proficiency in 15 index cases. We identified 4 distinct categories with varied alignment of training and demand: (1) high volume and high proficiency, (2) high volume and low proficiency, (3) low volume and high proficiency, and (4) low volume and low proficiency. Using this framework, we believe that case requirements can be decreased for procedures that fall under the "high volume and high proficiency" category (e.g., cystoscopy) to allow for mastery of procedures with lower resident proficiency but high workforce demand (e.g., robotic prostatectomy and PCNL). Moreover, we found that several procedures fell under the "low volume and low proficiency" category (e.g., urethroplasty). It is possible that the decreased resident proficiency is not only driven by lower exposure but also by the higher complexity of these procedures. Nonetheless, procedures in this category appear to have lower utility to the workforce and, thus, the ACGME may be able to decrease requirements for resident graduation or suggest further specialization through fellowship training. By implementing changes 

guided through this framework, we may help direct training time and effort into improving competency in surgical procedures with high workforce demand.

Adjustment of minimum case requirements is but one potential approach to improving competence and better align training with workforce demand. For example, driven by an effort to improve medical education and concerns over resident competency, there has been a push to adopt a competency-based approach to medical education. [23] [24] [25] Most recently, Entrustable Professional Activities (EPAs) have gained traction as an assessment method across medical subspecialties and the ACGME. 23, 26 Originally developed in 2005 by Olle ten Cate, EPAs represent "essential tasks of a discipline that an individual can be trusted to perform independently." 23 For example, the ability to "evaluate and manage a patient with an inguinal hernia" would represent an EPA for general surgery. 26 Upon graduation, general surgery residents should be capable of independently performing this task. EPAs thus provide a tangible way of evaluating residents' surgical competence and performance beyond global assessments such as ACGME Core Competencies and Milestones. 23 While a pilot study on EPA feasibility in general surgery is still underway, 26 in the future it might be possible to create EPAs for the urologic procedures that will be most required by the workforce. One such procedure is robotic prostatectomy. As seen in our guiding framework, robotic prostatectomy is an outlier and falls under the category of high workforce demand but low resident proficiency. Moreover, the median number of robotic prostatectomies performed by graduating residents is nearly 4 times higher than the minimum requirement. It appears that although residents are significantly surpassing ACGME requirements, only about one-quarter of graduating residents report proficiency upon graduation. Thus, increasing training requirements alone would likely be ineffective in improving resident proficiency for this procedure; creating an EPA and adopting a competency-based requirement for graduation might be a better alternative. Furthermore, with improving technology, surgical simulation represents another opportunity for increasing competence in robotic/laparoscopic cases that fall under the "high volume and low proficiency" category, while minimizing patient risk. 9, 17, 18 Based on survey data, nearly all residency programs have surgical training facilities with various forms of simulation. 4 Moreover, there is increasing work and interest in developing urologic training curricula with the use of simulation. [19] [20] [21] For example, the EAU Section of Robotics has developed a validated 12-week training program for robotic prostatectomy. 19, 22 Implementation of similar curricula may help improve the competence of residents training at programs with decreased robotic/laparoscopic volume.

This study must be considered in the context of several limitations. First, Medicare only covers patients over the age of 65, raising concern that our case numbers are not entirely representative of current workforce demand 

and, therefore, presents a bias towards the needs of an older population. Nonetheless, because aging patients require more urologic services, 8 and this patient group is expected to make up one-fifth of the entire United States population by 2030, 27 these results will likely be reflective of future need. Second, in the survey used to determine resident proficiency, a disproportionate percentage of responding residents (72%) pursued fellowships. It is important to note that approximately twothirds of residents who pursued fellowships did so because of perceived deficiencies in training, highlighting the need for significant adjustments. Third, only commonly utilized CPT codes 13 were used to calculate Medicare demand for ACGME procedure categories, meaning that less common procedures that may satisfy these requirements were left out of case totals. Although this is true, we have no reason to believe that using additional CPT codes for less commonly logged procedures would significantly impact the trends and findings of this study. Finally, when generating our framework for improving the alignment of training and workforce demand, we used the median number of cases for our 15 index procedures as an arbitrary cut-off to designate high versus low volume demand. Although this cut-off will change depending on the index procedures that are chosen, we believe that the generated framework provides a conceptual way of thinking when developing solutions for procedures with low resident proficiency.

In summary, our collective findings reveal a discrepancy between ACGME minimum case requirements, resident case logs, and procedures performed among the current urology workforce. These findings present an opportunity to better align urologic residency training to produce competent surgeons that can meet the operative needs of independent practice. The need to optimize surgical training is not unique to urology. Due to a recent halt in intraoperative training secondary to the COVID-19 pandemic, residency programs have had to think more about how best to prepare residents for completion of residency and transition to independent practice. Moving forward, we believe that other surgical specialties should conduct similar analyses.

Readiness of US general surgery residents for independent practice

Our trainees' confidence: results from a national survey of 4136 US general surgery residents

General surgery residency inadequately prepares trainees for fellowship: results of a survey of fellowship program directors

Evaluation of urology residency training and perceived resident abilities in the United States

Projecting the urology workforce over the next 20 years

Recent trends in the urology workforce in the United States

Current and future need for academic urologists in the United States

The demographic burden of urologic diseases in America

Longitudinal assessment of adult cases performed by graduating urology residents in the United States

The 2017 Part B National Summary Data File. Centers for Medicare & Medicaid Services

the United States from

Urology Case Logs-Nationa Data Report. Accreditation Council for

Index Categories, Minimum Numbers, and Common CPT Codes

Male urethral, penile, and incontinence surgery: is resident exposure adequate?

Effect of the 80-hour work week on resident operative experience in general surgery

High educational impact of a national simulation-based urological curriculum including technical and non-technical skills

Urology technical and nontechnical skills development: the emerging role of simulation

Simulation-based training and assessment in urological surgery

Development and content validation of the Urethroplasty Training and Assessment Tool (UTAT) for dorsal onlay buccal mucosa graft urethroplasty

Simulation in Urological Training and Education (SIMU-LATE): Protocol and curriculum development of the first multicentre international randomized controlled trial assessing the transferability of simulation-based surgical training

Pilot validation study of the European Association of Urology Robotic Training Curriculum

Competency-based resident education: the United States perspective

Urology milestones 2.0: the future looks bright

Accreditation Council for Graduate Medical Education: the urology milestone project

Entrustable professional activities in general surgery: development and implementation

The next four decades-the older population in the United States

SUPPLEMENTARY INFORMATION Supplementary material associated with this article can be found in the online version at doi:10.1016/j