key: cord-0961268-41xg6ylp authors: Thoma, Brent; Monteiro, Sandra; Pardhan, Alim; Waters, Heather; Chan, Teresa title: Replacing high-stakes summative examinations with graduated medical licensure in Canada date: 2022-02-07 journal: CMAJ DOI: 10.1503/cmaj.211816 sha: f433cc6ba336bd2fb8459ff331a8b90e3fb751d4 doc_id: 961268 cord_uid: 41xg6ylp Chan et al asserts that the COVID-19 pandemic has disrupted the medical assessment system in Canada. Examinations delivered by The College of Family Physicians of Canada (CFPC) and the Royal College of Physicians and Surgeons of Canada (RCPSC) were delayed, cancelled or adapted. As these examinations are required for independent practice, this had a negative impact on trainees, supervisors and patients during a time of great stress within our health care system. Although disruptive, these challenges provided an opportunity for change by unfreezing the historical approach to medical licensure in Canada--an approach that can be characterized as arduous, expensive, logistically challenging, poorly aligned with clinical practice and potentially biased. The ongoing use of high-stakes examinations for licensure has numerous unintended consequences. Standardized exams test nonsalient variables and are at risk of biases (e.g., financial hardship from paying to write and attend examinations, structural racism affecting examination literacy and preparation) that may inhibit the movement of our health care institutions toward equity, diversity and inclusion. T he COVID-19 pandemic has disrupted the medical assessment system in Canada. Examinations delivered by The College of Family Physicians of Canada (CFPC) and the Royal College of Physicians and Surgeons of Canada (RCPSC) were delayed, cancelled or adapted. As these examinations are required for independent practice, this had a negative impact on trainees, supervisors and patients during a time of great stress within our health care system. 1 Although disruptive, these challenges provided an opportunity for change by unfreezing the historical approach to medical licensure in Canada -an approach that can be characterized as arduous, expensive, logistically challenging, poorly aligned with clinical practice and potentially biased. The current medical licensure practices in Canada depend on high-stakes, standardized, summative examinations that were developed to uphold the medical social contract to guard patient safety and benefit society. 2 However, no evidence has shown that these exams contribute to this outcome. Although psychometrically sound exams provide superficial reassurance as a safety net that "catches" trainees who are not prepared for independent practice, the incredibly high pass rates of graduates from Canadian training programs suggest that this function is largely redundant. 3 In contrast with this approach, best practices in medical and continuing education support the use of competency-based assessments, guided by robust programmatic assessment models. Programs guided by these models employ frequent, low-stakes assessments within the clinical environment, along with specialty-wide or internally created local exams; such programs are being implemented throughout the medical education system, 4 and have resulted in increased assessment volume 5 and quality. 6 Although further evidence of competence in clinical practice may be required to maintain the social contract, licensing exams, as currently structured, do not provide this evidence. The ongoing use of high-stakes examinations for licensure has numerous unintended consequences. Standardized exams test nonsalient variables and are at risk of biases (e.g., financial hardship from paying to write and attend examinations, structural racism affecting examination literacy and preparation) that may inhibit the movement of our health care institutions toward equity, diversity and inclusion. 7 The medical licensing process directs valuable and limited resources (e.g., time, energy, focus) toward passing an exam, potentially detracting from the more meaningful goal of preparing for independent practice. Moreover, the focus on a single high-stakes examination does not support the development of lifelong, self-directed learning. These exams can also increase stress among trainees to a degree that threatens mental health and wellness. 8 Lastly, the exams remain painfully expensive for both the trainee and the regulatory body, at a time when student debt continues to soar. Although landmark changes have been made in response to vigorous advocacy and debate, 9 • No evidence supports the effectiveness of current licensure exams in ensuring patient safety; the current process causes undue stress and directs valuable and limited resources toward passing an exam, potentially detracting from the more meaningful goal of preparing for independent practice. • Best practices in medical and continuing education support the use of competency-based assessments, guided by robust programmatic assessment models, to judge fitness to practise, followed by life-long self-directed learning. • Credentialing and licensing authorities should advocate for the resources required to replace current high-stakes summative assessments with graduated licensure and to develop quantifiable, nationally synergized, specialty-specific practice standards that support both the oversight of graduated licensure and maintenance of competence. CMAJ | February 7, 2022 | Volume 194 | Issue 5 E169 from licensure requirements, passionate voices urge further modernization to increase flexibility and alignment with intended outcomes. A modern model of assessment for licensure that integrates both training programs' competency-based assessments and clinical practice data derived from trainees' clinical work would better align with evidence-informed educational practices, underscore the importance of continuous quality improvement for practitioners and refocus licensure on realworld clinical practice. 10, 11 The implementation of such a system would be challenging. Most international medical training programs use high-stakes, gatekeeping examinations similar to those used in Canada, so cannot provide robust examples to guide the way forward. Fortunately, the scaffolding for such a model already exists in the Competence By Design assessment programs (RCPSC), the Triple C Curriculum (CFPC) and the Core Professional Activities of the Residency Training Profile (CFPC). The direct observation of real-world clinical activities within these training programs occurs frequently and, when competency-based programs are implemented with fidelity, they provide substantive evidence of competence in the clinical setting. The formalization of "transition to practice" periods within these training programs could occur if this evidence was used to grant a graduated licence that allowed trainees to take on greater clinical autonomy. Trainees with graduated licences would have increased ownership of clinical outcomes that could be tracked through practice audits, supported by their training program and hospital system. The incorporation and oversight of such quality metrics would engage individuals in continuing to employ lifelong learning skills while supporting quality improvement practices within their discipline. Developing and socializing the cultural changes and quality improvement infrastructure required for this model would be resource intensive. Culturally, the creation of a graduated licence would need to be implemented without increasing medicolegal risk or decreasing compensation for supervisors. The successful socialization of this approach would offload some patient care responsibilities to the trainees with graduated licences, thereby allowing their supervising consultants to develop and oversee their practice audits. The oversight of clinical practice during this stage would require increased sophistication in how health care systems report clinical outcomes and conduct quality-improvement initiatives. These costs could be contained through other synergizing innovations. For example, nationally developed, locally implemented, discipline-specific standards for practice audits could be incorporated into both maintenance-of-competence programs and the assessment of a graduated licensure period. Logistically, the implementation of these changes would require substantial collaboration between the CFPC, RCPSC and provincial regulatory bodies. From a credentialing perspective, specialty-specific examination boards would need to be replaced with boards trained to review standardized practice audits at the end of the graduated licensure period. Additionally, new accreditation standards would need to be developed to ensure adequate oversight and support of trainees during the graduated training period. From a regulatory perspective, provincial medical licensing bodies would need to evolve to define and supervise a form of graduated licensure that provides trainees with greater clinical autonomy. If these changes were made collaboratively across Canada's provinces and territories, they could facilitate the harmonization of provincial licensure standards, or even the development of a portable national licensure that would improve physician portability. 12 By disrupting traditional approaches to examination, the COVID-19 pandemic has provided an opportunity to imagine alternative approaches to licensure that better align with best practices in education assessment, while also upholding the medical social contract. Credentialing and licensing authorities should advocate for the resources required to replace the current high-stakes summative assessments with graduated licensure; to develop quantifiable, nationally synergized, specialty-specific practice standards that support both the oversight of graduated licensure and maintenance of competence; and to streamline licensure requirements between jurisdictions. COVID-19 and its impact on CCFP(EM) residency training Medical licensure: Let's not lose sight of the objective Ottawa: The Royal College of Physicians and Surgeons of Canada The role of EPAs in creating a national system of timevariable competency-based medical education Evaluation of a national competency-based assessment system in emergency medicine: a CanDREAM study McMaster Modular Assessment Program (McMAP) through the years: residents' experience with an evolving feedback culture over a 3-year period The consequences of structural racism on MCAT scores and medical school admissions: the past is prologue The United States Medical Licensing Exam (USMLE) and medical student wellness: an ethnographic qualitative study at Cooper Medical School of Rowan University Ottawa: Medical Council of Canada Using electronic health record data to assess residents' clinical performance in the workplace: the good, the bad, and the unthinkable EMERGing" electronic health record data metrics: insights and implications for assessing residents' clinical performance in emergency medicine Physician mobility in Canada Contributors: All of the authors contributed to the conception and design of the work, drafted the manuscript, revised it critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons. org/licenses/by-nc-nd/4.0/ Correspondence to: Teresa Chan, teresa.chan@medportal.ca