key: cord-1047196-igsn3fv9
authors: Castro, Maria R.H.; Calthorpe, Lucia M.; Fogh, Shannon E.; McAllister, Sophie; Johnson, Christopher L.; Isaacs, Eric D.; Ishizaki, Allison; Kozas, Anna; Lo, Daphne; Rennke, Stephanie; Davis, John; Chang, Anna
title: Lessons From Learners: Adapting Medical Student Education During and Post COVID-19
date: 2021-04-05
journal: Acad Med
DOI: 10.1097/acm.0000000000004148
sha: 9e90355d7cdca9f9019291cebd43d2de22f37b05
doc_id: 1047196
cord_uid: igsn3fv9

In response to the COVID-19 pandemic, many medical schools suspended clinical clerkships and implemented newly adapted curricula to facilitate continued educational progress. While the implementation of these new curricula has been described, an understanding of the impact on student learning outcomes is lacking. In 2020, the authors followed Kern’s 6-step approach to curricular development to create and evaluate a novel COVID-19 curriculum for medical students at the University of California San Francisco School of Medicine and evaluate its learning outcomes. The primary goal of the curriculum was to provide third- and fourth-year medical students an opportunity for workplace learning in the absence of clinical clerkships, specifically for students to develop clerkship-level milestones in the competency domains of practice-based learning and improvement, professionalism, and systems-based practice. The curriculum was designed to match students with faculty-mentored projects occurring primarily in virtual formats. A total of 126 students enrolled in the curriculum and completed a survey about their learning outcomes (100% response rate). Of 35 possible clerkship-level milestones, there were 12 milestones for which over half of students reported development in competency domains including practice-based learning and improvement, professionalism, and interpersonal and communication skills. Thematic analysis of students’ qualitative survey responses demonstrated 2 central motivations for participating in the curriculum: identity as physicians-in-training and patient engagement. Six central learning areas were developed during the curriculum: interprofessional teamwork, community resources, technology in medicine, skill-building, quality improvement, and specialty-specific learning. This analysis demonstrates that students can develop competencies and achieve rich workplace learning through project-based experiential learning, even in virtual clinical workplaces. Furthermore, knowledge of community resources, technology in medicine, and quality improvement was developed through the curriculum more readily than in traditional clerkships. These could be considered as integral learning objectives in future curricular design.

In response to the COVID-19 pandemic, many medical schools suspended clinical clerkships and implemented newly adapted curricula to facilitate continued educational progress. While the implementation of these new curricula has been described, an understanding of the impact on student learning outcomes is lacking. In 2020, the authors followed Kern's 6-step approach to curricular development to create and evaluate a novel COVID-19 curriculum for medical students at the University of California San Francisco School of Medicine and evaluate its learning outcomes. The primary goal of the curriculum was to provide third-and fourth-year medical students an opportunity for workplace learning in the absence of clinical clerkships, specifically for students to develop clerkship-level milestones in the competency domains of practicebased learning and improvement, professionalism, and systems-based practice. The curriculum was designed to match students with faculty-mentored projects occurring primarily in virtual formats. A total of 126 students enrolled in the curriculum and completed a survey about their learning outcomes (100% response rate). Of 35 possible clerkship-level milestones, there were 12 milestones for which over half of students reported development in competency domains including practicebased learning and improvement, professionalism, and interpersonal and communication skills. Thematic analysis of students' qualitative survey responses demonstrated 2 central motivations for participating in the curriculum: identity as physicians-in-training and patient engagement. Six central learning areas were developed during the curriculum: interprofessional teamwork, community resources, technology in medicine, skill-building, quality improvement, and specialty-specific learning. This analysis demonstrates that students can develop competencies and achieve rich workplace learning through projectbased experiential learning, even in virtual clinical workplaces. Furthermore, knowledge of community resources, technology in medicine, and quality improvement was developed through the curriculum more readily than in traditional clerkships. These could be considered as integral learning objectives in future curricular design.

The COVID-19 pandemic drastically and abruptly changed the global medical education landscape. Most medical schools were forced to alter planned curricula and suspend clinical clerkships, necessitating innovation to continue training students. 1, 2 As medical educators move from the initial phase of immediate curricular adaptations due to COVID-19 into the next phase of medical education design, there is an urgent need to examine outcomes data to articulate learning gaps and guide future innovations.

In the face of limited to no availability of clinical clerkships at the outset of the COVID-19 pandemic, undergraduate medical education leaders redesigned workplace learning experiences for clerkship students. 3 Workplace learning is experiential learning in the clinical setting through which learners apply the clinical knowledge, skills, and behaviors of their professional roles and receive feedback in real time. 4 Clerkships are vital for medical student workplace learning, [5] [6] [7] enabling participation in clinical activities through workplace affordances that allow exploration of tasks and activities, relationships and interpersonal dynamics, and rules and norms. Workplace learning encourages learner engagement and shapes knowledge, skills, attitudes, and beliefs. 8, 9 To offer clinical learning opportunities without extending time to graduation, educators created novel workplace learning experiences for clerkship students. These new student roles mitigated exposure risk and involved students interfacing with patients via telemedicine, coordinating public health responses, and developing patient, provider, and community resources. 10, 11 While the implementation of some of these COVID-19-adapted medical curricula has been previously described, an understanding of student learning outcomes is, to our knowledge, lacking. [12] [13] [14] [15] We therefore set out to apply Kern's 6-step approach to curriculum development to the design and evaluation of a novel COVID-19 curriculum for clerkship medical students. [16] [17] [18] [19] Those 6 steps are:

• Problem identification and general needs assessment;

• Targeted needs assessment;

• Goals and objectives;

• Educational strategies;

• Implementation; and

• Evaluation and feedback.

We analyzed student projects, motivations, competencies, and learning themes. Here, we report medical student learning outcomes and lessons learned to inform education innovations aimed at engaging and empowering students in workplace learning during and after the COVID-19 pandemic.

Step 1: Problem identification and general needs assessment

In March 2020, the Association of American Medical Colleges recommended pausing clinical clerkships because of changed circumstances due to COVID-19. 20 That week, clinical clerkships were suspended at the University of California San Francisco School of Medicine (USCF), leaving third-and fourth-year medical students lacking a mechanism for workplace learning experiences.

Step 2: Targeted needs assessment

The associate dean for curriculum charged a group of faculty involved in project-based curriculum development for preclinical students with creating an educational opportunity for clerkshiplevel students. Four clerkship medical students participated in virtual focus groups with this curricular team and identified 2 priority needs: learners' need for safe and experiential workplace learning and the health system's need for additional support responding to new and changing problems due to conditions created by COVID-19.

The primary goal of the UCSF COVID-19 response curriculum was to provide an opportunity for workplace learning in the absence of traditional clinical clerkships. Specifically, our aim was for students to fill identified health systems gaps, and in doing so, develop clerkshiplevel milestones in the competency domains of practice-based learning and improvement, professionalism, and systems-based practice. Development in the competency domains of patient care, medical knowledge, interpersonal and communication skills, and interprofessional collaboration were not primary goals because student participation in direct in-person patient care was discouraged and students would likely be working primarily via virtual platforms. An additional goal was to enable students to continue career and specialty exploration.

Specific course objectives were to:

• Describe the challenges associated with altered health system resources and restrictions on traditional physicianpatient interactions in a pandemic;

• Articulate the changing roles of the health care team and the medical system to continue to provide excellent patient care in the pandemic environment;

• Within the context of the health care team, seek and select evidencebased strategies for an identified gap around clinical quality, safety, patient experience, or value;

• Contribute to care delivery efforts to improve the quality, safety, patient experience, or value for a panel or population of patients during a pandemic;

• Respond to mesosystem (e.g., community or public health) needs regarding access to care, health and health care disparities, and intersystem links between care providers (e.g., hospital to clinic, prison to community); and

• List one's own roles and responsibilities and those of professionals on the health care team and seek feedback throughout the experience.

The COVID-19 response curriculum was designed to match clerkship medical students with projects led by faculty at 3 UCSF-affiliated health systems: the UCSF Health system, the city and county San Francisco General Hospital (ZSFG)/ Department of Public Health, and the San Francisco Veterans Affairs (SFVA) Health Care System.

Faculty-mentored COVID-19 clinical projects for students were identified by health system site teams, composed of faculty and students affiliated with each site. These teams identified faculty partners from a variety of specialties to solicit and brainstorm experiences that would target the pandemic response. Faculty were encouraged to design experiences that could accommodate individual student availability, such that students could determine their own weekly participation schedules. Site teams anticipated that this structural flexibility could empower students to take ownership of their learning in the rapidly evolving context of the pandemic.

Faculty with potential projects met with site teams to discuss logistics and feasibility before submitting a project proposal. Site teams provided guidance to project leaders and assessed the educational value of the opportunities. Projects that did not meet logistical requirements (e.g., required exposure to COVID-positive patients, conflicted with mandatory student didactics) or were assessed as not providing sufficient educational value (e.g., lacked workplace learning component, data-entry only roles) were not assigned students.

Step 5: Implementation

All third-and fourth-year students were offered the opportunity to sign up for the curriculum and none were required to enroll. Students were queried by electronic survey about their health system onboarding access, scheduling availability, language fluency, and general interests. Students were then matched to projects by the curricular design team, with special consideration given to matching students to projects relating to their interests (e.g., pediatrics, population health). The number of students matched to each project was based on project need. Elective units, not required for graduation but reflected on students' final transcripts, were offered for participation. Students received a pass/no pass grade for this curriculum.

We encountered several challenges while establishing the curriculum. First, the immediate high demand from clinical sites interested in student participation challenged our administrative capacity. This high demand was likely due to the increased burden on the health system from COVID-19 combined with our clinical faculty's enthusiasm to continue teaching students. We addressed this difficulty by triaging projects by urgency, quickly matching students to active projects and efficiently supporting faculty with timely and short planning meetings. Second, we were challenged to ensure equity for students when assigning projects, as some students were interested in participating in multiple projects and had experience and skills applicable to several projects. We offered all students 1 project before allowing students to participate in multiple projects. Due to the high degree of faculty engagement, we were able to offer sufficient and equal opportunity for students to participate in projects aligned with their interests.

Step 6: Evaluating the effectiveness of the curriculum

We queried students by electronic Qualtrics survey following completion of the curriculum (version XM; Qualtrics, Provo, Utah; 2020) (the full survey may be found at https://ucsf.co1.qualtrics. com/jfe/form/SV_cZvkmbVNOov984B). The survey explored students' narrative motivations, goals, and learning outcomes and also included a checklist for learners to self-identify UCSF clerkship-level competency milestones developed during the curriculum in competency domains of patient care, medical knowledge, practicebased learning and improvement, interpersonal and communication skills, professionalism, systems-based practice, and interprofessional collaboration. 21 Students were required to complete this survey to receive academic credit for curriculum participation.

Survey responses were de-identified before coding. We tabulated quantitative data, including student appraisal of clerkship-level competency milestones developed during participation in the curriculum, in Excel (version 16.41; Microsoft; 2020).

Qualitative data included student responses about motivations for participating in the curriculum and learning points taken away from the curriculum. We conducted thematic analysis using the 6-step method described by Kiger and colleagues, adopted from Braun and Clarke's model. 22, 23 An inductive approach was taken, deriving themes from the data without a deductive preexisting framework. First, 3 authors involved with development of the curriculum, namely, 2 third-year medical students (L.M.C. and M.R.H.C.) and 1 faculty member (S.E.F.), familiarized themselves with the data by individually reading through all student survey responses. Then, responses were divided and distributed to each of the 3 authors, who began generating initial codes within their subset, allowing for multiple codes per response and reviewing responses as new initial codes emerged. The team then compared, contrasted, and combined initial codes identified within their subset of responses. Using these updated codes, the team synthesized and mapped out themes that emerged. After developing this set of themes and subthemes, authors (L.M.C., M.R.H.C., and S.E.F.) reviewed it to ensure these accurately reflected responses. Team members were then assigned a different subset of responses to review to ensure correct assignment and accurate reflection of themes and subthemes, such that all responses were reviewed by at least 2 team members. Before data analysis, the student authors received training in thematic analysis from faculty authors with extensive experience in this method.

A total of 126 medical students enrolled in the curriculum and submitted their postcurriculum survey (100% response rate). Student demographics are presented in Table 1 . A total of 38 projects across the 3 health systems (academic, county, and veterans) were created, of which 32 (84%) were completed remotely. Distribution of projects across departments was as follows:

• Internal medicine: 20 (53%)

• Obstetrics-gynecology: 4 (11%)

• Public health: 3 (8%)

• Emergency medicine: 3 (8%)

• Pediatrics: 3 (8%)

• Family and community medicine: 2 (5%)

• Interventional radiology: 2 (5%)

• Psychiatry: 1 (3%)

Projects addressed a range of health system gaps, spanning 8 specialties and a variety of patient populations. Projects' aims included providing inpatient or outpatient support, assisting public health efforts, creating patient and community or provider resources, and engaging in telemedicine (see Table 2 ).

All interested students were matched with 1 or more projects. The number of students assigned to each project ranged from 1 to 11. Mean project duration was 7 weeks (range, 1-20 weeks). Sixty (48%) students participated for less than 10 hours per week, 48 (38%) for 10 to 20 hours per week, 9 (7%) for 20 to 30 hours per week, 3 (2%) for 30 to 40 hours per week, and 6 (5%) for more than 40 hours per week. All students met the objectives of the curriculum and received a passing grade for participation.

Student-identified clerkship-level competency milestones achieved through participation in their COVID-19 curricular project are presented in Table 3 . 21 Of the 35 possible milestones, there were 12 milestones for which the majority of students reported development: practice-based learning and improvement milestone 2 (57%); interpersonal and communication skills milestones 1 (86%) and 4 (60%); and professionalism milestones 1 (73%), 2 (87%), 3 (64%), 4 (76%), 5 (80%), 6 (67%), 7 (68%), 8 (56%), and 9 (66%). The most commonly identified competency domains for which over half of students reported development of milestones Gather complete and focused histories from patients, families, and electronic health records 1 (1)

With appropriate supervision, manage patients with acute and chronic illness 16 (13) Present patient information with an assessment, differential diagnosis, and initial plans 44 (35) Observe, assist, or perform procedures and skills 10 (8) Conduct a physical exam efficiently, interpreting abnormalities, and maintaining patient comfort 28 (22) Medical knowledge

Establish and maintain knowledge necessary for patient care including the etiology, pathogenesis, manifestations, treatment, and management of medical problems 54 (43) Apply an inquiry-oriented and analytic approach to learning and patient care 33 (26) Select, justify, and interpret diagnostic tests and imaging 10 (8) Solve clinical problems using a systematic approach 2 (2)

Select and use the appropriate electronic decision support tool 1 (1) Select basic preventative, curative, and/or palliative therapeutic strategies 8 (6)

Reflect on and address impacts that personal biases, identity, and privilege have on interactions 38 (30) Employ strategies for seeking, receiving, acting upon, and delivering feedback 72 (57) were professionalism (all milestones), interpersonal and communication skills (half of milestones), and practice-based learning and improvement (one-third of milestones).

Students described 2 central motivations for participating in the curriculum: fostering their identity as physicians-intraining and patient engagement.

The most prominent motivation mentioned by participants was a "desire to contribute, " arising from a sense of responsibility tied to their identity as physicians-in-training. Students strongly identified as a part of the health care system, feeling simultaneously called to action but lacking opportunities to participate and develop as clinicians.

Recognizing the increased demands and decreased support clinical health care workers were facing, many learners were motivated to alleviate health care worker burden. Students also described a sense of responsibility to use specific skill sets, including language abilities, laboratory training, or proficiency from employment before medical school. Sample learner quotes include:

I felt compelled to contribute to pandemic response as a member of the larger medical profession. I was saddened by our removal from the clinical environment and was searching for another mechanism by which to help the health system. ( 

Students' qualitative survey responses conveyed 6 central learning themes: interprofessional teamwork, community resources, technology in medicine, skill-building, quality improvement, and specialty-specific learning (Table 4) .

Interprofessional teamwork. Medical students described learning from interprofessional team members, including social workers, laboratory technicians, nurses, and pharmacists. Specifically, they witnessed how the roles of each team member are coordinated to produce an efficient team, maximizing patient health. The importance of collaboration and communication within and across teams was impressed upon learners who recounted how teamwork had been successful or could have been improved. Learners also engaged with the clinical support processes that enable the clinical workplace to function. They detailed a newfound understanding of the complexities and efficiencies, or lack thereof, of many health systems processes, including managing orders, lab processing, and consulting other services.

Many participants reported learning about the workflow, processes, and impact of community resources on patient care. Students described expanding their knowledge about the services provided by community programs, government programs, and not-for-profit organizations, noting how these services are necessary for adequate and equitable health care and help to address disparities exacerbated by the pandemic. Learners also commented on how complicated accessing these services can be and how little training medical school provides regarding this process. Further, many responses detailed a new understanding of skills needed for delivering medical care in the community, including establishing connections, building institutional trust, and developing targeted programs to support specific communities.

Technology in medicine. Students described participating in various health care technology initiatives and gaining an appreciation for how telehealth can meet patient needs. They articulated how telehealth allowed patients who might otherwise have been unable to access care, due to burden from travel, work, and/or childcare, to quickly and relatively easily connect with their health care provider. Learners also chronicled how technology such as virtual platforms could be employed to combat patient isolation, including connecting hospitalized patients unable to have visitors with their family members and providing social connection to lonely elderly community members. 

• "I've also learned so much about what happens when you order a lab test! As a physician, you just put the order in and it magically comes back in the computer a day or two later, but there are SO MANY steps and such a complex, efficient and accurate system that needs to be in place in order to make that happen." Fourth-year student, Department of Internal Medicine, UCSF Health Interprofessional team members

• "One learning point that I am taking away from the COVID Response Elective about health care and health systems is the incredible value of the interprofessional team in helping to manage and coordinate care for patients." Third-year student, Division of Geriatrics, UCSF Health • "The tremendous power of interprofessionalism, when health care professionals come together from all fields it makes for a diverse and rich experience that is unforgettable." Fourth-year student, Division of Infectious Disease, ZSFG

• "Successful advocacy for public health requires the work of multiple team members from multiple different backgrounds."

Fourth-year student, Department of Internal Medicine, UCSF Health

Value of services: Services are necessary for adequate and equitable health; Services address health care disparities

• "It was sometimes heartbreaking to talk to contacts who did not have the means or ability to isolate (sole caregivers, workers without job security, for instance). It reaffirmed for me how deeply I believe in building a stronger safety net and societal infrastructure that can support all people and how advocacy on a broader scale will be part of my career." Fourth-year student, Department of Internal Medicine, ZSFG • "Health care and health systems disparities have always existed in the U.S., but COVID-19 has further exacerbated them. It is striking to see how vulnerable and underserved communities have become 'hotspots' for the coronavirus." Fourth-year student, Department of Public Health, ZSFG • "Whether related to insurance, finances, language, transport, or health literacy, patients face so many barriers in accessing medical resources and follow up care. There are many gaps in our system that may only be apparent once the patient leaves the clinic or hospital. I learned about the critical role of anticipating barriers, searching for solutions in advance, and following up from the side of the provider or provider's office. We cannot let our vulnerable patients fall through the cracks." Fourth-year student, Department of Family and Community Medicine, ZSFG Processes of social services: Complexity of social service systems; Community medicine

• "[This city] has a network of mental health care clinics and resources for patients (including patients with no insurance), but there is often a wait list to obtain these services." Third-year student, Department of Psychiatry, ZSFG • "Learned how it may look like resources exist on paper-but how hard it can be for a patient to take advantage of those resources, due to numerous barriers and obstacles in a system. 

Value of telehealth technology to combat patient isolation

• "An effective and efficient clinic visit does not have to take place in person. There is much that can be done via telehealth in terms of managing patient needs, answering any questions, and setting future goals." Third-year student, Department of Family and Community Medicine, ZSFG • "I think one learning point that I will take away is that health care workers do have a role in tech innovation in health care."

Fourth-year student, Department of Internal Medicine, UCSF Health

• "I've learned a lot about delivering bad news to patients (e.g., a positive COVID-19 test) during the clinical training I received through this elective. While I had received similar training in the past, it was very helpful to be able to relate that preclinical training to an actual situation" Third-year student, Department of Internal Medicine, SFVA • "I'm learning how to have end of life discussions." Third-year student, Department of Internal Medicine, ZSFG Public health

• "I've learned about the importance of translating medical knowledge to the community. I think I have really experienced how important it is for the work we do to be community based-that means seeking to get public health information out to people in an accessible culturally sensitive manner." Third-year student, Department of Family and Community Medicine, ZSFG Therapeutic empathy

• "A physician can be helpful just by exercising humanity, medical knowledge is not everything." Third-year student, Division of Geriatrics, UCSF Health • "Continue to trial-and-error processes. Plan to make a change, implement it, observe the outcomes of that change, and adjust accordingly." Third-year student, Department of Internal Medicine, SFVA • "Recognizing the limitations of our knowledge while continuing to work with the information that we have is key to efficient mobilization. Remaining adaptable as our knowledge grows is important to creating high quality care systems." Third-year student, Department of Obstetrics and Gynecology, ZSFG • "Quality health care is both reactive and proactive." Third-year student, Department of Internal Medicine, UCSF Health Specialty-specific learning offer this substantial benefit to patients. By genuinely listening and providing compassionate support, they learned to develop a therapeutic alliance with patients.

Quality improvement. Learners engaged in quality improvement at the level of teams, departments, and systems as they conducted Plan-Do-Study-Act cycles, 24 continually reviewing and updating their projects based on feedback and outcomes. Students were enthusiastic and encouraged by this fast-paced, iterative process and expressed interest in participating in quality improvement work in their future professional endeavors.

Specialty-specific learning. Participants explained how specific projects enabled them to explore their specialty interests, including in pediatric ophthalmology, radiology, obstetrics-gynecology, and emergency medicine. This filled a gap in students' available opportunities to explore their career interests and allowed them to develop relationships with faculty in specific specialties.

While clerkships are widely recognized as the central mechanism for clinical learning, 5-7 students can have rich workplace learning experiences in nontraditional workplaces. Traditionally, clinical skills and competencies are learned through an apprenticeship model in direct patient care, but the curriculum we developed demonstrates how virtual clinical workplaces and project-based teaching strategies can also enable clerkship students to develop competencies and achieve valuable workplace learning. This is not to suggest that clinical clerkships can successfully be conducted entirely remotely or in a project-based format, but instead that traditional clinical clerkships could be extended to include other learning opportunities and clinical settings (e.g., quality improvement, telemedicine, public health). For example, with the return to traditional clerkships, students could continue to be involved in telehealth outreach to vulnerable patient populations to address their health needs.

We found that an advantage of this curricular structure was the flexibility afforded to students to guide their own learning. Students primarily set their own schedules and often conducted projects remotely. Almost half of the projects required less than 10 hours per week. Many medical schools have recently focused on adapting curricular structure to support student wellness, 25 and as medical schools move curricula to virtual settings, we caution against overloading students with rigid, lengthy virtual learning and inducing virtual platform fatigue. 26 Our findings indicate that students can achieve important learning when they are empowered with flexibility in their education and schedule and when the learning opportunities match their interests and serve the community.

This curriculum enabled students to learn important clinical doctoring skills and health systems science knowledge that are missing from clinical clerkships as they are currently designed. 27, 28 Some of the themes students' learning embodied, such as interprofessional teamwork and clinical skill-building, are typically included in undergraduate medical education. However, familiarity with community resources, technology in medicine, and quality improvement, described by students and recognized by educators as essential components of medical education in preparing students to be effective physicians, have been less consistently included as learning objectives in the traditional clerkship curriculum. 11, [29] [30] [31] We posit that a curricular model such as the one we describe here could be incorporated to complement traditional clerkship experiences as a means of including these important health systems science topics throughout undergraduate medical education. 27, 28 Observations and Conclusions

Our application of Kern's framework to a novel COVID-19 curriculum sheds light on the student experience of COVID-19 curricula and may guide future curricular innovations in the context of a sudden and massive global or national event affecting medicine and medical education.

Our COVID-19-adapted curriculum achieved our primary goal of providing students with opportunities to engage in clinical workplace learning to advance the medical school's educational competencies. A majority of students reported development in clerkship-level competency milestones in our target competency domains of professionalism and practice-based learning and improvement, and surprisingly in interpersonal and communications skills as well. 21 Our thematic analysis of individual student experiences demonstrated that students cultivated important skills, including interprofessional teamwork, knowledge of community resources, technology in medicine, quality improvement, skill-building, and specialty-specific learning. Specialtyspecific learning matched to career interests was a difficult aim to achieve during the pandemic otherwise but was possible with this curriculum.

Strengths of our program evaluation include our intentional design of the evaluation strategy at the start of the curriculum and our systematic methods of thematic analysis. Multiple researchers reviewed each survey response and iteratively discussed and reevaluated the themes and subthemes we developed. A limitation of our evaluation is the lack of additional assessments to supplement data on students' self-reported learning. However, as students received a passing grade for participating in the curriculum, there was no incentive for students to overreport learning that did not occur.

As we transition back to prior curricula or continue with newly created educational formats, we must heed these lessons and intentionally design educational experiences that maximize student growth and prepare students to be effective physicians. This includes creatively providing workplace learning opportunities, integrating topics central to health care delivery, such as community resources, technology in medicine, and quality improvement, and empowering students with flexibility in their learning environment.

Acknowledgments: The authors acknowledge the commitment and creativity of the faculty and students who participated in the University of California San Francisco School of Medicine COVID-19 response curriculum. The authors thank Therese Denoga and Patricia Runquist for their assistance with curriculum implementation and data collection, as well as Dr. Bridget O'Brien for her manuscript review and thoughtful guidance.

Medical student education in the time of COVID-19

Association of American Medical Colleges. Coronavirus (COVID-19) resource hub

Medical and surgical education challenges and innovations in the COVID-19 era: A systematic review

Workplace learning

Learning styles and learning spaces: Enhancing experiential learning in higher education

Experiential learning: AMEE guide no. 63

Perspective: Successfully negotiating the clerkship years of medical school: A guide for medical students, implications for residents and faculty

Learning in the Workplace: Strategies for Effective Practice

Experience-based learning: A model linking the processes and outcomes of medical students' workplace learning

Medical student mobilization during a crisis: Lessons from a COVID-19 medical student response team

A role for telemedicine in medical education during the COVID-19 pandemic

Student engagement in a public health initiative in response to COVID-19

Medical students for health-care staff shortages during the COVID-19 pandemic

Contributions of health professions students to health system needs during the COVID-19 pandemic: Potential strategies and process for U.S. medical schools

Integrating a COVID-19 volunteer response into a year-3 MD curriculum

Application of Kern's 6-step approach in the development of a novel anesthesiology curriculum for perioperative code status and goals of care discussions

Application of Kern's six-step approach to curriculum development by global health residents

A stepwise model for simulation-based curriculum development for clinical skills, a modification of the six-step approach

Curriculum Development for Medical Education: A Six-Step Approach

Important guidance for medical students on clinical rotations during the coronavirus (COVID-19) outbreak

Thematic analysis of qualitative data: AMEE guide no

Using thematic analysis in psychology

The Improvement Guide: A Practical Approach to Enhancing Organizational Performance

Association between learning environment interventions and medical student wellbeing: A systematic review

Why is zoom so exhausting? Chronicle of higher education

Priority areas and potential solutions for successful integration and sustainment of health systems science in undergraduate medical education

Value-added clinical systems learning roles for medical students that transform education and health: A guide for building partnerships between medical schools and health systems

Engaging with communities-Lessons (re)learned from COVID-19

Quality improvement in medical education: Current state and future directions

COVID-19: Lessons from the disaster that can improve health professions education

Funding/Support: None reported.