key: cord-0906446-c2n66kb3
authors: Tanaka, Kara S.; Ramachandran, Rageshree
title: Perceptions of a Remote Learning Pathology Elective for Advanced Clinical Medical Students
date: 2021-04-23
journal: Acad Pathol
DOI: 10.1177/23742895211006846
sha: 94884bfbaaf37865fc167dc17039092c1e7a2b0d
doc_id: 906446
cord_uid: c2n66kb3

In mid-March 2020, our institution removed most medical students from in-person clinical clerkships due to the COVID-19 pandemic. The Department of Pathology responded by transitioning a fourth-year clinical elective to an all-remote format composed of synchronous didactics, daily clinical sign-out utilizing digital microscopy, and asynchronous learning materials. Thirty-seven medical students completed 2- or 4-week anatomic pathology electives tailored to meet their career goals and allowing them to progress toward graduation. Institutional Review Board approval was granted to survey students’ perceptions of engagement in the remote learning environment. Quantitative and qualitative data were collected using a standardized school-wide end-of-rotation survey, an online survey developed by the authors, and students’ self-directed learning goals. End-of-rotation data showed the remote pathology course performed well (4.88 of possible 5) when compared to all advanced clinical clerkships (4.51, n = 156 courses), all elective rotations (4.41, n = 50 courses), and the traditional in-person pathology elective (4.73). Core strengths in the virtual environment included high educational value, flexibility of content and schedule, organization, tailoring to an individual’s learning goals, and a positive education environment. Deficits included the inability to gross surgical specimens, inadequate observation or feedback about students’ skills, and impaired social connections. Areas for improvement included requests for in-person experiences and development of themed tracks for career exploration. Many aspects of anatomic pathology appear well-suited to the remote learning environment. While the remote model may not be sufficient for students pursuing careers in pathology, it can be adapted to increase nonpathologists’ understanding of interdisciplinary clinical collaboration with pathologists.

Clinical coursework in undergraduate medical education relies heavily on traditional in-person experiences to develop clinical skills and competence. Most rotations are dependent on the patient, provider, and learner being physically present in the same place at the same time and, until recently, there has been limited need for curricular innovation in that delivery model. Even medical specialties that can be highly integrated on digital platforms, like radiology, pathology, and clinics utilizing telehealth videoconferencing, continue to offer in-person rotations though new opportunities for remote rotations are being introduced. 1,2 Systematic reviews evaluating the efficacy of developing clinical skills in the remote learning environment have demonstrated that online teaching is comparable to in-person delivery 3 and that online instruction is equivalent or possibly superior to traditional forms of learning 4 in health professional schools. Yet, despite demonstration that online learning is not inferior to in-person learning, clinical coursework is still overwhelming dependent on in-person delivery. The heavy reliance on synchronous activities in the clinical learning environment is in contrast to the blended learning approaches commonly employed in medical school preclinical coursework that combine face-to-face seminars and small-group discussions with technology-enhanced resources, asynchronous delivery, and remote learning. 5 Preclinical students often access new material through prerecorded video lectures, while in-person sessions are reserved for active discussion and application of that newly acquired knowledge.

Medical student education in the United States was severely disrupted by the COVID-19 pandemic in the spring of 2020. The pandemic necessitated rapid and widespread institutional hospital changes centered on pandemic surge preparation including reductions in patient load and closure of clinic sites, shifts toward telemedicine video visits, and mandated county and state shelter-in-place orders. Our institution removed most medical students from in-person clinical clerkships in mid-March and kept them from returning to the wards based on recommendations from the Association of American Medical Colleges strongly urging that medical students discontinue direct patient care activities. 6 The rationale for the interruption of clinical education and the delayed return to on-site rotations included shortages of personal protective equipment, limited access to COVID-19 testing, an increased need for physical distancing, and the prioritization of worker and patient safety by limiting the number of learners in the health care clinical environment.

At the University of California, San Francisco (UCSF), students beginning their fourth and final year of medical school in April were removed from the wards, and all visiting rotations to other institutions were cancelled nationally, causing disruptions to subinternship schedules in preparation for residency application season. Concurrently, a cohort of graduating fourth-year medical students were completing their final month of clinical rotations, some of whom required additional clinical coursework to meet graduation requirements. The limited options for continued education under these unanticipated circumstances included off-site engagement in research activities, self-study preparation for board exams, a COVID-19 response elective focused on the UCSF health systems, a medical education elective, and 3 remote clinical electives including radiology, ECG interpretation, and toxicology. At the time, pathology coursework was not yet offered as a remote elective.

During the early pandemic shutdown, the School of Medicine prioritized adapting educational programs while minimizing risk to participants. To specifically meet the needs of advanced clinical students, the UCSF Department of Pathology responded by transitioning a fourth-year clinical elective to an all-remote format, offering 2-and 4-week courses during April and May of 2020. As educators, we saw this as a unique opportunity to transform the clinical experience for delivery in the remote setting and to study its utility as an alternative learning experience from the perspective of the learners using medical student-reported outcomes.

This study aims to assess medical students' perceptions of learning experiences in an advanced clinical elective course when offered via a remote learning environment, to assess the level of emotional, cognitive, and behavioral engagement with remote synchronous and asynchronous learning activities, and to determine the benefits and limitations of adapting a hands-on lab-based course for remote delivery.

Our questions are: (1) How does a clinical elective rotation perform in the remote learning environment in comparison to in-person rotations? (2) What contributes to the success of a remote learning course from the perception of the learner? (3) What learning deficits accrue as a result of the remote delivery of a traditionally hands-on course?

The study population includes fourth-year medical students at a single institution who completed a remote learning pathology elective rotation during April and May of 2020. All students meeting inclusion criteria had completed core clerkships in traditional in-person settings with limited exposure to telehealth or virtual clinical activities.

Surgical and Autopsy Pathology at our institution is a fourthyear clinical elective in which students gain experience in 3 branches of anatomic pathology including surgical pathology, cytology, and autopsy, by actively participating in the team's daily workflow. Some elements of the curriculum rely on the in-person experience such as grossing and working up surgical specimens, performing frozen sections and fine needle aspirations, completing an autopsy, and reviewing cases around a multi-headed microscope. The course objectives of this pathology elective include achieving competency in the student's ability to explain the diagnostic surgical pathology process from frozen section preparation to clinical sign-out, evaluating and dissect surgical specimens under supervision, understanding the role of fine needle aspiration cytology in clinical diagnosis, and understanding the role of the autopsy in medical diagnosis. The in-person clinical course was swiftly redesigned and adapted to meet these course objectives in the remote learning environment. The authors served as the course director and a medical student teaching assistant, both of whom played key roles in curriculum planning.

The pathology department accelerated an ongoing transition to a fully digital workflow built on whole slide imaging (WSI) to convert from glass slides to digital slides for clinical casework. On March 18, 2020, the UCSF Department of Pathology became the first pathology department in the United States to implement a comprehensive and completely digital operation for primary sign-out. After subspecialty validation, the WSI program enabled in-person multi-headed microscope sessions to transition to live-streamed digital sessions with remote reviewing access, with permission for remote sign-out granted by an emergency waiver from the Centers for Medicare & Medicaid Services. 7 The institution simultaneously adopted the Zoom videoconferencing platform (Zoom Video Communications, Inc), with access provided to all students and employees of the university, to conduct remote tumor boards, educational conferences, and clinical activities such as consensus conferences. These adaptations allowed multiple learners to attend microscope sign-out and gave students remote access to a core component of the daily workflow of surgical pathology. The new course was structured for students to participate in sign-out with physicians and residents via remote microscopy while substituting missing in-person activities like cytology and autopsy with live online seminars dedicated to those topics.

A major consideration in curriculum design was ensuring equitable access and ability to participate in the new format, thereby allowing students to meet the course objectives despite the change in delivery. Potential areas of inequity included access to a reliable internet connection and equipment for videoconferencing (eg, computers capable of streaming content, video cameras, headphones), ability to focus on learning during disruptions in the home environment (eg, responsibilities for children or other family members, quarantine demands, or active COVID-19 caretaking of self or household members), and differences in the ability to engage in and actively participate in the remote environment. To meet these variable needs, synchronous live sessions designed for medical student education were planned with consideration for timezone differences, as many students returned home during the pandemic, and live activities were limited to one or two sessions per day. Didactic sessions were recorded and made available to students for review to ensure interruptions in the home environment did not limit their access to the material. Students were given autonomy to tailor the remainder of their daily activity schedule and were offered guidance on how to maximize their learning experience. They were invited to join residents and attendings in clinical sign-out, consensus conferences, tumor boards, resident teaching conferences, unknown sessions, and departmental grand rounds, and had access to a frequently updated Outlook Calendar (Microsoft Corporation) to help plan their daily activities. Students utilized a digital library of learning resources for asynchronous independent study, which included videos, readings, and access to the departmental website containing the core anatomic pathology resident curriculum. Additional learning materials made freely available by professional societies and colleagues were shared with our students. 8 The remote elective was available for student enrollment within 2 weeks after cessation of inperson clinical activities, so while we were forward-thinking about removing as many barriers to access as possible, we did not assess formally if issues of inequity limited student involvement in the remote course.

We are cognizant that working in the remote learning environment has the potential to become an isolating experience secondary to limited interpersonal connection, and therefore we prioritized maintaining connectivity with students. The electives began with an orientation session for virtual introductions; thereafter, we offered biweekly office hours for guidance and mentoring, moderated discussions on mobile messaging apps for updates and electronic resource troubleshooting, encouraged weekly social hours with residents, and provided the opportunity to join a course book club.

Our medical students created SMART (Specific, Measurable, Achievable, Relevant, Time-bound) goals at the start of the elective to identify and attain self-directed learning goals. 9,10 Students communicated their personalized SMART goals to the course leadership who provided guidance to tailor daily activities to meet a student's stated interests and career goals. Students were also invited to investigate self-study specific topics available in the digital library.

To assess students' perceptions of learning experiences in this clinical elective course when offered via a remote learning environment, we developed an online survey eliciting students' perceptions of the course. This survey was used in conjunction with other data collected from students, including personal learning goals, future career plans in medicine, and selfidentified specific interests in the course. The survey was developed by the authors and included a 19-item instrument from a previously published study assessing emotional, cognitive, and behavioral engagement in the technologically enhanced learning environment, 11 a 14-item set of coursespecific questions created by the authors, and an 11-item standardized institutional end-of-course evaluation, including 2 open-ended questions for additional comments, for comparison of the remote course to other advanced clinical electives offered at the university. With the exception of the 2 openended free-text questions, all survey question responses utilized a 5-option Likert rating scale (1-5 respectively: strongly disagree, somewhat disagree, neither agree nor disagree, somewhat agree, strongly agree; or poor, fair, good, very good, excellent). Institutional Review Board approval (IRB 20-30955) was obtained for this investigation.

Anonymous quantitative and qualitative student course assessment and feedback was collected from 3 sources: deidentified SMART learning goals submitted by students, a standardized school-wide end-of-rotation evaluations collected using Med-Hub software (MedHub, Inc, Ascend Learning), and the previously described online survey collected using Qualtrics software (Qualtrics Labs, Inc, SAP American Inc), which was developed by the authors and was aimed at understanding the students' perceptions of the efficacy and authenticity of an advanced medical student pathology rotation when offered as a remote learning experience.

After completion of the remote learning pathology elective, participants received a recruitment email from the course director explaining the purpose of the study, an informed consent form, and a link to the 44-question anonymous online survey with an estimated completion time of 15 minutes. A reminder email was sent at 2-week intervals over a 2-month period.

Quantitative data from the online survey was used to assess the level of engagement achieved with technology-enhanced learning resources and to identify strengths and limitations specifically related to the remote pathology elective. To evaluate overall course performance, quantitative data from Med-Hub end-of-course evaluations was compared to the UCSF Clinical Electives Evaluations Dashboard (version November 2019) that included data from all clinical electives offered between May 1, 2018, and November 19, 2019. To evaluate students' perspectives on the remote learning experience, inductive content analysis was performed using qualitative data from open-ended survey responses and SMART goals. Students' responses were systematically coded into categories for theme identification by one coder. To reduce bias in content analysis, coded data and themes were verified for appropriateness by a second coder. Quantitative counts were made for each theme in order to identify the most frequent themes and frequency was used as a marker for the salience or relevance within the learning experience.

Thirty-seven medical students completed 2-or 4-week anatomic pathology electives during April and May of 2020 and met inclusion criteria for the study. One author, who served as the course teaching assistant, also enrolled as a student in the elective, and was excluded from study participation. Students enrolled in the course included 33 rising fourth-year students (MS4A) and 4 graduating fourth-year medical students (MS4B) with stated medical career interests in internal medicine (40.5%), surgery (13.5%), pathology (2.7%), other specialties such as radiology or anesthesia (24.3%), or not yet decided on a career path (18.9%; Table 1 ).

Thirty-seven students (100%) submitted SMART goals at the beginning of the elective that were deidentified and analyzed for content themes to highlight the students' learning priorities ( Table 2 ). The most frequent topics reflected in the goals were understanding the role of the pathologist, gaining medical knowledge in topics in anatomic and clinical pathology, reviewing histology, achieving career-specific goals, and strengthening interprofessional communication.

Twenty-seven students (73%) completed the MedHub end-ofrotation evaluation survey for the remote pathology elective. Content analysis of qualitative data from open-ended questions was grouped according to thematic content and frequencies of themes were calculated ( Table 3) . The most common comments on course strengths included flexibility of the daily activities, course organization, individually tailored experiences, and high educational value. The most frequent comments for course improvement included requests for inperson clinical activities, development of specialty tracts, and more interaction in the remote classroom.

Ten students (27%) submitted responses to the postelective online survey including 9 who completed all questions and were enrolled in both the 2-and 4-week courses (4 and 5, respectively) and one student who answered 37% of the questions. Evaluation of the students' engagement in the course utilized Pickering's technologically enhanced learning environment survey items and was composed of questions addressing emotional, cognitive, and behavioral engagement. 11 The mean scores out of possible 5 for each category were 3.86 for emotional engagement, reflecting student satisfaction; 3.93 for cognitive engagement, reflecting goal setting and planning; and 3.75 for behavioral engagement, reflecting physical interaction. Course-specific questions assessed the acquisition of medical knowledge, attainment of course-specific objectives and individual learning goals, and identification of deficits in learning or engagement. Student responses revealed deficits in connecting with others in the remote learning environment (mean 2.90) and evaluating and dissecting surgical specimens under supervision (mean 2.22). Standardized end-of-rotation evaluations showed highest scores for "overall organization of the course/clerkship" (mean 4.89) and "overall quality of the course/clerkship" (mean 4.67), and lowest scores in "adequacy of direct observation of your clinical skills" (mean 2.89) and "adequacy of feedback about your performance" (mean 2.89).

End-of-rotation data showed the remote pathology course is among our institution's courses with high evaluation scores (87 out of 156 scored a mean of greater than 4.5 out of possible 5). The elective's score was 4.88 in comparison to all advanced clinical clerkships (n ¼ 156 courses, mean 4.51, range 2.63-5.00) and all elective rotations (n ¼ 50 courses, mean 4.41, range 2.63-5.00), and the traditional in-person pathology elective (4.73). Core strengths attributing to students' success in the remote learning environment included flexibility of content and schedule, overall organization of course, ability to tailor course to an individual's learning goals, perceived high educational value, and a positive education environment. Deficits included the inability to gross surgical specimens, inadequate observation or feedback about students' skills, and impaired social connections. Students' emotional, cognitive, and behavioral engagement with the course (3.86, 3.93, 3.75, respectively) suggest that there is continued opportunity for curricular improvement, which could include creatively meeting the requests for in-person experiences and developing of themed tracks for career exploration. Additionally, remote sign-out sessions could be improved by guiding the faculty, fellows, and residents on how to provide opportunities for students to develop and practice clinical skills at the remote microscope while also providing feedback for improvement. We established suggestions for best practices for institutions considering transitioning in-person clinical electives to remote delivery; institutions should maximize digital possibilities, incorporate flexibility and adaption, encourage human connection, establish community, and increase access to course leadership (Table 4) . Potential limitations in survey analysis include that students were unable to make direct comparison between remote versus in-person learning for the pathology elective and the outcomes relied on comparison between groups of students that participated in either version of the elective but not both. Only a small sample size of students (10 of 37) completed the anonymous online survey. Possible contributing factors include timing of survey release which corresponded with the start of residency for the graduating students, return to the wards for continuing students, simultaneous stressors due to the pandemic, and survey fatigue during a time of increased online learning. Additionally, the delayed release of the online survey may contribute to recall bias. In order to preserve the anonymity of the participants, the only demographic data collected was the length of their enrollment in the course. While we cannot say definitively that we had a representative sample of the students, the participants provided variation in their responses and represented enrollment in both the 2-and 4-week courses. In our content analysis of open-ended questions, we chose to use frequency of theme occurrence as a marker for salience and acknowledge that this may potentially overlook topics that are not as easily discussed within the context of an end-of-course evaluation survey. 12 The goal of our study was to provide a broad overview of the student experience in the new remote learning setting within the context of program evaluation. While the Qualtrics online survey contained the same standardized end-of-rotation questions as the MedHub evaluation, the Qualtrics online survey responses were used to identify strengths and deficits rather than a composite score for the elective due to the fact that the timing and delivery could not be reliably matched to other end-of-rotation data. The authors used the MedHub standardized end-of-rotation evaluations when comparing course performance in relation to other rotations offered in the School of Medicine because of (a) the consistency in timing of evaluation survey release for more immediate recall of experiences and (b) the similar confounding factors, such as the perception of impact on clerkship grading and the influence of evaluations of the students submitted by residents and preceptors. Interestingly, not all students completed the MedHub end-of-rotation evaluation and, in our cohort, only 27 of the 37 participants submitted anonymous evaluations. This response bias is another reason why it was imperative to only use the MedHub data for comparison between rotations. The halo effect, a type of cognitive bias where positive global evaluation skews judgment toward the positive end of the spectrum, 13 is a potential confounding factor affecting the end-of-course evaluations, stemming from the students' gratitude for the ability to continue their clinical studies remotely, their positive response to the autonomy and highflexibility in the course structure, and from an appreciation for emotional support provided by course leadership during a time of great uncertainty. Future directions include continued use and increased availability and access of the remote learning format as an additional or alternative educational experience to expose medical students to the field of pathology. Mobile technologies allow for opportunistic learning, an advantageous learning strategy that facilitates continued educational gains in moments of downtime. 14 A remote learning elective can offer similar benefits by moving a clinical experience to a platform that is accessible to students who are physically distanced from the hospital or clinic. Possible iterations of this remote elective include: (a) specialty-specific tracks for fourth-year medical students entering specialties that interface with pathology designed with a focus on interprofessional communication, collaboration, and development of a collegial understanding of the scope of pathology practice to enable greater collaboration with future clinicians, (b) virtual electives for visiting students from outside institutions to reduce the financial burden and provide more equitable access to audition rotations, and (c) adaptation of the curriculum to meet the needs of a career exploration elective for third-year medical students. Success in a remote offering of a clinical radiology elective structured for fourth-year medical students during residency interview season 1 also highlights another potential place in the fourth-year curriculum where a remote pathology elective may be beneficial. By preparing and implementing remote options now, the medical education community can position courses to overcome unforeseen barriers to in-person activities in the future including pandemics, natural disasters, medical illness, or parental/family leave, and can expand the pathology pipeline by hosting students who do not have access to a pathology rotation at their home institution.

The overall positive response of students to our remote clinical elective adds supportive evidence that many aspects of anatomic pathology are well-suited to the remote learning environment. 2 For students considering pathology careers, the remote elective can provide additional exposure to the field in a flexible environment and function as a complement to their in-person training. While the remote model may not be sufficient for students pursuing careers in pathology due to the lack of hands-on skill-building for surgical specimen grossing, autopsy, and cytology, it appears to be particularly wellsuited to increasing the understanding of the role of pathology in patent care and fosters interdisciplinary clinical collaboration between pathologists and future clinicians. Table 4 . Tips for Establishing a Remote Pathology Elective.* Maximize digital possibilities: The cornerstone of our remote elective is live-streamed digital sign-out sessions; this can be achieved either with digital slides and screen-sharing or from cameras capturing glass slides on the microscope. Students should be welcomed and encouraged to be active participants. Build in flexibility and adaptation to ensure equity in the virtual space: Normalize that educational interruptions commonly occur when learning remotely including difficulties with internet connectivity and use of video, extenuating life circumstances, and learning differences. Allow multiple avenues for students to fulfill their learning experience. Encourage human connection: Establish an expectation for residents and attending physicians to have video cameras turned on during clinical sign-out to facilitate communication and an interactive learning environment. Build in optional social sessions to substitute for the normal learning that happens in casual conversations during an in-person elective. Establish community: Provide a live orientation session to establish group norms, to give a sense of place and to improve comfort levels in the remote learning environment. A synchronous end-of-course wrap-up session to provide a sense of completion is highly recommended. Increase access to course leadership: Be available to students through multiple channels (eg, office hours, social media, email), especially until the remote learning course is well-established and participants are comfortable with the use of remote technology. Students notice and appreciate the extra concern from course facilitators. * These best practice recommendations are based on our experiences in the UCSF Department of Pathology.

An internet-based radiology course in medical school: comparison of academic performance of students on campus versus those with absenteeism due to residency interviews

Remote anatomic pathology medical student education in Washington state

A systematic review evaluating the impact of online or blended learning vs. face-toface learning of clinical skills in undergraduate nurse education

Online eLearning for undergraduates in health professions: a systematic review of the impact on knowledge, skills, attitudes and satisfaction

Using technology to increase student (and faculty satisfaction with) engagement in medical education

AAMC guidance on medical students' participation in direct patient contact activities

Clinical Laboratory Improvement Amendments (CLIA) laboratory guidance during COVID-19 public health emergency

Leveraging technology for remote learning in the era of COVID-19 and social distancing

Clinical skills-related learning goals of senior medical students after performance feedback

The Power of SMART Goals: Using Goals to Improve Student Learning. Solution Tree

Exploring the dimensions of medical student engagement with technology-enhanced learning resources and assessing the impact on assessment outcomes

Content analysis and thematic analysis: implications for conducting a qualitative descriptive study

The halo effect: evidence for unconscious alteration of judgments

Medical students: what educational resources are they using

The authors thank Patricia O'Sullivan, EdD, Director of the UCSF Research and Development Center for Faculty Educators, for advising on survey design and IRB application and Christy Boscardin, PhD, for comments and guidance. We are grateful to Sarah Bowman for administrative support and to Mary Roach for participating in our book club discussion of her book Stiff: The Curious Lives of Human Cadavers. We also thank the faculty, residents, and staff from the UCSF Department of Pathology and the UCSF School of Medicine for supporting our educational endeavors.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: the article processing fee for this article was funded by an Open Access Award given by the Society of '67, which supports the mission of the Association of Pathology Chairs to produce the next generation of outstanding investigators and educational scholars in the field of pathology. This award helps to promote the publication of highquality original scholarship in Academic Pathology by authors at an early stage of academic development.

Kara S. Tanaka https://orcid.org/0000-0003-0825-186X