key: cord-0749069-z449y3oa
authors: Said, Jordan Taylor; Schwartz, Andrea Wershof
title: Remote Medical Education: Adapting Kern’s Curriculum Design to Tele-teaching
date: 2021-01-25
journal: Med Sci Educ
DOI: 10.1007/s40670-020-01186-7
sha: 132c96ef7dcb27f596f3a7b4eedc883f36067dd8
doc_id: 749069
cord_uid: z449y3oa

COVID-19 has necessitated a rapid shift to the remote delivery of medical education. We present a timely collection of tips, techniques, and strategies for the facilitation of remote teaching sessions and modification of curriculum design, assessment, and evaluation. We step through Kern’s six-step curriculum design, recommending to (1) consider session necessity and a variety of teaching models; (2) inform your session with surveys and polls; (3) keep session endpoints consistent; (4) make the most of technology and translate in-person strategies to virtual forms; (5) engage with individual learners and eliminate distractions; and (6) consider online methods of assessment and evaluation methods.

Medical education has dramatically changed in the era of COVID-19, necessitating remote and virtual teaching for medical students and post-graduate trainees [1] . The traditional medical teaching settings [2, 3] have been upended by medical students being pulled from clerkships and classrooms due to the pandemic. In response, teaching sessions that are conducted virtually at a distance from the learners-or remote medical education (RME)-have exploded across medical education as a method to engage medical students and residents while physically away from their medical schools, clinics, and hospitals. Despite the recent expansion of RME, "e-Learning" and other virtual modalities are significantly under-examined [4] [5] [6] [7] , and a significant lack of medical education-specific resources exists for educators who are now teaching in these structures.

We present a collection of techniques and strategies for both the facilitation of online teaching sessions and the accompanying appropriate modification of curriculum design, assessment, and evaluation, based on Kern's model of curriculum design for medical education [8] . RME is an essentially required teaching modality during the COVID-19 pandemic, with little time for medical educators to adapt curriculum. Especially now, it is critically important that all medical educators-both classroom facilitators and clinical teachers-have a toolkit of effective teaching skills to educate and engage their learners, beyond the challenges posed by unexpected shifts to RME.

The wide adoption of remote medical education in the setting of the COVID-19 pandemic presents unique challenges for medical educators, including the need to rapidly develop mastery over novel teaching technologies that have become increasingly essential to medical education over the past two decades [9] [10] [11] . While some medical teaching technologies used for in-person sessions-such as the pre-session concept videos in case-based collaborative learning [12] , spaced education [13] , and "Tweetorials" [14] -can translate easily to RME, interactive large-group video conferencing is likely new to many medical educators.

The current generation of medical students and trainees have been described as "digital natives," predominantly young adults who were born into a technologically oriented society that afforded them both technological fluency and the ability to pick up and master new technologies with ease [10, 15] . For example, a five-year study at Wayne State University School of Medicine demonstrated an overwhelmingly positive student response to video streaming modalities in their curriculum [7] . In the clinical arena, students have responded positively to "tele-presenting" on rounds and the use of virtual reality [16] [17] [18] . Some educators may not embrace novel teaching technologies as enthusiastically as digital natives. The challenge thus falls not on learners but rather on educators, as we face how well we can adapt our teaching styles to facilitating teaching sessions over video conferences and other virtual modalities from afar.

Rather than focus on the new technologies themselves, we recommend medical educators keep the focus of curriculum design and teaching session planning on the pedagogy, rather than the program or platform [19] . Educators adapting to online teaching can follow the map of Kern's curriculum design framework, a six-step model for critical development of a curriculum that begins with needs assessment and ends with curriculum evaluation [20] . Here we consider how to adapt each step of curriculum design to fit RME delivered by large-group video conferencing ("tele-teaching"), with a specific focus on teaching session facilitation techniques and strategies per Kern's non-linear, interconnected stepwise model.

Step 1. General Needs Assessment: Consider Session Necessity and a Variety of Teaching Models The first step of Kern's curriculum design model is the general needs assessment, in which we identify a problem for the teaching session to address, the current teaching approach, and our ideal teaching approach, which likely remain the same whether taught in person or online.

The first consideration for developing an RME session is to consider whether the session needs to be taught online at all; is it immediately necessary for learners? Should the session be postponed, cancelled, or mitigated with an online teleteaching session? These decisions should be made on the basis of their value for learners and feasibility of translating inperson sessions to tele-teaching equivalents: for example, a clinical didactic session on physical exam maneuvers may translate well to a tele-teaching session that includes audio and video exam findings and interpretation, whereas an observed inpatient history-and-physical might be replaced by an observed tele-visit or a simulated online case. If a decision is made to translate an in-person session to RME, we recommend considering modifications to both the session structure and teaching model. During the current pandemic, learners are largely engaging with educational material virtually over tele-teaching, with up to several hours daily spent in large-group video conferences and meetings with instructors. Educators can consider reducing the duration of their inperson teaching sessions if translated to RME, to avoid digital eye strain ("screen fatigue") and loss of student attention [21] .

Toward the same goal, we recommend considering transitioning traditional lecture-style sessions to some version of flipped classroom learning, which has been demonstrated in multiple forms to improve student engagement in science and medical education [22] [23] [24] [25] . For example, a combination of inperson lecture followed by a small-group flipped classroom session (i.e., problem-based learning [PBL] or patient/lab demonstration) can be converted to a virtual modified case-based collaborative learning (CBCL) session, in which students learn material and complete a readiness assessment exercise in advance of a tele-teaching teaching session that introduces new clinical vignettes and cases [12] . Replacing time formerly devoted to lectures with interactive tele-teaching prioritizes student engagement with the instructors and material, which in turn fosters student attention and engagement even from afar.

In Kern's second step, the targeted needs assessment, the instructor must identify what is known and unknown to the learners with as much specificity as possible. This can occur either prior to the learning session to direct session planning or at the start of a session to tailor premade material to students' particular knowledge gaps and learning needs.

Prior to a session, consider sending out a survey to assess student baseline knowledge on the teaching content [12] . This survey can be similar or identical to a readiness assessment exercise in CBCL, using whichever digital platforms educators are most comfortable with [12] . A judicious review of the learners' responses to this pre-session survey will allow a medical educator to appropriately tailor the level of the session's depth and breadth. This targeted approach allows a medical educator to aim the session toward the learner's zone of proximal development (ZPD), indicating the boundary between a learner's established fund of knowledge and the novel content they are capable of learning and understanding with assistance [26, 27] . Asking students to identify one or two new concepts that they learned from the session can also target students toward their newly acquired knowledge at the ZPD.

At the beginning of a session, the results of a pre-session survey can be reviewed with the entire group of learners over tele-teaching to establish the general group level of content knowledge and familiarity. During the session, audience response systems can be used to poll medical learners, including phone applications, websites, or embedded features of teleteaching platforms [28] , serving to engage students and deliver feedback to the teacher in real time [29] . Beyond session planning, polls can be also used to gather students' attention with an interactive case-based question, assess group understanding of a basic science concept before moving on to clinical application, and initiate a large group discussion on the content of the poll.

The third step in Kern's development is the creation of both broad goals and specific teaching objectives. These goals should not vary significantly from learning objectives set for an equivalent in-person teaching setting, unless the learning objectives require in-person attendance (i.e., "Students will demonstrate a comprehensive cardiovascular exam"). Just as with in-person medical education, we recommend creating objectives that focus the learner and can be revisited at the end of a session to summarize content. We recommend the use of verbs from the revised Bloom's taxonomy (remember, understand, apply, analyze, evaluate, create) to link learning objectives to hierarchical cognitive domains [30] . This approach can be used to create "SMART" learning objectives: specific, measurable, attainable, relevant, and time-based [31] . Of note, goals built in this framework have recently been shown to increase selfdirected educational actions taken by learners, which could improve individualized learning in RME [32] .

The fourth step of Kern's curriculum focuses on educational strategies, which includes specific tele-teaching techniques. Many experienced medical educators already have a personal trove of well-developed teaching strategies; however, we must now consider how to adapt our methods to tele-teaching, translating successful in-person teaching strategies to the RME setting. We provide a summary of the below discussed strategies in Table 1 .

For example, many medical educators have favorite strategies for balancing lecturing and small group discussions in a single flipped classroom teaching session. A common inperson medium-to-large group teaching strategy is to pose a question to the entire group and give learners a few minutes to discuss in partners or small groups first. To adapt this for teleteaching, virtual "breakout rooms" can be utilized; this technique allows students to be sorted manually or randomly into groups of a chosen number, and the students then discuss in these smaller breakout rooms before the facilitator reunites the entire large group. Utilizing this technique is seamless since the video conference quickly toggles between the large group and breakout room, ideal for modeling some flipped classroom models such as CBCL [12] . Other teaching strategies compatible with breakout rooms include the "jigsaw technique"-in which students rotate through mixed groups to work on small problems that the class collates into a final outcome-and twoperson "think-pair-share" method [33, 34] .

Maintaining student engagement is of the utmost importance in RME, as we discuss in the fifth step of Kern's model. Visual engagement can be very effective [35, 36] ; for example, visual learning tools such as slide decks and animated presentations have been shown to be effective for keeping medical students attentive and engaged during in-person teaching sessions [37] . Tele-teaching software designates a space for sharing a set of teaching slides, whiteboard, or shared desktop screen that can be displayed on all students' screens.

Students in large-group video conferences may be less likely to volunteer discussion in class, due to unfamiliarity or perceived awkwardness with speaking up in the teleteaching format. We recommend mitigating this issue by engaging with students with questions that invite students to start a discussion, rather than to simply answer a question.

We recommend using open-ended questions, so students feel less intimidated about getting a question with a clearly correct answer wrong in front of a virtual audience. Just as with inperson sessions, we recommend giving learners an appropriate amount of time to think about questions before proceeding or make use of the chat box to share ideas.

The tele-teaching format can provide a novel solution to this classic issue of classroom silence or poor student engagement with audience questioning. One classical technique to engage quiet learners would be to "cold-call" students virtually in the large-group video conference. However, in addition to intimidating chosen students and potentially introducing bias in selecting students, cold-calling students through tele-teaching can also be hampered by technological issues with an individual microphone or Internet connection, and thus is a less reliable method.

We recommend the "warm call," which utilizes the aforementioned featuring of privately messaging students in the tele-teaching session to inform them that they will answer an upcoming question or be called upon soon. This strategy is founded upon the Yerkes Dodson Law, which posits that students achieve maximal engagement and performance with an appropriate balance of low-stress (i.e., anonymous polling) and high-stress (i.e., answering in front of a classroom) settings, with poor student performance expected in the exclusive setting of either extreme [38] . Alerting the student in advance to give them time to prepare a response provides a low-stress warning, while the act of answering a question in front of the entire student audience creates a high arousal state. This maximizes student engagement, which prior work has linked to higher level thinking and improved learning outcomes amongst medical students [39] [40] [41] .

The fifth stage of Kern's model encompasses actual implementation of the curriculum. For tele-teaching, we consider methods to optimize logistical setup of tele-teaching, achieve student investment in RME, and remember the importance of accessible medical education even in virtual spaces.

For logistical setup, we recommend encouraging students to mute their audio output when they are not speaking. This helps to eliminate noise from unexpected interruptions as well as white noise from individual participants. In order to keep students attentive and connected to the facilitator, we discourage instructors from disabling their video output on their web cameras. Finally, the chat tools allow students to emulate inclass behaviors, such as hand-raising, to notify instructors that they have a question or answer to volunteer to the large group. Some platforms allow students to submit "thumbs up" or "thumbs down," emulating the in-person strategy of asking a straw poll to an entire room of learners.

Students arriving onto the tele-teaching platform shared space presents the first opportunity to welcome students and-as we do with patients-"invest in the beginning" [42] . Distribute a password for entering the virtual space in advance, to create a secure space for students. We recommend early preparation for facilitating sessions: aim to create and arrive to the shared video conference call at least a few minutes early to greet participants over video or the chat box as they arrive. Employ the dashboard view that shows all participants' video feeds at the beginning of class to show the virtual community of the classroom. If possible, devote a few minutes prior to a tele-teaching session to troubleshooting any technology issues to ensure the session runs smoothly.

Perhaps one of the largest areas of apprehension surrounding RME and tele-teaching is the propensity for distraction or low student engagement amongst remote learners. Since students have the ability to mute their audio and disable their web cameras at will, it may appear seamless for students to engage minimally in tele-teaching or multitask on other projects during class. To circumvent this issue, we propose three solutions. First, we recommend including a discussion of online etiquette ("netiquette") at the start of a teaching session, including attention and minimizing any individual distractions. Second, a variety of teaching methods can and should be converted to digital formats (Table 1) . Third, we importantly recommend that instructors increase the frequency of session breaks. Even remotely, students need to stretch and take time away from the computer screen; frequent session breaks, even if brief, are thus an investment in student attention and engagement.

Finally, we recommend that RME facilitators consider the accessibility of their tele-teaching. When converting sessions to near-equivalent tele-teaching sessions, it is essential that educational materials and opportunities to learn remain equally accessible to learners with disabilities. To review, the Americans with Disabilities Act of 1990 requires medical schools to provide "reasonable accommodation or modification" to physically disabled learners such that they are equally able to engage with the curriculum; this remains true across virtually delivered sessions [43] . In this vein, some learners with physical disabilities may benefit from or require modifications to engage fully and equally with tele-teaching. If a Instructor can activate dashboard view in order to see all participants' video feeds on 1 screen, emulating the feel of speaking to and interacting with an entire class of students. Instructors should look into the camera directly when possible, rather than at the screen, so participants experience eye contact.

Chat box, raise hand tool, mute tool Learners can ask questions by typing in the shared chat box; they can also use chat box tools such as "raise hand" to indicate when they have a question. The "mute tool" should be used whenever the student is not actively participating to reduce white noise and unexpected interruptions. Handouts QR codes, links Instructor can provide digital versions of physical handouts, easily accessible by link to file-sharing or online host or by photographic QR code.

Polling tools Instructor asks a closed-ended question with discrete answer choices; using smart-phone applications or native video conferencing software (i.e., chat tools), students can all quickly vote. 

Instructor asks an open-ended question to the entire class, then sets the sessions to breakout rooms to automatically move students into smaller conference calls groups. Instructor can "visit" each of these small groups, and then reconvene the entire class.

QR, quick response session is recorded, we recommend reading every word and describing every picture on featured slides if some learners are anticipated to have challenges viewing or accessing the visual component of video conferencing. For students with hearing impairment, some institutions or platforms can provide closed captioning services for recorded sessions, as well as amplification devices. To share digital handouts, some tele-teaching platforms allow for direct file-sharing; otherwise, the facilitator can share a link to a web-hosted file or generate a QR code that participants can scan with their phone cameras.

Step 6. Assessment, Evaluation, and Feedback:

Finally, we briefly consider appropriate changes to the assessment of the learner; we then conclude with recommendations for the evaluation of tele-teaching sessions, using a classical model of evaluation and with practical considerations for feedback delivery.

The assessment of student knowledge and ability is necessary to consider in the delivery of a "tele-teaching" curriculum. A variety of formative and evaluative assessment tools are commonly used in undergraduate medical education, many of which can be translated to virtual equivalents. For in-session formative assessment, for example, instructors can use polling functionality integrated into teleconferencing software to capture student understanding or ask quick questions during a session. This modality can replace short in-class quizzes or readiness exercises accompanying some flipped classroom sessions [12, 44] . Proctoring virtual exams presents a variety of significant challenges for educators, including ensuring exam security and fairness and creating virtual assessment methods that compare to their in-person equivalents for clinical skills. For more traditional exams, such as preclinical end-of-course exams or standardized exams, remote proctoring has been utilized during the ongoing pandemic to ensure security and fairness during remote assessments. Outside of medical education, remote proctoring has been feasibly implemented for the Graduate Record Examinations (GRE) General Test, Graduate Management Admissions Test (GMAT), and Law School Admissions Test (LSAT) with good effect [45] . Medical curriculum developers and course directors can also utilize remote proctors for their formerly in-person exams; however, it is important to remember that parallel strategies to ensure exam security are also important, such as maintaining a strict exam availability window, allowing once-only access to the exam, randomizing question order, and/or administering tests using a "locked down" browser [46] . Finally, early evidence suggests that despite their feasibility, "e-exams" may be associated with increased student stress amongst medical undergraduate students, particularly related to technical problems, Internet connectivity challenges, and concerns over peer dishonesty [47] . For the assessment of clinical skills, the classic "objective structured clinical examination" (OSCE) can be converted to a virtual "tele-OSCE"; Examples of this are discussed later in this section.

The Kirkpatrick model of evaluation provides four sequential steps that link evaluation of a session to its ultimate impact on learner behavior and outcomes [23, 48, 49] . The first level, the response, refers to the session evaluation. Simple chat tools, such as the "thumbs up" and "thumbs down" tools, as well as polling tools, can quickly assess if learners were satisfied with a session at its end. Session and course evaluations should be converted to digital formats in RME. The second level of Kirkpatrick's evaluation model concerns learner knowledge or skill after the session, which can be assessed by postsession surveys or questions or course quizzes and exams if the teaching session is embedded in a larger curriculum.

The third level is behavior change; in medical education, this is often assessed as a longitudinal endpoint, such as looking at OSCE scores or by grading of video-recorded standardized patient (SP) interactions. Institutions can consider how to facilitate digital equivalents of these sessions, such as "tele-OSCEs," which have been used in medical school curricula to simulate video patient encounters prior to the pandemic. In one reported example, a "tele-OSCE" experience was designed and implemented to assess learners' competencies caring for a patient with depression and demonstrate the utility of telemedicine, while still allow for direct faculty observation of student-standardized patient encounters [50] . Similar novel "tele-OSCEs" can and indeed have be created in the COVID-19 era, demonstrating the feasibility of converting formerly live OSCEs to equivalent experiences delivered over teleconferencing software [51, 52] .

The final degree of Kirkpatrick's model of evaluation is results, in medical education related to patient impact; this final stage is concerned with how the curriculum affects the learners' clinical outcomes. This is challenging to demonstrate for single sessions, but can be theoretically demonstrated by chart review and clinical outcomes for clinical didactic topics. Considering the content of feedback, surveys evaluating sessions should include questions that assess the tele-teaching format and use of the platform technologies; ideas for improving RME sessions should be collected. Finally, in-person feedback sessions can be converted to video meetings, in order to maintain one-on-one time for individual learners with course facilitators.

We provide a summary of our recommendations organized along the six steps of Kern's curriculum design in Table 2 .

The COVID-19 pandemic has quickly transformed our model for delivering medical education in an unprecedented way and highlighted the urgent need for training in tele-teaching techniques for medical educators. From the classroom to the clinic, medical educators must adapt to dynamic and challenging circumstances by quickly mastering the practice of remote medical education. We have summarized here a collection of practical strategies for using video conferencing tele-teaching, applicable to a wide array of settings, following the framework of Kern's curriculum design. We hope that these recommendations can instill confidence and improve the virtual classroom, allowing for the broad delivery of high-quality remote medical education to all learners affected by COVID-19 and beyond.

Conflict of Interest The authors declare that there is no conflict of interest.

Disclaimer Some of this material is the result of work supported with resources and the use of facilities at the VA Boston and New England GRECC. The contents do not represent the views of VA or the United States Government.

Medical student education in the time of COVID-19

Optimal clinical setting, tutors, and learning opportunities in medical education: a content analysis

Basics in medical education

Underst Med Educ Evidence, Theory Pract. Second Ed

The impact of e-learning in medical education

Leveraging elearning in medical education

The effectiveness of streaming video on medical student learning: a case study

Overview: a six-step approach to curriculum development. Curriculum Development for Medical Education: A Six-Step Approach

Educational technology in medical education

Using technology to meet the challenges of medical education

This is not your grandfather's medical school: novel tools to enhance medical education

Assessing the effectiveness of case-based collaborative learning via randomized controlled trial

Adaptive spaced education improves learning efficiency: a randomized controlled trial

Why is a cow? Curiosity, tweetorials, and the return to why

Problem identification and general needs assessment Consider session necessity in short-and long-term. Consider session length. Consider teaching model

Targeted needs assessment Inform session content and format with pre-session survey. Use in-session polls to assess learner knowns and unknowns in real time

Goals and objectives Keep educational targets consistent with equivalent in-person session. Create SMART goals driven by Bloom's taxonomy

Educational strategies Substitute breakout rooms for small groups. Employ shared-screen visuals (slides, whiteboard)

Implementation Establish a group "netiquette" for tele-teaching sessions. Greet entering students as they arrive to virtual space. Devote time to technology troubleshooting. Consider session accessibility such as closed captioning

Assessment, evaluation, and feedback Use chat box and polling tools to quickly elicit end-of-session feedback. Convert physical evaluations to digital formats. Include assessment of tele-teaching platform in session evaluations

SMART, specific, measurable, attainable, relevant, and time-based

Digital natives, digital immigrants part 1

Tele-attending can emulate and even improve bedside teaching and learning

Cardiopulmonary resuscitation training by avatars: a qualitative study of medical students' experiences using a multiplayer virtual world

Virtual reality lead extraction as a method for training new physicians: a pilot study

Teaching Remotely -Best Practices for Online Pedagogy

Curriculum development for medical education

Digital eye strain: prevalence, measurement and amelioration

The flipped classroom: a modality for mixed asynchronous and synchronous learning in a residency program

Flipping the classroom in graduate medical education: a systematic review

Flipped classroom improves student learning in health professions education: a meta-analysis

Active learning increases student performance in science, engineering, and mathematics

Scaffolding student learning in clinical practice

How educational theory can inform the training and practice of plastic surgeons

Tips for using mobile audience response systems in medical education

Audience response systems in medical student education benefit learners and presenters

How to write well-defined learning objectives

Bloom's Taxonomy of Educational Objectives

A randomized trial of SMART goal enhanced debriefing after simulation to promote educational actions

Cooperative learning through jigsaw classroom technique for designing cast partial dentures -a comparative study

Measuring the impact of the flipped anatomy classroom: the importance of categorizing an assessment by Bloom's taxonomy

Learning styles and approaches to learning among medical undergraduates and postgraduates

Learning styles and approaches: implications for medical education

Does a presentation's medium affect its message? PowerPoint, Prezi, and oral presentations

Yerkes-Dodson: a law for all seasons

Student engagement: three models for its investigation

Handbook of research on student engagement

Assessing student engagement and self-regulated learning in a medical gross anatomy course

Getting the most out of the clinical encounter: the four habits model

Without handicap: issues of medical schools and physically disabled students

The case-based collaborative learning peer observation worksheet and compendium: an evaluation tool for flipped classroom facilitators

Never let a crisis go to waste: large-scale assessment and the response to COVID-19

Internet-based proctored assessment: security and fairness issues

Stress and behavioral changes with remote Eexams during the Covid-19 pandemic: a cross-sectional study among undergraduates of medical sciences

The Kirkpatrick model: a useful tool for evaluating training outcomes

Developing scholarly projects in education: a primer for medical teachers

Insomnia Telemedicine OSCE (TeleOSCE): a simulated standardized patient video-visit case for clerkship students

Remote assessment of clinical skills during COVID-19: a virtual, high-stakes, summative pediatric objective structured clinical examination

A Telehealth clinical skills education adaptation

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations