key: cord-1054907-alkpx8p9 authors: Newman, Jessica R.; Fink, Jennifer; Clough, Lisa A.; Johnston, Shane title: Internal Medicine Clerkship ID Curriculum Flip: Will They Prefer to Pre-learn? date: 2021-09-01 journal: Med Sci Educ DOI: 10.1007/s40670-021-01384-x sha: e3c9ac7592012af870a94e8299fe622d06488df1 doc_id: 1054907 cord_uid: alkpx8p9 Varying pedagogical approaches of undergraduate medical education are utilized in clerkships to supplement bedside teaching. The flipped classroom mode, in which self-paced study precedes the in-person session, is often used in pre-clinical education. This shift allows time with the instructor to focus on guided application of pre-learned concepts. At our institution, the Internal Medicine Clerkship Infectious Diseases lecture was substituted to a flipped classroom with two pre-learning videos. Student satisfaction scores were higher for the flipped classroom and comments were more negative for the traditional lecture. This suggests that senior medical students favor flipped classroom pedagogy despite pre-learning requirements. Numerous modes of instruction supplement clinical teaching during required clerkships including traditional lecture and learner-center models such as flipped classroom (FC), problem-based learning (PBL), case-based collaborative learning (CBCL), and simulation. Use of video-based lecture can also effectively supplement in-person instruction [1] . The FC format has been increasingly applied to pre-clinical instruction in undergraduate medical education (UME). The general structure of FC is such that the student is assigned material to review prior to face-to-face teaching and in-class time is used to apply knowledge, often in the form of cases [2] [3] [4] . Content provided for pre-review can include text-based resources, videos, or games, though video-based resources are utilized most frequently by learners [5] . The FC model has shown learning benefits in health professions education. For example, graduate students in physiology scored higher on assessments after implementation of a FC compared with lecture [4] . Surgery and radiology clerkship students had high satisfaction, effective knowledge acquisition, and non-inferior NBME performance using FC [6] [7] [8] . Flipped learning was also associated with increased academic achievement and greater task value in radiology clerkship when compared to lecture [9] . Though utilization of FC in medical education is fairly novel, it is well-received amongst students [10] and shows promise in increasing motivation and engagement [11] . The University of Kansas School of Medicine (KUSOM) recently underwent a curriculum shift to active, competencybased learning. In years 1-2, there was a substantial reduction in lecture and corresponding increase in CBCL and FC. The third-year of instruction at KUSOM includes a series of required core clerkships including Internal Medicine (IM). On the Kansas City campus, the KUSOM IM Clerkship has approximately 22-24 enrolled students rotating for each of their assigned 8-week blocks. Clerkship classroom instruction primarily takes place week 1 of the clerkship, before entry into clinical services. Didactic content prior to the intervention incorporated a mixture of lecture, PBL, and skills-based instruction. Given the time required for pre-learning, it was unclear how FC would be perceived by third-year medical students. We sought to assess if a transition from an Infectious Diseases (ID) lecture to FC with required pre-learning would be well-accepted amongst students and/or enhance selfperceived learning. This project built upon previous research by assessing differences in satisfaction using pre-post comparisons of Likert-type questions and sentiment analysis of evaluation comments. Compliance with the flip pre-learning was also assessed via self-report. In the 2018-2019 academic year (AY), the IM clerkship included a 2-h ID lecture, covering topics of endocarditis and antibiotic use. For 2019-2020 AY, content topics were maintained, though were reorganized into two 20-min pre-learning Panopto™ videos and a 1-h live FC which reviewed an endocarditis case in detail with shorter antibiotic application cases. Case required consideration of history of present illness, system review, social and exposure history, and diagnostic data threaded with interactive questions. To maintain continuity, the same two ID faculty delivered pre-learning and FC sessions in year 2, though several additional faculty contributed to lectures in year 1. Subjects included third-year students in the required IM clerkship in 2018-2019 (control) and 2019-2020 (intervention) AY. Cohort sizes were 168 and 170 learners, respectively. During AY 2019-2020, the final block was virtual considering the COVID-19 pandemic. Student satisfaction across years was evaluated by comparison of an identical six-question satisfaction survey. In 2018-2019, this survey was administered to each student twice (once for each of the two lecture hours of (1) endocarditis and (2) antibiotic therapy). In 2019-2020, students only completed this survey once following the 1-h flip. Satisfaction surveys were administered immediately following the flip. In addition to survey questions that were repeated between years, a new survey instrument was utilized in 2019-2020 AY. The first of six questions assessed completion of pre-learning. The remaining five items used a 5-point Likert scale ranging from "strongly disagree" to "strongly agree" to assess satisfaction, self-perceived learning, and comfort in knowledge in topics covered. Both the satisfaction survey across years and new satisfaction survey tool welcomed comments. Participation in the study was voluntary and consent was implied by return of the anonymous survey. An ethical review was conducted, and the study was exempted from formal informed consent by the Human Subjects Committee of the University of Kansas Medical Center. A primary goal of the project was to compare levels of satisfaction with FC instruction during the clerkship in 2019-2020 compared to traditional lecture used in 2018-2019 AY. For the student satisfaction survey across years, differences in satisfaction were measured on the scale "yes," "neutral," and "no." At α = 0.05, Wilcoxon-Mann-Whitney tests showed no evidence of differences in ratings for any question between the first and second survey in 2018-2019; therefore, subsequent tests utilized pooled ratings. Sentiment analysis used unpaired t-tests to compare differences in satisfaction between years via comment analysis. The technique classifies text sentiment on a numeric-continuous scale by matching individual words to those found in previously validated, positive and negative world lists [12] [13] [14] . Prior applications include satisfaction in online discussion boards, evaluating teachers, and comparing tweets about health care [15] [16] [17] . The R package "sentimentr" was used to analyze comments (N = 147). The program uses a modified "bag of words" approach, because amplifiers (i.e., "very") and negation words ("not") help account for word grouping and ordering [18] . In total, 364 of 676 surveys were collected, yielding an overall response rate of 54% (this was 43% response rate for 2018-2019 AY and 64% response rate for 2019-2020 AY). In evaluation of student satisfaction surveys across years, ratings were higher in AY 2019-2020 compared to 2018-2019. Students in the FC group were more likely to report liking the format (t = 6.30, p < 0.001, effect size = 0.71), acquiring new information (t = 6.21, p < 0.001, effect size = 0.70), and recommending the session to their colleagues (t = 8.70, p < 0.001, effect size = 0.99) (Fig. 1) . For the AY 2019-2020 (new) survey (Table 1) , students responded more negatively in rotation 6 (this session was conducted live virtually given the pandemic). Students overall reported being satisfied with the FC format (Q3, 90% agree or strongly agree), believing the format extended their knowledge (Q5, 89% agree or strongly agree), and supporting expanded use of FC (Q4, 82% agree or strongly agree). Students specifically expressed preference for the FC in response to a forced comparison with lecture (Q2, p < 0.01, effect size = 1.6). Though ratings were lower in rotation 6, these responses did not change the statistical significance associated with any year-to-year differences in item ratings. Using sentiment analysis, student comments were consistent with survey responses and more negative for AY 2018-2019 as compared to AY 2019-2020 (t = −4.83, p < 0.01, effect size = 0.81). Both the satisfaction survey across years and new satisfaction survey included options for comments (Fig. 2) . There are several limitations that should be acknowledged. First, we had a narrow scope and small sample size. In the 2018-2019 AY, several additional ID faculty conducted a lecture session, so while the same slide deck was utilized, this may have resulted in differences in satisfaction based on faculty personality or delivery. Also, students in AY 2019-2020 had been exposed to FC in the updated KUSOM pre-clinical curriculum and it is unclear if the pedagogy would have still been wellaccepted amongst learners with no previous exposure to FC. In summary, students in our curriculum now experience a variety of learner-centered instructional methods including FC during pre-clinical studies. This has primed learners for acceptance of radical shifts of clinical didactic approaches, such as FC, simulation, and CBCL. Similar to prior studies, students surveyed appreciated the FC [11] . The same faculty conducted much of the teaching year 1 and year 2 and a considerable amount of content remained identical (shifting from in-class to pre-learning). We believe this indicates it was the FC delivery that was favored by the students specifically which is reinforced by their more positive comments AY 2019-2020. In assessing responses for the new survey tool for AY 2019-2020, the FC was rated as preferable to lecture. Students also felt that the FC extended their knowledge of the topics. At the Kansas City campus, classroom instruction primarily takes place prior to assignment to clinical services. This schedule ensures that students have no clinical duties competing for time to dedicate to pre-learning. In our model, it would be feasible to transition additional didactics to FC with required pre-learning. Available pre-learning time is often assumed in research models; however, it is not always protected time and is paramount to success of a flip. In this study, a majority of students reported completing most/all pre-learning prior to the session (83%). Only 11% and 6% reported completing "some" or "none" of the material, respectively. This is notably higher compliance than observed in one emergency medicine clerkship example where only 31% of learners completed pre-learning [19] . The FC model is being increasingly introduced into GME [20] . However, effectiveness of FC in a curriculum designed for student and resident learners who are concurrently providing clinical care may be limited based on preparation time needed [21] . Exploration of the utilization of alternate scheduling platforms such as the pre-clinical clerkship didactic week or residency non-call weeks could create dedicated time for pre-learning [22] . More research is needed to determine if FC is effective for teaching in situations when formal didactics are integrated with time dedicated to providing patient care. A randomized trial comparing digital and live lecture formats The flipped classroom for medical students Medical education reimagined: a call to action Flipped classroom model improves graduate student performance in cardiovascular, respiratory and renal physiology An innovative blended preclinical curriculum in clinical epidemiology and biostatistics: impact on student satisfaction and performance Effectiveness of the surgery core clerkship flipped classroom: a prospective cohort trial Student perceptions of a simulation-based flipped classroom for the surgery clerkship: a mixed-methods study Implementation of a flipped classroom approach to promote active learning in the third-year surgery clerkship Flipping radiology education right side up Advances in medical education and practice: student perceptions of the flipped classroom A systematic review of the effectiveness of flipped classrooms in medical education Mining opinion features in customer reviews A new ANEW: evaluation of a word list for sentiment analysis in microblogs The general inquirer: a computer system for content analysis and retrieval based on the sentence as a unit of information Sentiment analysis of health care tweets: review of the methods used Sentiment analysis in MOOC discussion forums: what does it tell us? Sentiment analysis to evaluate teaching performance Sentimentr: calculate text polarity sentiment version 2.7.1 Does the concept of the flipped classroom extend to the emergency medicine clinical clerkship? Flipped classrooms in graduate medical education: a national survey of residency program directors A flipped classroom in graduate medical education Flipping out: does the flipped classroom learning model work for GME? Jessica R. Newman, Jennifer Fink, Lisa A. Clough, and Shane Johnston contributed equally to this work.