key: cord-0010802-6d0u79f1 authors: Jackson, Jennifer M.; Skelton, Joseph A.; Peters, Timothy R. title: Medical Students’ Clinical Reasoning During a Simulated Viral Pandemic: Evidence of Cognitive Integration and Insights on Novices’ Approach to Diagnostic Reasoning date: 2020-05-06 journal: Med Sci Educ DOI: 10.1007/s40670-020-00946-9 sha: 57b57a4fe1e6702ccc018cdb7d31858813a585c9 doc_id: 10802 cord_uid: 6d0u79f1 INTRODUCTION: Cognitive integration from multiple disciplines is essential to clinical problem-solving. Because it is not directly observable, demonstrating evidence of learners’ cognitive integration remains a challenge. In addition, little is known about preclinical medical students’ approach to diagnostic reasoning despite widespread implementation of clinical reasoning curricula for these early learners. The objectives of this study were to characterize how first-year medical students integrated knowledge to problem-solve during a simulated viral pandemic and to characterize students’ diagnostic reasoning approach to this clinical scenario. MATERIALS AND METHODS: Student teams analyzed clinical data to formulate hypotheses for the pandemic’s source and submitted reports justifying their hypotheses and treatment recommendations. A content analysis on students’ reports identified codes and themes characterizing the learning content integrated and students’ approaches to diagnostic reasoning tasks. RESULTS: Sixteen problem-solving codes were identified, demonstrating integration of new and previously encountered content from multiple disciplines. A compare-contrast analytical approach was the most commonly employed diagnostic reasoning approach (100%), with a smaller subset of teams also using a causal approach (20%). DISCUSSION: Content analysis of preclinical students’ diagnostic justification tasks provided insights into their approach to diagnostic reasoning, which was most consistent with the search-inference framework rather than a causal approach, likely due to limited pathophysiological knowledge at that point in training. CONCLUSIONS: Evidence of cognitive integration can be made explicit through learners’ narrative justification of diagnostic reasoning tasks. Preclinical students’ diagnostic reasoning development has implications for curricular design and implementation for this learner group. To make appropriate diagnostic and management decisions, physicians must cognitively integrate and apply knowledge and skills from multiple disciplines. Cognitive integration is defined as "conceptual, cognitive connections between different types of knowledge," occurring within the mind of the learner [1] . Previous studies suggest novices' cognitive integration of basic science and clinical concepts leads to improved diagnostic performance in this early stage of training [2] [3] [4] [5] [6] [7] [8] [9] [10] . Because cognitive integration is not directly observable, demonstrating evidence of successful cognitive integration remains a challenge. The diagnostic reasoning processes of experts versus novices have been extensively studied, and characteristic patterns of diagnostic task performance have been described that correlate with level of expertise. Previous studies have suggested that novices primarily utilize a pathophysiologic analytical (i.e., causal) approach to diagnostic reasoning and rely less on the compare-contrast analytical approach or on pattern recognition, both of which are thought to develop only after trainees begin building robust illness scripts based on clinical experience [11] [12] [13] [14] [15] . However, the novices in published studies examining diagnostic reasoning approaches have been limited to intermediate (third year or fourth year) medical students, resident physicians, or nonmedical health professions students (e.g., massage therapy students, undergraduate kinesiology students, physical therapy students) [5, 8, [16] [17] [18] [19] [20] ; little is known about the diagnostic reasoning approaches of early, preclinical medical students. Published studies involving preclinical medical students consist of intervention studies reporting results of knowledge-based testing or diagnostic accuracy rather than characterization of how these students formulate their differential or working diagnoses [9, [21] [22] [23] [24] . Since clinical reasoning curricula now extend into the preclinical years for many medical schools, understanding these learners' approach to diagnostic reasoning from the beginning of their development is important for both curriculum development and learner assessment in these cognitive skills. Simulation is an instructional delivery method thought to promote cognitive integration and diagnostic reasoning by providing learners with opportunities to apply basic science knowledge in relevant clinical contexts, including opportunities to practice applying their causal knowledge-that is, using basic science concepts to explain clinical manifestations of disease [12, [25] [26] [27] [28] . With this in mind, we developed a collaborative, simulation-based learning activity for our firstyear medical students requiring them to integrate knowledge from multiple disciplines in order to problem-solve during a simulated viral pandemic. In this study, we analyzed students' reports from this learning activity in order to (1) characterize how early preclinical students integrate new and existing knowledge as reflected in their written diagnostic justification tasks and (2) characterize early preclinical students' diagnostic reasoning approaches to this case. A secondary objective of this study was to identify the resources these students used to approach their self-directed learning during this activity. The subjects in this study were 140 first-year medical students who participated in a 2-hour simulated pandemic activity in January 2018. The event was part of students' virology course and occurred following core virology instruction. The virology course occurred approximately 6 months into students' first year of medical school and followed the anatomy, biochemistry, and bacteriology courses. Prior to this event, the students had had pathophysiology instruction on metabolic, bacterial, genetic, fungal, and immunological diseases as well as sepsis. Concurrently throughout the first year, these students also participated in longitudinal clinical skills, epidemiology, and bioethics courses. Prior to this simulation event, these students had learned how to collect a full history in their longitudinal clinical skills course; this included history-taking skills practice in numerous real and simulated patient encounters in each of the 11 clinical skills class sessions leading up to this event. These students had also participated in 24 sessions of a longitudinal, problem-based learning (PBL) course prior to this simulation event. The PBL course is case based and focused on differential diagnosis formulation using a causal approach, guided by a mnemonic ("VINDICATEM-P") to systemically consider diseases in each pathophysiological category. During these sessions, students engaged in self-directed learning as needed when their baseline knowledge was insufficient to generate a relevant list of diseases from each category. For the simulated viral pandemic activity, students were assembled into 20 teams of 6 to 7 students, each of which was tasked with collecting and analyzing data to formulate hypotheses for the source of the pandemic. Each team rotated through a series of 3 stations: a 10-minute viewing of a simulated inpatient room, during which student teams gathered data from bedside clinicians about their observations of patients' physical exam findings and clinical course, and two 10minute encounters with actors portraying eyewitnesses, during which students gathered data about patients' initial symptoms (of note, students could not directly interview the patients themselves, as the simulated illness was characterized by severe aggression and disorientation). During portions of the activity when they were not rotating in one of the above stations, students worked with their teammates to review and analyze other data provided on a simulated "Emergency Operations Center" website which housed simulated epidemiological data and video clips of frontline clinician reports. The epidemiological data included the case definition of the illness; 2 distribution maps illustrating the number of cases of the disease across the world and among states within the USA, respectively; epidemic curves for 6 US cities and one other country where the illness was first identified; and tables of risk ratios for 5 US states, which included risk ratios and 95% confidence intervals for various age groups, males versus females, different racial/ethnic groups, and rural versus urban locations. The video clips of frontline clinician reports provided clinical observations about infected patients from the perspectives of an emergency medicine physician, a neurologist, a pathologist, and a critical care physician (videos were 3 to 7 min, each). Links to ethics resources were also provided, including brief descriptions of a list of relevant ethical considerations (e.g., duty to care, fairness, transparency, etc.) as well as a brief video of an ethicist highlighting several ethical issues that have arisen during the simulated pandemic, including acting in patients' best interest, struggles with resource allocation, challenges in protecting frontline healthcare providers who are at high risk of infection, and mandatory quarantine (though specific strategies for addressing these issues were not provided). Periodically throughout the activity, students were also emailed a series of "updates from the field," the first of which was a simulated chart review summarizing the clinical characteristics noted at presentation of patients seen in the local emergency department, including the mean and range of patient ages, frequencies of specific presenting signs and symptoms, laboratory and radiological findings, and clinical outcome (discharged to home vs. ongoing hospitalization vs. death). The second email provided students with electron micrograph images of the mystery pathogen, including captions providing clues to the viral structure. The third email, sent in the latter half of the activity, presented students with risk ratios of adult and pediatric patients treated with fresh frozen plasma and/or cryoprecipitate, indicating lower risk of mortality among patients treated with the latter; this email presented students with the additional task of developing an ethically sound strategy for allocating this limited resource (which was in short supply) to infected patients. Students were instructed to electronically submit a team report using a prepared template on Microsoft Word (Microsoft Corporation, Redmond, WA); the template included prompts directing students to (1) analyze clinical, epidemiologic, and molecular data to determine the most likely pathogen causing this outbreak; (2) develop recommendations for treatments or treatment strategies, based on the leading hypothesis, and (3) develop recommendations for public health measures in the local community to prevent further spread of the infection. Students were required to explicitly justify each of these items, using supporting evidence from the data they collected during the activity and/or relevant information collected during self-directed learning. Students were also instructed to cite the sources they used during their selfdirected learning. Following the conclusion of the 2-hour attendance-required activities, student teams were allowed an additional 2 hours to work on their team reports, if desired, before final submission. Students' reports were analyzed using an exploratory sequential mixed method design. First, a content analysis approach was used to characterize students' diagnostic reasoning and patient management tasks [29] . We systematically reviewed all reports and formulated descriptive codes for the data through an iterative data analysis process and then organized these codes into related themes. We then repeatedly reviewed the reports to clarify and refine the codes and themes using constant comparison; we then calculated the frequency of each code among students' reports to provide insight into which problem-solving tasks students selected most frequently to address this case scenario. These codes were initially organized by themes according to the course or discipline representing the source of learning content described in each code (e.g., virology, epidemiology), to characterize the disciplines integrated by students in their reports for this problemsolving exercise. Next, we identified a subset of these task codes as diagnostic reasoning tasks by identifying the codes indicating students' use of data to identify potential pathogens for students' differential diagnosis and/or students' use of data to justify their working diagnosis. These diagnostic reasoning task codes were then categorized by diagnostic reasoning approach type: a compare-contrast approach (i.e., illness script elements of patients' disease presentation are compared with those of known diseases) vs. a causal approach (i.e., a postulated pathophysiologic mechanism explaining patients' clinical findings is used to select diagnoses for the differential) [11, 12] . Compare-contrast approach codes were characterized as such if a task involved a direct comparison of the simulated patients' features to the features of known diseases. Next, we analyzed the learning content of students' reports using the same exploratory sequential mixed method approach described above. Once coded, we identified which of these topics had been covered previously in the curriculum prior to this simulation event by comparing this list of topics with students' learning materials (e.g., lecture slides, notes, assigned readings) from all courses leading up to this simulation. We also met with applicable course directors to confirm which content was new versus previously encountered. Topics not identified in students' prior learning materials, or those that were only briefly mentioned but not covered in detail, were considered new learning content. Finally, we analyzed the resources students cited from their self-directed learning in a similar fashion, organized them into categories of source type (specific websites, UpToDate®, journal articles, textbooks, or other online sources), and calculated the citation frequency of these sources, among all sources cited in students' reports. Twenty student team reports were collected and analyzed (139 total students). We identified 16 codes for problemsolving tasks, organized into themes by the source course or discipline; these findings are summarized in Table 1 . These task codes demonstrated that students integrated knowledge and skills from a variety of disciplines in order to address their assigned problem-solving tasks, including virology, epidemiology, clinical skills, and bioethics. Eleven of the above 16 problem-solving task codes were identified as diagnostic reasoning task codes. When these codes were categorized according to the diagnostic reasoning approach used to formulate the differential diagnosis, all (100%) student team reports used a compare-contrast approach to identify candidate diseases for their differential diagnoses, and a subset (4, 20%) of team reports also included a causal approach to diagnostic reasoning. Among the virology topics cited in students' reports (Table 2) , most had been encountered previously in the microbiology course prior to this simulation event, though several viruses and viral disease treatments mentioned in the reports represented new learning content. Aspects of disease states (e.g., disseminated intravascular coagulation, shock) mentioned in students' reports had been introduced previously, but management of these conditions represented new content. As for epidemiology topics, the biostatistics topics and the disease surveillance, transmission, and prevention measures concepts explicitly mentioned in students' reports had all been encountered previously, though management of large-scale disease outbreaks at the community and healthcare system levels represented new content. Among the sources selected for self-directed learning cited by students' reports, we identified 6 codes for source types, listed in Table 3 . With the exception of 1 hard-copy textbook cited in 1 team report, all sources cited were externally derived, online resources. Analysis of students' hypotheses and treatment recommendations for this simulated viral pandemic activity provided data demonstrating evidence of cognitive integration of both previously encountered learning content from multiple • Identifies similarities between unknown pathogen's cellular/molecular structure and diseases with similar features a 95% (19) • Rationale for recommended treatments linked to diseases in the differential diagnosis for this case 95% (19) • Identifies similarities between patients' physical exam findings and diseases with similar presenting exam findings a 90% (18) • Identifies similarities between patients' geographic location and those of the diseases in the differential diagnosis a 70% (14) • Rationale for recommended treatments linked to suspected pathophysiologic mechanism occurring in patients in this case 70% (14) • Identifies similarities between patients' lab findings and diseases with similar presenting lab findings a 60% (12) • Identifies direct exposure risks among patients related to risk factors for the diseases in the differential diagnosis for this case a 40% (8) • Explains abnormal clinical findings based on pathophysiologic mechanisms of diseases in the differential diagnosis b 20% (4) Tasks applying clinical skills content (data collection, communication skills) % Reports (n) • Identifies similarities between patients' symptoms and diseases with similar presenting symptoms a 100% (20) • Identifies similarities between patients' illness narrative (tempo, timing, and sequence of symptoms) and diseases with similar presenting symptoms a 80% (16) Tasks applying epidemiology content % Reports (n) • Rationale for recommended public health measures linked to transmission route(s) of the diseases in the differential diagnosis for this case 75% (15) • Identifies similarities between likely transmission route and that of diseases in the differential diagnosis, based on interpretation of epidemiologic data a 45% (9) • Identifies groups with higher vs. lower risk of mortality, based on interpretation of epidemiologic data 45% (9) • Identifies groups with higher vs. lower risk of acquiring disease, based on interpretation of epidemiologic data a 15% (3) Tasks applying ethics content % Reports (n) • Rationale for recommended resource allocation explicitly linked to relevant, sound ethical principles 95% (19) a Diagnostic reasoning task codes using a compare-contrast approach b Diagnostic reasoning task codes using a causal approach disciplines and newly acquired learning content. These reports also provided insights into first-year medical students' diagnostic reasoning approach, which consisted primarily of a compare-contrast process between the clinical features of patients in the case scenario versus illness scripts of known diseases, though a subset of students also applied a causal approach to identify candidate diseases. Our findings were somewhat surprising given that, over the preceding initial 6 months of medical school, the medical students in our study had been participating in a longitudinal case-centered, PBL course that had routinely tasked them with using a causal (i.e., pathophysiologic analytical) approach to differential diagnosis formulation. In addition, previous studies on diagnostic reasoning among medical students indicate these novice learners tend to utilize a causal approach [11] [12] [13] [14] [15] . However, in our study, only a small proportion of student reports alluded to using a causal approach. These contrasting findings are likely in part due to the fact that most prior studies were conducted on students at more advanced stages of training than that of our learner group. The use of a compare-contrast approach we observed among students' reports following the pandemic simulation is likely explained by the fact that their biomedical knowledge base was not yet sufficient at that point in their training to allow them to analyze the clinical data of the pandemic case with a causal approach. Indeed, these students had not yet been exposed to much of the pathophysiology implicated by the signs and symptoms of the patients in the simulated case scenario (excessive bleeding and aggression). Our findings may be explained by the search-inference framework proposed by Aberegg et al., which is not unlike what patients often do to investigate their own symptoms [30] . In this framework, novice problemsolvers-motivated by a specific goal-develop a list of possibilities by seeking their memory or external sources and then use inference to compare and evaluate these possibilities by seeking evidence for and against each one, again using their memory or external sources. The problem-solver then selects the solution mostly likely to satisfy the goal, based on his/her evaluation of the evidence. However, as Aberegg and colleagues point out, such an approach-though practical for the novice-is susceptible to many types of cognitive biases that, left unchecked, could easily lead the clinician astray. Without an adequate understanding of the underlying pathophysiological mechanisms of disease, simply using a compare-contrast approach is not likely to be a successful long-term diagnostic strategy for novices. In fact, understanding and connecting the underlying causal mechanisms to their corresponding clinical manifestations of disease has been shown to be beneficial to the diagnostic reasoning performance of novice learners, with improved long-term memory retention of diagnostic categories and superior diagnostic accuracy when solving difficult cases [5] [6] [7] . As novices' biomedical knowledge base continues to expand during their preclinical training, practicing diagnostic reasoning through challenging cases, such as the one presented in our simulated pandemic activity, has benefits for further developing their mental representations of disease. Chamberland and colleagues found that challenging cases appear to activate medical students' biomedical knowledge by stimulating pathophysiological inferences during self-explanation [18] . Other studies have demonstrated the benefit of self-explanation during diagnostic reasoning, resulting in improved diagnostic accuracy of complex cases, even without receiving feedback on one's clinical reasoning [17, [31] [32] [33] . Self-explanation is thought to improve learners' understanding of the learning material by activating cognitive processes that ultimately strengthen the cognitive connections between elements of the information learned [32] . Some investigators have found that the use of specific prompts can enhance the effectiveness of learners' selfexplanations on learning [19, 34, 35] : justification prompts (prompts to justify one's reasoning with underlying principles and concepts) are thought to trigger learners' biomedical knowledge by focusing them on the underlying pathophysiological mechanisms; by contrast, mental model revision prompts (prompts to compare and contrast one's existing knowledge to new knowledge) are thought to help students revise and make corrections to their existing knowledge. Despite the fact that justification prompts were included in the report template and student instruction documents for the simulated pandemic activity, our students' written diagnostic justifications did not include much discussion of pathophysiological mechanisms. Again, this discrepancy may be due to our more novice learner group and their limited biomedical knowledge at the time of the simulation. This study provided insights into how preclinical medical students select sources of information during their self-directed learning, which is consistent with the findings of several prior studies. Graber and colleagues found that third-year medical students use electronic resources extensively during problemsolving, particularly Google's search engine; online journal articles and other clinical decision tools such as UpToDate® were used much less frequently. Focus group findings from their study suggested potential reasons for these source usage patterns related to students' familiarity with Google and their inability to assess the relative efficacy of online decision-support resources [36] . More recent studies have shown similar findings, with Google, Wikipedia, and social media being the information-seeking resources most frequently used by contemporary medical students, whereas online journals, scholarly databases, and medical texts are accessed relatively infrequently [37] [38] [39] . Features such as ease of use and efficiency of access appear to heavily influence today's medical students' choices of such resources [39] [40] [41] . Limitations to this study include that it was a single institution study, and the generalizability of our findings may be limited by the specific case scenario and learning activity used in this study. Student team reports were submitted as a group and therefore may not fully reflect every individual student member's diagnostic justification performance. The 4-hour time limitation for the simulated pandemic activity may have affected students' performance of the assigned tasks, particularly with respect to their selected information-seeking behaviors. In addition, students' actual diagnostic reasoning approaches while working in their small groups may have included other strategies beyond those documented in their reports; their use of causal inferences, for example, could have been more prevalent than what was reflected in their written reports. Similarly, students' information sources during self-directed learning may have included other sources beyond those cited in their reports. This study provides insight into early medical students' diagnostic reasoning approach and information-seeking behaviors during self-directed learning tasks. Future study is needed to determine if the same behaviors we observed in this study would occur in this learner group when presented with other clinical problem-solving scenarios. Such data would add further support to the search-inference framework hypothesis and could have important implications for clinical reasoning curricula for learners at this stage in medical school training. Collaborative, task-based problem-solving during a simulated pandemic can provide preclinical medical students opportunities to integrate learning content from multiple disciplines. Evidence of learners' cognitive integration can be made explicit through their narrative justification of problem-solving tasks. Patterns of learners' task performance can provide important insights into their problem-solving, clinical reasoning, and self-directed learning approaches. Our study suggests that early medical students tend to rely on a compare-contrast approach to diagnostic reasoning similar to the search-inference framework described by previous authors. Students' sources of information for self-directed learning were consistent with findings of other studies of this generation of digital native learners. These findings could have implications for clinical reasoning curricular design for these early learners. Author's Contribution All authors contributed to the study conceptualization and design. Material preparation and data collection were performed by Jennifer Jackson and Timothy Peters. Data analysis was performed by Jennifer Jackson, Timothy Peters, and Joseph Skelton. The first draft of the manuscript was written by Jennifer Jackson, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. Ethical Approval All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee (Wake Forest School of Medicine Institutional Review Board, IRB00050949) and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Cognition before curriculum: rethinking the integration of basic science and clinical learning Integration of basic sciences and clinical sciences in oral radiology education for dental students The role of basic sciences in diagnostic oral radiology Cause and effect: testing a mechanism and method for the cognitive integration of basic science Exploring cognitive integration of basic science and its effect on diagnostic reasoning in novices Science is fundamental: the role of biomedical knowledge in clinical reasoning It all make sense: biomedical knowledge, causal connections and memory in the novice diagnostician The value of basic science in clinical diagnosis: creating coherence among signs and symptoms The value of basic science in clinical diagnosis Expanding the basic science debate: the role of physics knowledge in interpreting clinical findings A universal model of diagnostic reasoning How expertise develops in medicine: knowledge encapsulation and illness script formation A cognitive perspective on medical expertise: theory and implication The role of basic science knowledge and clinical knowledge in diagnostic reasoning: a structural equation modeling approach Development of clinical reasoning in an undergraduate medical program at a Brazilian university Adjusting our lens: can developmental differences in diagnostic reasoning be harnessed to improve health professional and trainee assessment? Examining the effect of selfexplanation on cognitive integration of basic and clinical sciences in novices Students' self-explanations while solving unfamiliar cases: the role of biomedical knowledge Self-explanation in learning clinical reasoning: the added value of examples and prompts Examining the diagnostic justification abilities of fourth-year medical students Illness script development in pre-clinical education through case-based clinical reasoning training To think is good: querying an initial hypothesis reduces diagnostic error in medical students Concept mapping enhances learning of biochemistry Assessing the effectiveness of using mechanistic concept maps in case-based collaborative learning The use of simulation in teaching the basic sciences Early bedside care during preclinical medical education: can technology-enhanced patient simulation advance the Flexnerian ideal The role of causality in judgment under uncertainty Causal learning and inference as a rational process: the new synthesis Analyzing qualitative data: systematic approaches The searchinference framework: a proposed strategy for novice clinical problem solving The influence of medical students' selfexplanations on diagnostic performance Self-explaining expository texts: the dual processes of generating inferences and repairing mental models The effects of self-explanation training on students' problem solving in high-school mathematics Learning argumentation skills through the use of prompts for self-explaining examples Testing the instructional fit hypothesis: the case of self-explanation prompts Resources medical students use to derive a differential diagnosis Selection and use of online learning resources by first-year medical students: cross-sectional study Informationseeking behaviors of medical students: a cross-sectional web-based survey Expediency-based practice? Medical students' reliance on Google and Wikipedia for biomedical inquiries Qualified doctor and medical students' use of resources for accessing information: what is used and why? Project information literacy: what can be learned about the information-seeking behavior of today's college students Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations The authors declare that they have no conflicts of interest.