key: cord-0721841-wyxixmj7
authors: Logan, Alexander A.; Rao, Mayuree; Cornia, Paul B.; Hagan, Scott L.; Newman, Thomas A.; Redinger, Jeffrey W.; Woan, Jessica; Albert, Tyler J.
title: Virtual Interactive Case-Based Education (VICE): A Conference for Deliberate Practice of Diagnostic Reasoning
date: 2021-05-19
journal: MedEdPORTAL : the journal of teaching and learning resources
DOI: 10.15766/mep_2374-8265.11159
sha: baa1d58ed791c42e2c60e5490d42f15e36b6a5c0
doc_id: 721841
cord_uid: wyxixmj7

INTRODUCTION: Current approaches to teaching diagnostic reasoning minimally address the need for deliberate practice. We developed an educational conference for internal medicine residents to practice diagnostic reasoning and examine how biases affect their differential diagnoses through cognitive autopsies. METHODS: We formatted the Virtual Interactive Case-Based Education (VICE) conference as a clinical problem-solving exercise, in which a facilitator presents a case to a single discussant selected from the audience. We delivered VICE on an internet-based conferencing platform with screen-sharing capability over approximately 30 minutes. To maximize learners’ psychological safety, we employed an active facilitation model that normalized uncertainty and prioritized the diagnostic process over arriving at the correct diagnosis. RESULTS: Resident attitudes toward VICE were assessed by utilizing a postconference survey and gathering descriptive data for 11 sessions. Ninety-seven percent of respondents (n = 35) felt that VICE was a novel and valuable addition to their curriculum. Qualitative data suggested that positive features of the conference included the opportunity to practice diagnostic reasoning, the single-discussant format, and the supportive learning environment. Discussants reported that holding the conference in person would have negatively impacted their experience. DISCUSSION: Internal medicine residents universally valued the opportunity to engage in deliberate practice of case-based reasoning in a psychologically safe environment during the VICE conference. The virtual nature of the conference contributed significantly to discussants’ positive experience. This resource includes all materials necessary to implement VICE, as well as an instructional video on facilitation.

Diagnostic errors are a common source of avoidable patient harm and excess health care expenditures, 1 yet medical training fails to adequately develop trainees' skills in diagnostic reasoning. 2 In response to this problem, the National Academy of Sciences has called on medical educators to improve teaching and assessment of clinical reasoning. 3 Most current approaches to developing diagnostic reasoning skills focus on undergraduate medical learners, 4 and so, more effort is needed to develop dedicated programming that addresses this educational gap at the graduate medical education level, 1 particularly within internal medicine. 2 concepts, and diagnostic skills. [5] [6] [7] There are a number of resources available in MedEdPORTAL addressing diagnostic reasoning. These include introductory didactic materials on cognitive errors and decision-making, 8, 9 case-based curricula to teach medical students to recognize cognitive errors, 10, 11 and a simulation-based curriculum for emergency medicine residents on cognitive error avoidance. 12 These existing open-source resources largely target the identification and avoidance of cognitive bias. Although empiric evidence regarding optimal methods for teaching diagnostic reasoning is limited, 4 this approach, when undertaken alone, is likely insufficient. 13 It is crucial that learners have the opportunity to engage in supported deliberate practice of diagnostic reasoning. 14, 15 Interventions that prompt recall and reorganization of knowledge regarding a clinical problem as well as case-based reflection have been more successful at reducing error. 13 Additionally, existing resources prioritize diagnostic reasoning with inpatient cases and do not adequately address trainees' needs to practice diagnostic reasoning in the outpatient setting.

The COVID-19 pandemic has significantly disrupted in-person medical education, necessitating innovative approaches to virtual learning. 16 This resource contains the materials necessary to implement a virtual interactive case-based educational conference (VICE). VICE is intended to facilitate deliberate practice of diagnostic reasoning skills around common clinical problems, coupled with stimulated casebased reflection in the form of a structured cognitive autopsy. 17 Rather than simply transitioning an existing conference to the virtual setting, VICE leverages virtual learning to build an environment of psychological safety in which learners feel free from judgment. 18 We successfully implemented VICE to practice diagnostic reasoning in both the inpatient and outpatient settings.

To develop VICE, we drew on self-determination learning theory, 19 the dual-process theory of diagnostic reasoning, 14, 15, 20 and the concept of psychological safety. 18 We incorporated a number of elements that have been called for in diagnostic reasoning education: acknowledging and embracing uncertainty, 2 reflecting on what went well/what could be improved rather than targeting the identification and naming of biases, 2,13 making implicit reasoning decisions more explicit, 21 encouraging learners to make commitments, 22 ensuring a nonjudgmental and supportive environment for trainees to display their thinking, 2 and emphasizing the diagnostic process rather than getting the right answer. 2,23

We modelled VICE after the New England Journal of Medicine Clinical Problem-Solving series 24 and the medical podcast The Clinical Problem Solvers, 25 drawing its basic format as a clinical problem-solving exercise from the former and its tone and focus on psychological safety from the latter. We designed VICE to be implemented on an Internet-based videoconferencing platform with screen-sharing capability (Zoom).

A facilitator began the conference by introducing the format, affirming that the goal was to practice diagnostic reasoning (rather than to get the right answer), and encouraging participants to acknowledge the limits of their knowledge or experience. The facilitator then solicited volunteers for the role of designated discussant. Using a dedicated PowerPoint template (Appendix A), the facilitator next presented a case in brief aliquots (discrete, predetermined portions of clinical data) to the discussant (see Appendix B for prepared cases). After the facilitator had presented each aliquot, the discussant commented on their problem representation and differential diagnosis while the facilitator recorded and represented the discussant's thoughts visually for the group, utilizing the screen-sharing function. (See the VICE -outpatient -edema.pptx file in Appendix B, as well as Appendices C and D for facilitator instructions and tips.) The facilitator offered ample positive feedback and used prompting questions to draw out the discussant's reasoning. 19 Prior to revealing the final diagnosis, the facilitator asked the discussant to commit to the diagnosis that they felt was most likely.

Throughout this process, the facilitator promoted group engagement with audience-specific questions; however, group participation in diagnostic reasoning was limited to maintain the integrity of the discussant's thought process. Following the case presentation, the facilitator guided the discussant through a cognitive autopsy of their diagnostic reasoning (see Appendix C for instructions). The discussant reviewed the evolution of their differential diagnosis over time, examining how cognitive biases may have played a role and identifying how they could further develop relevant diagnostic schema or illness scripts. Time permitting, the facilitator presented a brief didactic relevant to either the final diagnosis or the clinical problem.

We implemented VICE as a weekly conference with internal medicine residents at all PGY levels. Either a faculty member or a chief resident facilitated each conference. Facilitators were familiar with the concepts of diagnostic schema, problem representation, illness scripts, and cognitive biases and had reviewed the conference materials ahead of time. We asked facilitators to review a facilitation guide (Appendix C) and an instructional video (Appendix D) prior to their conference. Attendance was optional for residents. As facilitators adapted and successfully implemented cases, we added them to a case library (Appendix B). We modified case data to protect patients' identities.

To evaluate VICE, we assigned an observer to record descriptive data on attendance and time spent during each component of the conference. Additionally, we designed a survey (Appendix E) to assess residents' perceptions of the conference's educational value and the relative importance of individual elements of the conference format. We administered this voluntary, anonymous survey to residents (both discussants and audience members) at the completion of each conference session.

Our survey tool included free-text prompts. Two coders (Tyler J. Albert and Mayuree Rao) systematically analyzed responses to these prompts through inductive iterative review. They created codes after independently reviewing the responses and then came to consensus about the code assignment. We tabulated the frequency of comments by code using Microsoft Excel and then synthesized the comments into generalizable themes. A third reviewer (Alexander A. Logan) independently examined the uncoded data and the themes and categories derived by the other two coders as a final check. All researchers reached consensus regarding the categorization of the data. 26

We recorded descriptive data for 11 conference sessions: eight using outpatient cases and three using inpatient cases. Attendance ranged from five to 35 residents (median = 12). Conferences were generally 30 minutes long, with a mean of 3.2 minutes spent on cognitive autopsy, 1.7 minutes spent on a brief didactic, and the remainder of the session spent in introduction and discussion of the case. In all 11 sessions, the facilitator explained the conference format, normalized uncertainty, and facilitated a cognitive autopsy. In four sessions (36%), the facilitator skipped the didactic portion due to time constraints.

Thirty-seven residents (Table 1 ) filled out the survey (one partial completion has been included in the analysis). Thirtyfive respondents (97%) agreed or strongly agreed with the statement "VICE provides something valuable that is otherwise missing from my residency curriculum." In qualitative comments (Table 2) , residents appreciated having a designated opportunity to practice diagnostic reasoning; for example, "Great to hear how other trainees work through diagnostic dilemmas. I don't think we talk frequently enough about diagnostic reasoning." In view of the uniquely vulnerable nature of the discussant role, we asked discussants (n = 10) about potential negative experiences during the conference (Figure) . Although some discussants endorsed feelings of stress about being wrong or making mistakes (60%) and lack of confidence in their diagnostic skills (90%), none reported feeling judged by peers or attendings. Discussants universally valued their experience, despite sometimes feeling anxious or uncomfortable, as indicated by the following sample comment:

[The experience was] very positive. Of course it is a little anxiety-provoking showing your own clinical reasoning amidst a group of brilliant peers, but it was [a] super helpful process to go through. Really highlights your own strengths, weaknesses, oversights. Great to be aware of these things to improve my own clinical reasoning moving forward.

We asked residents to rate their willingness to serve as the discussant both before and after the session using a 5-point Likert scale (both questions were asked retrospectively). Mean willingness increased from 3.2 (before the session) to 4.0 (after the session; p < .01 by Wilcoxon signed rank test).

We explored how the unique components of VICE (singlediscussant format; learning atmosphere; virtual, rather than in-person, venue) affected residents' experiences. Relative to working through a case as a group, 100% of discussants and 65% of audience members felt that the single-discussant format enhanced their learning. Residents mentioned each component favorably (Table 2) , often contrasting VICE's single-discussant format and learning environment to morning report conferences. One resident commented:

I really enjoy the practice of thinking through something start to finish. It disrupts the traditional [morning report] model in which people only speak up when they "know" something, which I think can sometimes create an unhealthy learning environment. Watching someone else or doing it yourself in the [VICE] way allows a humbler, frankly more realistic approach that reflects real thinking on the part of one person or team.

Another resident wrote:

In contrast [to morning report], the VICE format allows you to see people work through something and usually say "I don't know" or "I think I remember something about this," etc. It disrupts the traditional paradigm of group learning wherein people's already fully-formed knowledge gets rewarded and instead offers an environment where an actual thought process (full of deficits, twists, turns, biases) gets put on display.

To explore the effect of VICE's virtual venue, we asked residents how VICE should be offered once in-person conferences could safely resume. The majority of residents (78%) supported offering it as an in-person conference. However, when we asked discussants how such a change would affect their experience, the majority felt it would worsen their feelings of being judged by peers (60%) and attendings (70%) as well as increase their level of stress about making mistakes (70%).

Diagnostic reasoning is a crucial skill for internists, yet approaches that afford internal medicine residents opportunities to explicitly practice this skill are lacking. Here, we present a conference format in which trainees engage in deliberate practice of diagnostic reasoning and participate in a cognitive autopsy. This resource includes all materials necessary to implement this conference, as well as a number of tailored cases that can be utilized with this format with minimal preparation.

We structured VICE to induce guided metacognition-the application of analytical reasoning to one's intuitive thought process. 15, 27 In contrast to traditional clinical problem-solving formats, we often placed the chief complaint as the initial aliquot, with background information (such as the past medical and social history) as the second, rather than the history of present illness. This better simulated the modern clinical environment, in which clinicians often have access to this background information prior to taking a history. We found that this approach allowed the discussant to reflect on the intuitive hypothesis generation that begins with the chief complaint and continues by integrating background information. Although the discussant could fall victim to cognitive bias by engaging in intuitive or heuristic reasoning with this limited information, it could equally lead them to the correct diagnosis. 13 In either case, we found that this approach yielded rich material for reflection and discussion. We instructed the facilitator to moderate a cognitive autopsy and deliver a brief didactic after leading the discussant through the case. We found that visually representing the discussant's thought process and capturing how it evolved over time enabled a productive cognitive autopsy. We structured the conference to last 30-45 minutes, depending on the length of the case. Facilitators occasionally omitted didactic teaching due to time constraints. Conversely, we considered the cognitive autopsy a core element of the conference and an important opportunity for trainees to engage in deliberate reflection. We used a simple four-question approach to facilitate a brief but meaningful cognitive autopsy (see Appendix A for prompts).

Residents felt that this conference format was both a novel and valuable addition to their training. They cited VICE's explicit focus on diagnostic reasoning skill development and the singlediscussant format as positive factors. We believe that having a single discussant (rather than a group) work through a case was important for resident learners. This format allowed for a relatively uninterrupted and unaided process of reasoning to unfold in a way that more accurately simulated how a trainee would approach a case in actual practice. We feel that this helped participants focus on the process of diagnostic reasoning and reflect on that process, rather than exclusively aiming to get the right answer. Additionally, we believe a single, coherent thought process offered a better substrate for a cognitive autopsy than a group thought process would. However, learners such as medical students or residents unfamiliar with the format were sometimes reticent about serving as the discussant. In these scenarios, we found it helpful to slightly modify the format by having a different learner serve as the discussant after each aliquot. Alternatively, we could have asked two volunteers to trade off interpretation of aliquots.

We recognized that asking a single resident to discuss a case in front of a group of their peers could lead to significant anxiety. We therefore intentionally structured the conference format and facilitation model to create an environment of psychological safety for all involved. 18 In doing so, we aimed to construct a learning environment in which discussants experienced stress in a way that enhanced, rather than detracted, from learning. 28 We instructed facilitators to normalize uncertainty, foster a culture of courage and risk-taking, emphasize process over outcome, and provide ample positive feedback. Our survey results demonstrated that while discussants did experience some stress, none feared being judged by colleagues or attendings. Moreover, our participants reported a significant increase in their willingness to serve as the discussant after experiencing the session, suggesting that the learning environment successfully mitigates some of the intimidating aspects of the format. However, not every learner responded well to being in the warm seat, as some discussants struggled to build a differential diagnosis or reflect on their reasoning process in a productive way. We have included strategies to mitigate these scenarios in the facilitator guide (Appendix C).

Facilitators strove to keep the audience engaged while preserving the integrity of the discussant's thought process. In survey responses, audience members supported the singlediscussant format at a lower rate than discussants (65% vs. 100%). This was not unexpected given that serving as a discussant entailed a considerably more active learning process. However, we were encouraged that both audience members and discussants felt VICE added value to their curriculum and that the single-discussant format was often mentioned favorably in qualitative responses. In our experience, the audience members often contributed significantly to the positive learning environment by providing encouragement to the discussant. We have included a number of strategies to build audience engagement while preserving the integrity of a single discussant's thought process in the facilitator guide, including encouraging audience members to play along on paper and building audience-specific questions into the case. If VICE were implemented in a longer format, virtual tools such as audience response polls and breakout rooms could also be employed.

The choice of who attends and facilitates educational conferences likely affects psychological safety. 18 Both chief residents and attendings facilitated our VICE conferences; we felt that each of these options worked well. However, it is critical that facilitators be well prepared, given VICE's unusual format and complex logistics, as well as the importance of maintaining a safe and productive learning environment. We recommend that prospective VICE facilitators observe at least one session, read the facilitator guide, familiarize themselves with the case and slide set, and watch the instructional video prior to hosting a conference. We did not invite medical students to participate in VICE, as we felt that having a relatively uniform population of learners helped build psychological safety. However, VICE could easily be implemented for medical students or for a mixed group of learners. Based on our experience and the data we collected from discussants, the presence of attending physicians in VICE did not seem to impact the conference's psychological safety, but we asked that attendings refrain from participating in discussion until facilitators invited them to do so. Given that almost all discussants experienced lack of confidence in their diagnostic skills, we felt it was crucial to avoid the impression that this activity was evaluative. Thus, we asked program leadership not to attend the conference.

We initially conceived of VICE in response to the disruption of in-person educational activities as a result of the COVID-19 pandemic. 29 We were interested in how the virtual aspect of the conference affected learners' experiences. Although residents in our sample preferred transitioning to an in-person format once feasible, discussants reported that implementing the conference in person would negatively impact their fears of being judged. We believe that holding the conference virtually introduces some psychological distance between the discussants and their peers, thereby contributing to the experience of a safe learning environment. For this reason, we plan to continue hosting the conference virtually when social distancing is no longer required, although we could easily administer the conference materials in an in-person venue.

This resource has a number of limitations. Assessment of educational interventions to improve diagnostic reasoning is limited due to the lack of consensus regarding relevant outcomes and the dearth of high-quality, reliable tools to measure diagnostic accuracy or reasoning styles. 5, 30 We did not feel it was feasible to directly measure residents' skill acquisition. Attendance at VICE was not mandatory, and it is possible that residents who chose to attend were systematically different from those who did not, potentially introducing bias into our survey results. Additionally, because we did not compel residents to complete the assessment tool following the conference, our results are subject to nonresponse bias. However, within the discussant population, we achieved near saturation (10 responses out of 11 total discussants), so results from discussants are unlikely to be significantly affected by nonresponse bias.

VICE added to our residents' education as a stand-alone conference, but it may function best when situated in a longitudinal curriculum that includes training on reasoning theory and cognitive bias recognition. 5, 7 Although we implemented VICE with a relatively limited population of learners (internal medicine residents), we have also piloted it with medical students and believe it would be generalizable to most medical learners, as long as cases are selected and adapted appropriately.

We found VICE to be a unique, engaging, and fun addition to our residency curriculum-in both the inpatient and outpatient settings-during the COVID-19 pandemic. It was very well received by residents, teaching faculty, and program leadership alike. The flexible format of VICE can offer a variety of trainees the important opportunity to participate in a clinical problemsolving exercise, thereby practicing diagnostic reasoning skills in a supportive and productive learning environment. Future work will focus on implementing VICE with other learner populations (e.g., medical students) and in longitudinal curricular contexts, as well as on assessing more directly how VICE affects learners' sense of self-efficacy in diagnostic reasoning and the likelihood that they will continue with self-guided deliberate practice. 

Clinical reasoning as a core competency

Diagnostic reasoning: an endangered competency in internal medicine training

Improving Diagnosis in Health Care

How to improve the teaching of clinical reasoning: a narrative review and a proposal

Development and evaluation of a clinical reasoning curriculum as part of an internal medicine residency program

Teaching about how doctors think: a longitudinal curriculum in cognitive bias and diagnostic error for residents

Embedding a longitudinal diagnostic reasoning curriculum in a residency program using a bolus/booster approach

Decision -diagnosis: an introduction to diagnostic error and medical decision-making

Understanding and preventing cognitive errors in healthcare

Teaching cognitive biases in clinical decision making: a case-based discussion

Bridging the gap between the classroom and the clerkship: a clinical reasoning curriculum for third-year medical students

Medical heuristics and cognitive decision errors: modeling thought processes through an integrative curriculum

The causes of errors in clinical reasoning: cognitive biases, knowledge deficits, and dual process thinking

Teaching critical thinking: a case for instruction in cognitive biases to reduce diagnostic errors and improve patient safety

Teaching clinical reasoning: case-based and coached

Innovation in response to the COVID-19 pandemic crisis

Exploring the construct of psychological safety in medical education

Ten Cate OTJ. Twelve tips to stimulate intrinsic motivation in students through autonomy-supportive classroom teaching derived from self-determination theory

Dual processing and diagnostic errors

Critical thinking in critical care: five strategies to improve teaching and learning in the intensive care unit

Twelve tips for teaching expertise in clinical reasoning

Uncovering the relation between clinical reasoning and diagnostic accuracy -an analysis of learner's clinical reasoning processes in virtual patients

Browse Clinical Problem-Solving

Clinical Problem Solvers

Three approaches to qualitative content analysis

A universal model of diagnostic reasoning

The stress paradox: how stress can be good for learning

Seattle VICE: virtual interactive case-based education

Dual-process cognitive interventions to enhance diagnostic reasoning: a systematic review