key: cord-0883894-18o5tm1p authors: Hofstädter-Thalmann, Eva; Rotgans, Jerome I; Aybar Perez, Noelia; Nordquist, Jonas title: Effective Learning in Virtual Conferences: The Application of Five Principles of Learning date: 2022-01-10 journal: Journal of European CME DOI: 10.1080/21614083.2021.2019435 sha: 64ef01a3d179b538191adac44940718e0a56ee05 doc_id: 883894 cord_uid: 18o5tm1p In this article, we examine the adaptation of learning among scientists and healthcare professionals in conferences and symposia from face-to-face to fully virtual meetings accelerated in the last years. Advantages and limitations for both settings have been described in different research studies but the effectiveness of learning can be reflected similarly by applying five fundamental principles of learning, which are based on empirical research in cognitive psychology. From a practical context, we compared the individual learning outcomes from two satellite symposia conducted face-to-face in 2019 and virtually in 2021 at the European Congress of Urology, EAU. Although both conference formats were almost identical, the five principles of learning were applied in both symposia. There were also some differences due to adaptation to online conferences, and our findings suggest that the virtual conference was perceived as significantly more effective than the face-to-face conference on all five criteria, and digital learning is a valid alternative to face-to-face conferences. What still needs to be better understood and analysed is the informal learning that is taking place during conferences, but suggesting an active design of any digital event by combining “technical literacy· with “learning literacy” will enable us to better analyse and study the impact of learning using the five learning principles in the design of other events in the future. Medical conferences and seminars have always provided important opportunities for sharing and discussing scientific knowledge among scientist and health-care professionals. Besides the central objective of knowledge dissemination, these events provide significant opportunities for social interactions and networking. In recent years, however, the landscape started to change. Whereas these events were traditionally conducted face-to-face, many are now conducted online or in a hybrid fashion [1, 2] . Although the shift from face-to-face was accelerated due to the COVID-19 pandemic, changing to virtual events was on the agenda for quite some time [3] . This was mainly due to technological advancements, such as better internet access and speed and the availability of user-friendly virtual meeting software, such as Zoom, Skype, Microsoft Teams, and FaceTime. There are significant advantages of conducting conferences and seminars online in a virtual environment. According to a recent review [4] , virtual events have a number of advantages. First, no travel is required, which equates to significant savings in transportation and accommodation costs [4] . In addition, registration fees for virtual conferences are generally cheaper and sessions can be accessed anywhere with a variety of devices ranging from computers to hand-held tablets and smart phones [5] . As a consequence of the easy access to virtual events, they have the potential to ease social inequalities, since easier access allows disadvantaged individuals to participate in the scientific discourse, independent of gender, race, geography or social status [6] [7] [8] . There are also some downsides of conducting virtual events. Sometimes, there are technical glitches that limit accessibility and differences in time zones make it difficult to attend for participants at the same time from different parts of the world. It has also been noted that participants, who attend virtual events are easily distracted by work-related matters (emails need to be answered) or home routines [9] . There are also difficulties in reading non-verbal reactions from presenters and the audience, which can make it a less interactive experience. All in all, however, virtual conferences have successfully been implemented in many disciplines and since the COVID-19 pandemic, there is growing evidence that it is a feasible approach and is here to stay in one way or the other. With many organisations moving towards virtual events, research has sprung up documenting this transition [9] [10] [11] . Besides these mainly descriptive accounts, there are now also papers that provide practical suggestions and tips to consider when planning and executing virtual events [7, 12, 13] . However, largely missing in the literature are accounts that focus on principles that enhance learning during such events. Arguably, learning is at the heart of all conferences or seminars. Presenters share new insights with the audience and engage in an active academic dialogue, which constitutes learning on a large scale. Many activities at conferences and seminars are intuitively directed towards enabling learning (e.g. team challenges, interactive presentations, breakout sessions), but a formal framework is often missing. A fundamental question that needs to be addressed is, how do people learn during conferences or seminars and are there any differences between face-to-face events and virtual events that need to be considered? It is possible that learning in virtual events is impaired due to lack of face-to-face interactions that significantly limit attendees' ability to acquire new knowledge and insights. The objective of the present paper was to make a first attempt to address these questions by providing an overview of five fundamental principles of learning. We will briefly explain how these principles can be applied to face-to-face events as well as virtual events. We will endeavour to do this from a practitioner's perspective by referring to an actual large-scale conference that has been conducted face-to-face and virtually. Finally, we will present empirical findings that provide first insights in the effectiveness of transferring the conference to a virtual event. We will end the paper by elaborating on the findings and proposing future avenues for research. The five principles of learning adapted for this study, were derived from the active-learning literature, particularly from the research on problem-based learning [14] . Problem-based learning is an effective instructional method for adult learners and is anchored in research on how people learn [15, 16] . From this body of research, we extracted five key learning principles, which we believe provide an adequate blueprint for evaluating learning at educational conferences and professional development programmes. See Figure 1 for an overview of these five principles and their suggested overlapping characteristics. We will briefly summarise these principles below. A common misconception is that learning constitutes "filling empty vessels with knowledge". All individuals have knowledge stored in memory, referred to as priorknowledge. Learning is activating this prior knowledge first and then linking new information to this knowledge. Only if prior knowledge is activated, new information can be assimilated into the knowledge structure of the learner. See, for example [17] [18] [19] [20] [21] [22] [23] . Learning happens in context. If new information is presented in an abstract form, without context, it is difficult to assimilate this information into memory. Providing context enables the learner to (1) better understand the new information (e.g. providing examples) and (2) part of this context is encoded together with the newly acquired knowledge (e.g. a patient example of a medical condition). The fact that acquired knowledge is more likely to be retrieved in the context it was learned, is an important learning mechanism. See, for example [24] [25] [26] [27] [28] [29] . It is not sufficient to merely present information to learners and assume that the learner will remember that information. Even if examples are provided and prior knowledge is activated, it is no guarantee that learning will be successful. Learning is activity during which the learners deliberately construct their own meaning by making sense of every piece of information and linking it to what they know. This also includes elaborating of what one knows and what one does not know about the topic in question. The latter is a powerful mechanism that is often a successful element in the construction of meaning and understanding. See, for example [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] . In its simplest form, knowledge is organised in semantic networks as concepts. Concepts are linked with each other via propositional statements. Memory retrieval is activation of a concept and activating linked concepts in that network (spreading activation). If knowledge becomes specialised through experience and many years of expertise development, memory network structures can be encapsulated to increase storage and re-activation efficiency. Note that knowledge consolidation in the brain takes time. See, for example [40] [41] [42] [43] [44] . Finally, learners must be willing to invest effort and be motivated to learn. Situational interest is a powerful mechanism that entails arousing a learner's interest in the moment (situation) by providing a learning stimulus, such as a problem, presenting new and incongruent information, or presenting unexpected causal events. Situational interest is activated by learners realising that they have a knowledge gap regarding the stimulus presented. It has proven to be a powerful mechanism that leads to information-seeking behaviour (i.e. learning) to close the knowledge gaSee for example [14, [45] [46] [47] [48] [49] [50] [51] [52] . Before we provide an example of how these five principles of learning can be applied in a real-life context, it should be highlighted that these five principles should not be considered in isolation. Instead, they are intertwined and closely related with each other. For instance, prior knowledge can be activated by means of arousing situational interest. If learners are presented with an incongruent problem or puzzle during the start of a learning event, they will activate their prior knowledge (Principle #1) from memory (Principle #4) and elaborate what they know about the problem (Principle #3). The problem provides context (Principle #2). If they come to the realisation that they have a knowledge deficit with regard to the problem at hand, their situational interest will be triggered (Principle #5), which leads to a desire to find out more about the problem until they know the answer. This in turn leads to learning which represents consolidating new information into memory (Principle #4). Returning to the question whether learning is fundamentally different between face-to-face and online learning, there seems no reason to believe that there are substantial differences in the application of these five principles to face-to-face and virtual learning modes. For learning to be successful in the virtual context, learners still need to be provided with a context to activate their prior knowledge, be situationally interested and engaged in active construction of meaning. Hence, from the psychological perspective it appears that all five principles of learning apply when engaging in a face-to-face event or a virtual event. As example we selected a satellite symposium conducted in 2019 as a large-scale face-to-face conference of the European Urology Association, EAU. This conference was conducted as a virtual satellite at the EAU conference in 2021. For a comparison of both formats see Table 1 . Overall, there were 13 digital satellites at this year's EAU. 6 In 2019 delegates could send in their questions via a website to the co-chairs in 2021 we had the Q&A chat function open for the satellite to encourage delegates to ask questions or send in comments. See Appendix for a detailed breakdown of the programme with associated learning principles. Although both conference formats were almost identical, there were also some little differences. For instance, the organisers changed the length of the satellite from 90 minutes in 2019 and 63 minutes in 2021. In both satellites the following elements of the learning principles were included: 1. Content dependent memory, 2. Context learning on a patient case, 3. Context learning of novel information 4. Active construction of meaning for each of the three specific clinical topics. Presentations were shortened compared to the 2019 satellite and patient cases were used in the following clinical situations -low and high volume metastatic Hormon Sensitive Prostate Cancer and metastatic Castration Resistant Prostate Cancer to ensure activation of prior knowledge and situational interest for delegates. Although it is difficult to objectively compare the outcomes of both conferences in terms of learning gains, we made an attempt to explore to what extent the overall objectives of both conferences were met. To that end, we compared the quantitative feedback obtained at the end of both events. The evaluation consisted of five items: (1) Learning Objectives met; (2) Impact on practice; (3) Educational quality of symposium compared to other educational interventions; (4) Content relevance; and (5) Scientific objectivity. The items were scored as percentage. For an overview of the results see Figure 2 . Overall, the findings suggest that the virtual conference was perceived as significantly more effective than the face-to-face conference on all five criteria. The effect sizes (eta-squared) were moderate, explaining about 10% of the variance. The overall Net Promotor Score (NPS) covering the question "Would you suggest this satellite to a colleague", in 2019, was 56 and in 2021 73. In addition, the feedback suggests that on the other items such as "Impact on practice" and "Educational Quality compared to other educational interventions", the score of the virtual event was significantly higher. The largest differences were observed on "Impact on Practice" (83% vs. 96%). Virtual conferences and other digital learning events are certainly here to stay on all levels of training; undergraduate, post graduate and continuing professional development. We have now reached a stage where the "technical literacy" seems to have increased significantly and many people are better prepared to either present behind a camera or organise an event/ activity. Modern communication services/platforms have made all this rather smooth. What now remains is to add the "learning literacy" on top of this, i.e. apply an evidence-based design. Our data suggest that the five evidence-based learning principles discussed in this paper apply both to face-toface learning events and to digital learning and that digital learning is a valid alternative to face-to-face when comparing similar events run in 2019 and 2021. We consider this as an encouraging finding as we can also see that many digital events tend to attract more non-traditional attendees and have the potential to reach a much larger and geographically distributed audience. We conclude by suggesting that active design of any digital event should combine "technical literacy" with "learning literacy". This will also enable us to better analyse and study the impact of learning, longitudinally and by comparison. Apart from the formal curriculum at a digital meeting, what still needs to be better understood and analysed is the informal learning. Networking and peer-to-peer learning outside formal sessions are important features of face-to-face meetings but how or if this happens in formal digital learning events needs further systematic attention. Chatbox content, frequency, communication patterns, etc., can easily be traced and this could lead to a number of studies both quantitative and qualitative. Our present study has clear limitations in that it only analyses 2 events but with a deliberate approach to educational design using the principles as outlined above for future events, we can start to analyse, and not just describe digital learning events. EHT is lead of external scientific relations EMEA at Janssen Cilag Pharma GmbH and stockholder of J&J. She was involved in the design of the satellites, preparation and final approval of the manuscript, but she was not involved in the interpretation of the data. JIR is adjunct Associate Professor at Erasmus University Medical Center, Institute for Medical Education Research Rotterdam (IMERR) and has prepared parts of the Introduction and the data analysis, NAP is lead of external scientific relations EMEA at Janssen Cilag Pharma GmbH and stockholder of J&J. She was involved in the design of the satellites, preparation and final approval of the manuscript, but she was not involved in the interpretation of the data. JN is the director of the Medical Case Centre at Karolinska Institutet, Sweden. He was involved in the design of this study, in the writing process and final approval of the manuscript. The satellite educational programs were funded by Janssen Cilag Pharma GmbH. No honoraria were given for this publication but was based on an academic/industry collaboration. Context-dependent learning -novel informaƟon is linked to a specific context and also to context-dependent memory from the delegates' pracƟce 18:00--18:05 Discussion with all faculty moderated by co-chairs AcƟve construcƟon of meaning -discussion about new informaƟon 18:05 One-slide introducƟon nmCRPC The next secƟons organise knowledge 18:05-18:15 What is high-risk nmCRPC and how important is MFS? -Including two interacƟve polling quesƟons Context-dependent learning -novel informaƟon is linked to a specific context and also to context-dependent memory from the delegates' pracƟce 18 :15-18:20 Discussion with all faculty moderated by co-chairs AcƟve construcƟon of meaning -discussion about new informaƟon 18 :20-18:40 Why, when and how to treat nmCRPC? Taking a closer look at the results -PresentaƟon Context-dependent learning -novel informaƟon is linked to a specific context 18 :40-18:45 Discussion with all faculty moderated by co- Professional conferences in the COVID-19 era Online conferences-Towards a new (virtual) reality Online Learning amid the COVID-19 pandemic: students' perspectives. Online Submission Reducing the carbon footprint of academic conferences: the example of the Exploring professional online conferences for the adult learner Gender equity in science: the global context Virtual conferences raise standards for accessibility and interactions. eLife Better for whom? Leveling the injustices of international conferences by moving online Virtual and face-to-face academic conferences: comparison and potentials The future of conferences, today: are virtual conferences a viable supplement to "live" conferences? EMBO Rep Virtual conferences becoming a reality Online conferences for better learning Lessons of COVID-19: virtual conferences The process of problem-based learning: what works and why Innovations in problem-based learning: what can we learn from recent studies? Problem-based learning is compatible with human cognitive architecture: commentary on Kirschner Effects of prior knowledge activation modes and text structure on nonscience majors' comprehension of physics The effects of prior knowledge and schema activation strategies on the inferential reading comprehension of children with and without learning disabilities Formats and prior knowledge on learning in a computer-based lesson The relevance of prior knowledge in learning and instructional design Effects of prior knowledge on learning from different compositions of representations in a mobile learning environment Effects of mobilizing prior knowledge on information-processing -studies of free-recall and allocation of study time How including prior knowledge as a subject variable may change outcomes of learning research The impact of selected contextual factors on experts' clinical reasoning performance (does context impact clinical reasoning performance in experts?) Context-dependent memory in two natural environments: on land and underwater Chewing gum and context-dependent memory: the independent roles of chewing gum and mint flavour Stress disrupts context-dependent memory. Learn Memory Background music and context-dependent memory Environmental context-dependent memory: a review and meta-analysis Effects of three types of elaboration on learning concepts from text Elaboration-based learning strategies: powerful theory= powerful application Effect of mobilizing prior knowledge on learning from text Explanatory models in the processing of science text: the role of prior knowledge activation through small-group discussion Prior knowledge activation: inducing engagement with informational texts Conditions and effects of example elaboration. Learn Instruction Elaboration during problem-based group discussion: effects on recall for high and low ability students Collaborative learning tasks and the elaboration of conceptual knowledge. Learn Instruction Training students to use elaboration learning strategies Cue recognition and cue elaboration in learning from examples Functional neuroimaging correlates of thinking flexibility and knowledge structure in memory: exploring the relationships between clinical reasoning and diagnostic thinking Knowledge encapsulation and the intermediate effect Encapsulation of biomedical knowledge, In: Advanced models of cognition for medical training and practice How expertise develops in medicine: knowledge encapsulation and illness script formation Knowledge restructuring in expertise development: evidence from pathophysiological representations of clinical cases by students and physicians A theory of human curiosity Curiosity and learning. Motivation Emotion Curiosity and exploration Situational interest and learning: thirst for knowledge. Learn Instruction Situational interest and academic achievement in the active-learning classroom. Learn Instruction The Wick in the Candle of Learning: epistemic Curiosity Activates Reward Circuitry and Enhances Memory This outcomes evaluation was conducted and lead by Excerpta Medica and Janssen, the Pharmaceutical of Johnson & Johnson.