key: cord-0952624-rznxsjgx authors: Yo, Edward Christopher; Witjaksono, Anissa Nindhyatriayu; Fitriani, Dewi Yunia; Werdhani, Retno Asti; Parikesit, Dyandra title: Assessing webinar outcomes for health professionals: a perspective from Indonesia during coronavirus disease 2019 pandemic date: 2021-05-26 journal: Korean J Med Educ DOI: 10.3946/kjme.2021.190 sha: bf9bc6627d2b52cccfc4f2d56eab12a8db355fe0 doc_id: 952624 cord_uid: rznxsjgx PURPOSE: With the ongoing coronavirus disease 2019 (COVID-19) pandemic, health institutions and hospitals are increasingly relying on e-learning for continuing education. However, in many countries there is still limited data on the effectiveness of online learning particularly in the healthcare field. This study aims to evaluate whether webinar as a form of online educational intervention is satisfactory and effective for the continuing education of health professionals in Indonesia. METHODS: We collected participants’ demographic information including health profession, place of work, work unit, and year of graduation. There were six independent webinars included in this study. Webinar outcomes included satisfaction and learning scores. Regarding satisfaction, participants were told to complete a satisfaction survey and asked whether they would recommend the webinar to their colleagues. Regarding learning, information on their mean pre-test and post-test scores was collected. RESULTS: A total of 3,607 health professionals were enrolled, with the highest participation in webinars about emergency cases and COVID-19 management. The response towards satisfaction was overwhelmingly positive. In all six webinars, post-test scores were statistically significantly higher than pre-test scores. Recently graduated physicians scored higher in learning than senior physicians, while place of work and work unit did not significantly affect the scores. CONCLUSION: The use of webinar for health professionals training in Indonesia was well-received amid the ongoing pandemic. In the future, health institutions and teaching hospitals should optimize the implementation of webinar training as it is associated with low cost, high flexibility, and less time commuting. In the era of the Fourth Industrial Revolution, educators from all sectors are required to keep up with the latest information and communication technology. Today's health professions education, for example, cannot escape the influence of technology on its content and delivery. Medical schools, teaching hospitals, and other health institutions are seeing an increase in the use of e-learning in educating the next generation of health professionals. E-learning, also known as online learning, is referred to as the use of the Internet or other information technologies to acquire knowledge and relevant skills [1] . It can be either asynchronous or synchronous, but the main advantage that e-learning offers is flexibility for participants to adjust their own pace of learning [1] . Although it was historically viewed as a replacement for traditional face-to-face teaching, e-learning is now seen as an adjuvant to transcend the learning process and make it more effective [2] . With the ongoing coronavirus disease (COVID-19) pandemic, online learning has attracted an unprecedented level of fame and has been incorporated by many medical schools around the world in their latest curriculum. By utilizing technology such as videoconferencing platforms, medical educators spend less time commuting to classrooms and more time tending to their work in hospitals and clinics [3] . One of the forms of e-learning commonly used in today's continuing education for health professionals is webinar. Webinar is a form of online educational intervention supported by audiovisual software that connects participants and tutors [4] . Its synchronous nature allows people in different locations to easily exchange questions, ideas, and feedback. Previous studies by Gegenfurtner et al. [4, 5] concluded that webinars are more effective than both asynchronous learning and no intervention, and they are equivalent to or slightly more effective than traditional face-to-face teaching. Although the learning efficacy and satisfaction of webinars in other academic fields are well-documented, these measures are still understudied for webinars related to continuing education for health professionals [4] [5] [6] . Moreover, healthcare and medical education vary between countries. They are affected by country-specific factors, namely economy, technological advances, political situation, social values, and public view of the educational system [7] . To our knowledge, there has not been any study that specifically addresses webinar outcomes for health professions in Indonesia. Hence, the aim of this study is to evaluate the experience of health professionals in Indonesia after participating in webinar, in terms of their satisfaction and learning outcomes. We hope the result of this study will be able to help health institutions, especially in Indonesia, create more effective webinars for health professions education. This is a cross-sectional study that used three set of multi-item questionnaire to evaluate the satisfaction and learning output of participants following six separate webinars conducted by Universitas Indonesia Hospital via Zoom videoconferencing platform and Youtube Live from the hospital's channel. The webinar series, known as i-CORPS (Interactive Course for General Practitioners), was part of Universitas Indonesia Hospital's program for continuing education for health professionals. The hospital is part of a higher education institute that also manages a leading medical school in Indonesia. The reason why the hospital conducted this webinar series was due to pandemic constraints on medical education, where there is a need to re-invent continuing medical education (CME) in a way that does not involve face-to-face teaching. Moreover, we acknowledged that many hospitals in Indonesia are facing staff shortage due to the pandemic, and this has led to a significant increase in workload for health professionals. However, this crisis must not discourage health professionals from participating in CME activity as it is part of their professional obligation. Ethical clearance was given by an Institutional Review Board which is the Ethical Committee of Universitas Indonesia Hospital (005/SKPE/KKO/2020/00). Data from parti-cipants were safeguarded from potential privacy breaches and reserved only for this research. There were six webinars included in the i-CORPS webinar series. Each webinar was delivered in 2 consecutive days with four to five subtopics and lasted for 1 hour each. Arranged in order from the first to sixth webinar, the themes were as follow: emergency cases in daily practice, disorders of the special senses and nervous system, clinical approach towards COVID-19, medicolegal aspect of medicine and non-clinical medicine, metabolic disorders and nutrition, and emergency cases of the vital organs. Albeit having different themes, all six webinars essentially had the same intended goal which was to provide high-quality and comprehensive CME for health workers in Indonesia in spite of the ongoing pandemic. Ultimately, it was hoped that health workers in Indonesia could still gain positive experience and significant learning improvements after participating in the webinar, since CME in Indonesia before the pandemic was heavily reliant on face-to-face teaching only. Registration was done separately for each webinar through the Universitas Indonesia Hospital official website (https://rs.ui.ac.id/webinar/). Therefore, the webinars were independent of one another in terms of scoring and data collection. Each subject could register as participant in as many webinars as they liked. Subjects enrolled were physicians, nurses, pharmacists, other healthcare workers, and medical students and had to be based in Indonesia. Only those who gave consent to the webinar terms and conditions, followed all the procedures, and completed all the questionnaires were included as the study subjects. We collected relevant personal information about the subjects' place of work or institution, health profession, year of graduation, and work unit. Following the end of each webinar, subjects were required to complete a satisfaction questionnaire and a post-test. Post-test could be submitted multiple times (up to 30 attempts) to achieve a score of 60 or more, in order to pass the test. The passing score (≥60 out of 100) was set the same across different groups as all participants were given exactly the same material by the same speaker in a particular webinar. Moreover, it was also adjusted with the passing score for the Indonesia National Board Medical Examination (>65) as well as the number of questions in the pre-test and post-test (10 items). In this study, a satisfaction survey was used to assess satisfaction outcome and the combination of pre-test and post-test scores were used to evaluate learning Indonesia participated in the webinar series ( Table 1) . Most of the subjects were physicians (93.6%) and worked Table 3 shows the distribution of overall satisfaction mean score and response about webinar recommendation among physicians and non-physicians across all six webinars. Moreover, participants were also asked whether they would recommend this kind of webinar to their colleagues in the future. The responses were overwhelmingly positive, with more than 98% of all physicians and non-physicians saying "yes". To our knowledge, this is the first and largest study that explores how webinar affects the satisfaction and learning experience of Indonesian health professionals. In this study, satisfaction is defined as the extent to which participants feel satisfied and find the webinar engaging, whereas learning is referred to as the degree to which participants obtain the intended knowledge from the webinar. Since all methods and data collection were performed fully online, we were able to obtain an overview of webinar outcomes from thousands of health professionals spread across Indonesia as well as make meaningful comparisons between them. Previously, several systematic reviews already evaluated the evidence regarding the benefits of online learning for continuing education for health professionals. Some studies found that e-learning intervention was able to provide a satisfactory learning experience for health professionals and attained comparable learning outcomes to face-to-face teaching [8] [9] [10] . Its strengths lie in its ability to offer a convenient mode of learning for busy professionals, enhance collaboration by bringing together participants from different geographical areas, and provide educational material at a low cost [11] . However, some other studies adopted a more ambivalent or even disapproving attitude towards e-learning, in which they reported that e-learning had negligible effect on health professionals' knowledge and that it was difficult to directly compare e-learning with traditional face-toface teaching [12] [13] [14] . Since conclusive evidence is still lacking, further investigation is warranted. As the webinar series was mainly marketed to physicians particularly general practitioners, it is not surprising that they constituted 93.6% of all subjects and most of them (78.9%) were recent graduates. The fact that Universitas Indonesia teaching hospital was located in Depok, which is a part of satellite Jakarta metropolitan area, may explain why more subjects were from Java Island than from outside Java. We initially In addition, since the pandemic has worsened staff shortage in health facilities [20] , senior workers may feel the need to re-educate themselves outside their area of expertise. The mean overall satisfaction score reported by all physicians and non-physicians ranged from 8.70 (lowest) to 9.00 (highest). These numbers concluded that participants found the webinar series highly satisfying (Table 3 ). Our result supports the findings from researchers in Korea University College of Medicine. Due to the pandemic, its medical education department has changed the format for all seminars to be online. In addition to a surge of increase in participation, they found that participants were able to engage better with webinars through the "chat" feature. Although certain issues such as Internet connectivity and unfamiliarity with the webinar platform may cause negative experience, it is evident from the participants' responses that the benefits outweigh the limitations [21] . A meta-analysis on three different learning methods found that webinar offers higher enjoyment than asynchronous online teaching but lower enjoyment than face-to-face teaching [22] . Nevertheless, the differences are unlikely to be statistically significant. It is too early to tell, however, whether this observed effect is likely to be maintained when face-to-face teaching becomes avail-able again in the future [21] . The pre-test and post-test questions were designed to be at general practitioner level knowledge, so it may be obvious that recent graduates achieved high post-test scores as they could still easily recall the knowledge from their time in medical schools. The main concern is why senior physicians were not able to score as highly as their junior counterparts. Like in other countries, physicians in Indonesia are obliged to participate in CME for re-registration purposes and lifelong learning [23] . Very limited study has been conducted to evaluate the current condition of CME in Indonesia, but studies in Pakistan and Australia found that physicians in Western countries are much more likely to attend CME than physicians in developing countries. Among those who attend, physicians prefer face-to-face activities to online-based activities. A high number of senior physicians have positive attitude towards CME but do not participate. Reasons vary, but the most common ones are lack of information about the benefits of CME, lack of time, and clash with duty hours [24, 25] . The impact of E-learning in medical education E-learning in medical education in resource constrained low-and middle-income countries Evaluation of e-learning for medical education in low-and middle-income countries: a systematic review Evaluating webinarbased training: a mixed methods study of trainee reactions toward digital web conferencing Webinars in higher education and professional training: a meta-analysis and systematic review of randomized controlled trials Assessing the evidence for the educational efficacy of webinars and related internet-based instruction Analysis of the key factors affecting the future of medical education discipline in 2025 based on STEPV model: a qualitative study How does tele-learning compare with other forms of education delivery?: a systematic review of tele-learning educational outcomes for health professionals A systematic review of the effectiveness of videoconference-based tele-education Res Interprofessional online learning for primary healthcare: findings from a scoping review E-learning for health professionals Impact of e-learning on nurses' and student nurses knowledge, skills, and satisfaction: a systematic review and meta-analysis A systematic review of the factorsenablers and barriers -affecting e-learning in health sciences education Clinical governance infrastructures and relational mechanisms of control in healthcare organizations A structuration framework for bridging the macro-micro divide in health-care governance Health governance: principal-agent linkages and health system strengthening ICT based health governance practices: the Indian experience State of emergency medicine in Indonesia The COVID-19 pandemic: effects on low-and middle-income countries Enforced format change to medical education webinar during the coronavirus disease 2019 pandemic Learning and satisfaction in webinar, online, and face-to-face instruction: a metaanalysis The odyssey of medical education in Indonesia Continuing medical education: a cross sectional study on a developing country's perspective The continuing medical education activities and attitudes of Australian doctors working in different clinical specialties and practice locations