key: cord-0720161-8amebgwo
authors: Kositanurit, Weerapat; Vivatvakin, Sarocha; Kaikaew, Kasiphak; Varachotisate, Pachara; Burana, Chuti; Chayanupatkul, Maneerat; Thanprasertsuk, Sekh; Wangsaturaka, Danai; Kulaputana, Onanong
title: Asynchronous online lecture may not be an effective method in teaching cardiovascular physiology during the COVID-19 pandemic
date: 2022-03-09
journal: BMC Med Educ
DOI: 10.1186/s12909-022-03217-w
sha: 468378919a03a71c2cb9163d6a438e95025fde7e
doc_id: 720161
cord_uid: 8amebgwo

BACKGROUND: Asynchronous online lecture has become a common teaching method in medical education, especially during the COVID-19 pandemic. However, the effectiveness and students’ attitudes towards this method under this special circumstance have not been exclusively studied. Hence, we aimed to evaluate these aspects of cardiovascular physiology teaching in an undergraduate medical curriculum. METHODS: We analysed and compared the academic achievement and attitudes of 613 medical students on cardiovascular physiology between pre-COVID-19 and COVID-19 years in which different teaching methods were implemented. In addition, we also explored the importance of teaching methods and teachers by subgroup analysis to evaluate whether they influenced the academic achievement and attitudes of students. RESULTS: Overall students’ academic achievement was significantly higher when lectures were taught by the traditional method than by the asynchronous online method. Moreover, subgroup analysis revealed that teachers were also a factor influencing students’ academic achievement. Although most students had positive attitudes towards asynchronous online lectures, overall satisfaction was slightly higher when all lectures were taught by the traditional method than by the asynchronous online method. CONCLUSIONS: Asynchronous online lectures might not be an effective teaching method especially during the abrupt change in education. Under the ‘new normal’ medical education, not only teaching methods but also teachers are the essential keys to the success in academic achievement and attitudes of undergraduate medical students.

Lecture in a classroom has been a traditional and widely accepted method for large group teaching in medical education [1] . In the last two decades, the internet and its relevant technology were introduced and have become a part of our daily life, including its assistance in remote learning, i.e., electronic learning (e-learning). Also, e-learning has been applied in undergraduate and postgraduate medical education for more than a decade [2] [3] [4] . Several e-learning modalities have been developed by integrating emerging technologies, such as simulation, synchronous online learning, and asynchronous online learning [2] . The effectiveness of internet-based learning has been studied and revealed that internet-based learning is as effective as traditional lecture [5] [6] [7] .

In 2020, the coronavirus disease 2019 (COVID- 19) was an emerging infectious disease and became a worldwide pandemic within months. Affected countries implemented various policies aiming to control the outbreak, such as physical distancing, lockdown, and the closure of crowded places including schools and universities [8] . Therefore, the COVID-19 outbreak substantially accelerates the development of online teaching and learning to lessen the disruption of all levels of medical education [9] . Online teaching has become a preferable method, or sometimes the only feasible teaching method, for preclerkship education [10, 11] .

COVID-19 outbreak emerged in Thailand in February 2020. The Ministry of Public Health of Thailand issued the policies for urgent prevention of disease transmission. Our institution, obliged by the government policies, also announced the regulations and guidelines for all academic activities during the pandemic. All teachings, with the exception of crucial clinical skills, must shift to an online method to keep physical distancing. In this situation, asynchronous online lecture might be one of the favourable teaching methods suitable for the 'new normal' standard during the outbreak [11] . We believed that asynchronous online lecture could mimic the traditional lecture since it fulfilled the fundamental basis of giving essential knowledge by teachers. Previous studies have shown that online teaching, when used, is as effective as on-site teaching, as seen by the improvement or similarity of students' academic performance [6, [12] [13] [14] [15] . Moreover, asynchronous online teaching is more favourable than on-site teaching since teachers (or instructors) and students can keep physical distancing and students can study without any time or space limit [11] . Nevertheless, there are some barriers interfering with the success of online teaching and learning, for instance, poor technological skills of teachers and inadequate institutional support, lack of time to develop e-learning materials, and feeling of disengagement between teachers and students [16] .

Until now, the effectiveness and students' attitudes towards asynchronous online teaching, particularly when online teaching is the only feasible teaching method for a large group of medical students during the pandemic, have not been studied exclusively. Thus, we aimed to investigate the effectiveness and students' attitudes towards asynchronous online lectures compared with the traditional lectures on the topics of cardiovascular physiology in a pre-clerkship course of the undergraduate medical curriculum. In addition, we also aimed to compare students' academic achievement and students' attitudes between the two different teaching methods taught by the same teachers and among different teachers teaching similar topics.

This study was designed to evaluate academic achievement and attitudes of the second-year medical students towards teaching methods used in the lecture topics on cardiovascular physiology, retrospectively for the pre-COVID-19 year and prospectively for the COVID- 19 year. All students enrolled in the Cardiovascular System I course of the Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand were included in the study unless they opted out. The Cardiovascular System I is the course taught in the second year of the medical curriculum comprising 16 lecture topics (24.5 h) and 17 non-lecture topics (44 h) involving anatomy, histology, biochemistry, physiology, and basic haematology. Of all lecture topics, 10 topics are relevant to cardiovascular physiology, listed in Table 1 . Details of the course were described previously [17] . In this study, we aimed to analyse only topics related to cardiovascular physiology since they were taught solely by lectures, which were the focus of this study, and accounted for a majority of knowledge content covered by the course.

Concerning the topics of cardiovascular physiology presented in Table 1 , some topics were taught by the same teachers while some topics were taught by different teachers between these two academic years. Nevertheless, all teachers were content experts in the respective cardiovascular topics. The topics covered in this course and the teaching duration of each topic were similar for both years. In the pre-COVID- 19 year, all lecture topics were taught in a large lecture theatre by traditional lectures. Due to the COVID-19 pandemic and the implemented policies of physical distancing, all topics were shifted to asynchronous online lectures in the COVID-19 year. Asynchronous online lectures were newly recorded by the lecturers who taught those topics. Most lecturers used computer software, such as Microsoft PowerPoint and OBS Studio, to record lecture slides together with their narrations. One recorded lecture covered one topic and typically lasted 45-60 min. The online lectures, a.k.a. e-lectures, were uploaded to the available online platforms of the faculty (the MDCU E-learning website, available at http://e-learn ing. md. chula. ac. th) and of the university (the learning management system myCourseVille, available at https:// www. mycou rsevi lle. com/). After the lecture sessions, an interactive question-and-answer (Q&A) session was conducted in the lecture theatre in the pre-COVID- 19 year and via an online platform in the COVID-19 year to answer students' questions and to interactively discuss any unclear points raised by the students. Of note, the Q&A session was a non-compulsory session.

Students' academic achievement was assessed by the summative exam taking place approximately 6 weeks after the course completion. In both academic years, the summative exams of these courses were held on-site in the examination hall. With the epidemic prevention measures in the COVID-19 year, the exam was arranged similarly to the pre-COVID- 19 year. The summative exam consisted of 5-option single best answer (SBA) questions, i.e., 112 questions for the pre-COVID-19 year and 108 questions for the COVID-19 year. Specifically, 64 out of 112 questions and 62 out of 108 questions directly aimed to assess the knowledge on cardiovascular physiology taught by lectures. The students' summative scores were analysed and reported as a percentage of the total scores. Of note, the remaining questions were excluded as they were not relevant to our study objectives, i.e., they were used to assess knowledge on anatomy, histology, biochemistry, or cardiovascular physiology that taught by other teaching methods such as online live conference, online small group discussion, and asynchronous laboratory demonstration.

To ensure the equivalence of the summative assessment used in these two years, five medical doctors were asked to evaluate the relevance and difficulty of all examination items using the modified Ebel method, resulting in the minimum passing level of each item [18] . The items from the two academic years were pooled and randomly ordered so that the raters would not know to which year each item belonged. All raters independently determined the level of relevance (important/essential, acceptable, or questionable/additional) and difficulty (easy, moderate, or difficult) of each item. The expected percentage of correct answers was categorised into the 3 × 3 relevance-by-difficulty matrix table [18] [19] [20] . The percent-correct values and discrimination indices of all items were also analysed [21] . Items with negative discrimination indices, i.e., one item from each year, were excluded. The reliability coefficients of the tests were calculated using the Kuder-Richardson 20 (KR-20) formula [22] . The KR-20 value between 0.70 and 0.90 was accepted as a reliable assessment [23] .

To identify if a difference in teachers/lecturers influences the academic achievement, we performed subgroup analysis by categorising each lecture topic into the topic taught by the same teachers (topics 4-8 and 10 in Table 1 ) or the topic taught by different teachers (topics 1-3 and 9 in Table 1 ). Then, a difference in academic achievement between academic years was analysed using unpaired t test.

To assess student attitudes, course evaluation was performed at the end of the course through an online questionnaire, in which all items had been created, reviewed, and approved by the course committee, using Google Forms (the free web-based Google Docs Editors). Students could anonymously and voluntarily fill in the questionnaire. Students evaluated the quality of each lecture with a 5-point rating scale as follows: 5, excellent; 4, good; 3, average; 2, fair; and 1, poor. In the COVID-19 year, 8 statements in Table 2 were included to evaluate student agreement (strongly agree, agree, uncertain, disagree, or strongly disagree) on asynchronous online lectures and the Q&A session.

Lastly, an open-ended question was included to specifically survey students' attitudes towards online teaching. For the open-ended question, students' comments were categorised into positive and negative comments by the consensus of 2 independent investigators. The questionnaire had been carefully reviewed and approved by the Cardiovascular System I course committee and used to evaluate student satisfaction for several years, except for the 8 newly added statements in Table 2 . In addition, all items in the questionnaire were assessed for content validity and reliability [24] . Content validity was determined using the index of item-objective congruence by 3 independent experts. Reliability was determined by Cronbach's alpha. Our analyses showed that the indices of item-objective congruence of the 8 newly added statements and the satisfaction questions were 0.79 and 1.00, respectively. Cronbach's alpha was 0.87 for the 8 newly statements and 0.75 for satisfaction questions representing an acceptable quality of the questionnaire.

To identify if a difference in lecturers influences the student satisfaction, we also performed subgroup analysis into topics taught by same or different lecturers (Table 1) . Then, a difference in student satisfaction between academic years was analysed using unpaired t test.

Categorical data, i.e., sex of the participants, were demonstrated as frequency and percentage and analysed by Chi-square test. Unless otherwise indicated, continuous data were presented as mean, standard deviation (SD), and 95% confident interval (95% CI). To compare the difference between groups, continuous data including satisfactory level and the subgroup analyses were analysed by unpaired t test. Statistical analysis was performed by Microsoft Excel (version 16.0.13530.20368) and Graph-Pad Prism (version 9.0.0). A p value of < 0.05 was used to infer statistical significance.

Of 613 students enrolled in the study, 301 were those in the pre-COVID-19 year and 312 were those in the COVID-19 year. Baseline characteristics of the students did not differ between these two years, as shown in Table 3 . As our main objective was to investigate the effectiveness of teaching methods between academic years by evaluating students' academic achievement, we compared the three parameters (minimum passing levels, percent-correct values, and discrimination indices) between 2 academic years and found that they were not significantly different (Fig. 1) . The reliability coefficients were also within the accepted range (KR-20 values = 0.83 for the pre-COVID-19 year and 0.84 for the COVID-19 year).

Academic achievement of students, as indicated by the summative scores, was significantly higher in the pre-COVID-19 year (when lectures were taught by only the traditional on-site method) than in the COVID-19 year Table 1 ), and C score grouped by topics taught by different teachers (topics 1-3 and 9 in Table 1 ). Data are presented by box-and-whisker plots using the Tukey method [boxes indicating an interquartile range (IQR), whiskers indicating 1.5 times IQR, lines in the box indicating the median, and plus signs in the box indicating mean]. Differences in scores between academic years in A and B were analysed by unpaired t test (when lectures were taught by the asynchronous online method) (Fig. 2A) . The difference between means was 3.48 ± 0.97% (95% CI: 1.58 -5.38). Subgroup analysis was performed to verify possible factors explaining the findings. The academic achievement of students in the lecture topics taught by the same teachers but different teaching methods was not significantly different between two academic years (Fig. 2B) . The difference between means was -1.93 ± 1.00% (95% CI: -3.90 -0.04). Interestingly, the subgroup analysis in topics taught by different teachers between two academic years showed that the score in the pre-COVID-19 year was significantly higher than that of the COVID-19 year (Fig. 2C) . The difference between means was 6.79 ± 1.13% (95% CI: 4.58 -9.00). Of note, this subgroup analysis (Fig. 2C) was a similar pattern observed in overall performance ( Fig. 2A) .

Course evaluation via an anonymous questionnaire was received from 268 students (89.0%) in the pre-COVID-19 year and 204 students (65.4%) in the COVID- 19 year. Overall satisfaction of lectures was significantly higher in the pre-COVID- 19 year than in the COVID-19 year (Fig. 3A) . In the subgroup analysis to compare different teaching methods by the same teachers, we found that student satisfaction was not different between these two years (Fig. 3B) . On the other hand, the subgroup analysis to compare different teachers lecturing the same topics showed that student satisfaction was significantly higher in the pre-COVID- 19 year than in the COVID-19 year (Fig. 3C) . Of note, this pattern was in line with the students' academic achievement presented in Fig. 2C . We studied students' perspectives towards asynchronous online lectures and found that the majority of students had positive attitudes towards asynchronous online lectures in all investigated aspects ( Table 2 ). In general, the most satisfactory statements indicated a preference for the voluntary Q&A session and the convenience of asynchronous online lectures. Interestingly, the least agreement was the item concerning whether the asynchronous online lecture is more preferable than the traditional lecture, i.e., only a half of students rated in favour of the online lecture (Table 2) . Lastly, students' feedback towards asynchronous online lectures was presented in Table 4 . Of note, some students reported drawbacks of asynchronous online lectures, such as the instability of the e-lecture platform and the longer time needed to finish each online lecture. Table 1 ), and C satisfaction grouped by topics taught by different teachers (topics 1-3 and 9 in Table 1 ). Data are presented by box-and-whisker plots using the Tukey method [boxes indicating an interquartile range (IQR), whiskers indicating 1.5 times IQR, lines in the box indicating the median, and plus sign in the box indicating mean]. Differences in satisfactory levels between academic years in A and B were analysed by unpaired t test

For our cardiovascular physiology lectures, the asynchronous online lecture was a teaching option before the COVID-19 outbreak but became the only teaching method during the outbreak due to an abrupt change in pandemic prevention policy. It was beyond our expectations that the academic achievement was higher in the pre-COVID-19 year (when all lectures were taught by the traditional on-site method) than in the COVID-19 year (when lectures were taught by the asynchronous online method). Our finding indicated that asynchronous online lecture may not be an effective substitute for the traditional on-site lecture. Possible explanation might be that a major limitation of asynchronous online lectures could be a lack of student-teacher interaction which might result in a loss of motivation and subsequently a decline in academic performance, in accordance with a previous study [25] . Therefore, an increase in interaction between students and teachers via interactive synchronous online sessions could be a solution. For example, three-fourths of the students in our study preferred the Q&A session which was synchronous and interactive. In addition, our notion was supported by previous studies demonstrating that highly interactive sessions could enhance students' academic performance and that medical students preferred more interactive sessions for online learning such as case-based discussion and interactive live lectures [26] [27] [28] [29] . Other explanations include an urgent shift to online learning without adequate preparation, limited resources, and insufficient technological support [30] which affect the quality of asynchronous online lecture.

Our finding also suggests that the teacher/lecturer might be another factor influencing student performance other than the teaching method or teaching platform alone. Our results revealed that different teachers/lecturers could bring about differences in academic outcomes of students. We believe that teaching skills might potentially be a key explanation. In higher education, good teaching skill has been the first rank characteristic of a good teacher [31] , leading to the better academic achievement of medical students [32] . One way to improve teaching skills of the academic faculties is a continuum of faculty development. Obviously, a previous systematic review has shown that faculty development has a strong impact on enhancing faculties' performance resulting in better students' academic achievement [33] .

A large body of research has shown that teaching methods have an influence on students' attitudes towards engagement in learning and possibly influence academic performance in medical education [7, [34] [35] [36] . Similar to previous studies [37, 38] , we found that a majority of the students in the COVID-19 year favoured asynchronous online lectures over traditional on-site lectures for many investigated aspects (Table 2) . However, we found that the asynchronous online lecture alone did not lead to higher student academic performance than the traditional on-site lecture. Although asynchronous online lectures have shown many benefits, only a half of our students admitted that they preferred learning by the asynchronous online lecture to the traditional lecture for the subsequent academic year. This finding is logically incompatible with most studies, reviewed by Tang et al. [34] , that indicated the students' satisfaction towards the online teaching method. The possible explanations might include the diversity of students' learning preference, technical issues (e.g. the instability of the e-learning platform and errors in internet connection), and the non-interactive nature of asynchronous online learning between teachers and students, as mentioned in previous studies [14, 16, 25, 38] .

The strength of our study was the large number of students, i.e., 613 students enrolled in the study. However, there are two major limitations. Firstly, our study partly employed a retrospective design that limits the determination of the direct causal relationship between teaching methods and the academic achievement of students. Secondly, since all students in the same academic year were taught by the same teacher with the same teaching -There are some technical issues, such as the instability of the e-learning platform and errors in internet connection. (24) -Students need more time to learn from asynchronous online lectures. (6) -Students lack the motivation to learn from asynchronous online lectures. (1) method in each topic (to limit the chance of inequality of knowledge distribution), we could not compare the effects of teaching methods or teachers on academic performance of students from the same class.

Asynchronous online lecture alone might not be an effective teaching method in undergraduate medical education particularly when the country's public health policy on physical distancing measures is required. Apart from teaching methods, teachers likely remain an essential key to the success in academic achievement and attitudes of undergraduate medical students. Moreover, we believe that the combination of asynchronous and synchronous online sessions together with a continuing development in teaching skills of academic faculties could be a potential solution in the 'new normal' medical education. 

The What or the How: a Review of Teaching Tools and Methods in Medical Education

The future of E-Learning in medical education: current trend and future opportunity

Computer assisted learning in undergraduate medical education

E-learning as new method of medical education

The effectiveness of a clinically integrated e-learning course in evidence-based medicine: a cluster randomised controlled trial

Computer-based teaching is as good as face to face lecture-based teaching of evidence based medicine: a randomised controlled trial

A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education

SARS-CoV-2 and the pandemic of COVID-19

The Transformational Effects of COVID-19 on Medical Education

The impact of COVID-19 on the undergraduate medical curriculum

Challenges and Opportunities of Preclinical Medical Education: COVID-19 Crisis and Beyond

Comparing face-to-face, synchronous, and asynchronous learning: postgraduate dental resident preferences

The comparison of teaching efficiency between massive open online courses and traditional courses in medicine education: a systematic review and meta-analysis

Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis

Learning online, offline, and in-between: comparing student academic outcomes and course satisfaction in face-to-face, online, and blended teaching modalities

Barriers and solutions to online learning in medical education -an integrative review

Two-dimensional integration approach to teaching cardiovascular physiology: effectiveness and students' perspectives

Setting and maintaining standards in multiple choice examinations: AMEE Guide No. 37

Psychometric characteristics of integrated multi-specialty examinations: ebel ratings and unidimensionality

Standard setting

convenient online submission • thorough peer review by experienced researchers in your field • rapid publication on acceptance • support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year • At BMC, research is always in progress. Learn more biomedcentral.com/submissions Ready to submit your research Ready to submit your research ? Choose BMC

Constructing Written Test Questions For the Basic and Clinical Sciences

Reliability: on the reproducibility of assessment data

Post-examination analysis of objective tests

Developing questionnaires for educational research: AMEE Guide No

Distance education during the COVID-19 outbreak: A cross-sectional study among medical students in North of Jordan

Asynchronous Distance Learning of the National Institutes of Health Stroke Scale During the COVID-19 Pandemic (E-Learning vs Video): Randomized Controlled Trial

Perceptions of medical students towards online teaching during the COVID-19 pandemic: a national cross-sectional survey of 2721 UK medical students

Utility of Real-Time Online Teaching During COVID Era Among Surgery Postgraduates

A novel structure for online surgical undergraduate teaching during the COVID-19 pandemic

Cecilio-Fernandes D. Solutions, enablers and barriers to online learning in clinical medical education during the first year of the COVID-19 pandemic: a rapid review

Students' views on student-teacher relationship: a questionnaire-based study

Is there a relationship between attending physicians' and residents' teaching skills and students' examination scores

The impact and effectiveness of faculty development program in fostering the faculty's knowledge, skills, and professional competence: A systematic review and meta-analysis

Online Lectures in Undergraduate Medical Education: Scoping Review

Can blended learning and the flipped classroom improve student learning and satisfaction in Saudi Arabia?

Comparison of the effect of lecture and blended teaching methods on students' learning and satisfaction

Students' and lecturers' perspective on the implementation of online learning in dental education due to SARS-CoV-2 (COVID-19): a cross-sectional study

Limitations and Recommendations for online learning during COVID-19 pandemic era

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

We thank Assoc. Prof. Rasmon Kalayasiri, Dr. Thanapob Bumphenkiatikul, and Dr. Tanat Tabtieang for the questionnaire validation and also thank Prof. Suthiluk Patumraj, Assoc. Prof. Prasong Siriviriyakul, Assoc. Prof. Wasan Udayachalerm, and all academic staff members of the Department of Physiology, Faculty of Medicine, Chulalongkorn University for organising the physiology lessons in the Cardiovascular System I course.

OK conceived the study. WK, KK, DW and OK designed the study. WK, SV, PV, and CB collected the data. WK, KK, MC, and ST analysed the data. WK, KK, and SV drafted the data. All authors provided valuable feedback on the manuscript and approved the final manuscript.

There were no specific grants from any source of funding for this research.

Since students' summative scores are personal data and we did not ask their consents to publicly publish the data, the datasets generated and/or analysed during the current study are not publicly available but are available from the corresponding author on reasonable request.

This study was approved by the Institutional Review Board of the Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand (COE reference no. 050/2020), and all methods were performed in accordance with the relevant guidelines and regulations. All second-year medical students during 2018 and 2020 who gave informed consent and participated in the knowledge assessment were included in the study. Their participation in the study was exclusively voluntary and whether participating or not would not affect their grades.

Not applicable.

The authors declare that they have no competing interests.