key: cord-0887696-axap90an
authors: Wilhelm, Jessica; Mattingly, Spencer; Gonzalez, Victor H.
title: Perceptions, satisfactions, and performance of undergraduate students during Covid‐19 emergency remote teaching
date: 2022-01-20
journal: Anat Sci Educ
DOI: 10.1002/ase.2161
sha: ef882a9bb4c32782c94be081628ea70a6dad2458
doc_id: 887696
cord_uid: axap90an

Due to the Covid‐19 pandemic, the education system worldwide faced sudden and unforeseen challenges. Many academic institutions closed their doors, forcing both educators and students to transition to Emergency Remote Teaching (ERT) for the remainder of the semester. This transition eliminated hands‐on experiences, increased workload, and altered curricula. However, these aspects, as well as students' perceptions, study habits, and performance in response to ERT remain poorly documented. This contribution describes changes in the curriculum of an undergraduate cadaver‐based laboratory, and explores students' performance, self‐perceived learning, and overall satisfaction during this educational crisis. Online content delivery for this course included both asynchronous instruction and synchronous discussion sessions. While formative assessments remained the same, online spotter examinations included short answer, multiple choice, multiple answer, ordering, and true and false questions. Despite examination grades improving 20% during ERT, students reported lower levels of learning, confidence, and engagement with the course materials when compared to the face‐to‐face portion of the class. The most prevalent challenges identified by students were those related to the loss of access to cadaver‐based learning, including difficulty identifying and visualizing structures in three dimensions, and the loss of context and sensorial cues. Flexibility in taking examinations and learning the material at their own pace were recognized as positive outcomes of the ERT transition. While the resulting student perceptions and performances are unsurprising, they offer insight into the challenges of fostering a productive learning environment in a future threatened by epidemic outbreak and economic uncertainty.

differing effects on different cohorts of medical students, with those in their clerkship years potentially being impacted more severely. To increase student involvement and to mitigate the effects of the academic disruption, Harvard Medical School formed a "medical student response team", which allowed students to assist in the Covid-19 relief effort both clinically and educationally (Soled et al., 2020) . In addition, the academic disruption may lead to increased inequality among students of different economic backgrounds (Burgess & Sievertsen, 2020) . For example, socioeconomically disadvantaged physical education students in Brazil were unable to access online classes due to the lack of computer or Internet, and many were only able to attend classes using their cell phone (Pacheco et al., 2020) . Finally, students graduating during the time of will likely enter the job market during a global recession, experiencing job uncertainty and insecurity plagued by the economic crisis.

Anatomy students and instructors have faced unique challenges in response to the ERT transition (Evans et al., 2020; Franchi, 2020; Pather et al., 2020; Ravi, 2020) . The online transition has reduced or eliminated hands-on experiences, increased workload, and altered curricula. Harmon et al. (2021) quantified changes to anatomy courses at medical schools throughout the US and abroad, finding that in-person lectures and the use of cadavers decreased dramatically after the outbreak of Covid-19. Numerous authors have warned of the lasting effects of Covid-19 on cadaver donation programs, as there is potential for a shortage of cadavers for future educational use (Bond & Franchi, 2020; Ooi & Ooi, 2020; Singal et al., 2020; Cheng et al., 2021; Lemos et al., 2021) .

One of the most concerning consequences of Covid-19 and the changes it has invoked to education has been its negative effects on mental health. Since March 2020, a great deal of attention has been afforded to understanding and documenting the psychological effects of the shift to ERT on undergraduate and graduate students (Li et al., 2020; Zhai & Du, 2020a) . For example, the origin of Covid-19 at Wuhan city of China has led to theories of its conception (Mohan & Nambiar, 2020) with collateral social stigmas and prejudice toward Asian peoples (Roberto et al., 2020) . Nearly one-in-four Chinese undergraduate students experienced some form of anxiety regarding Covid-19 (Cao et al., 2020) . These fears were amplified among Chinese students living abroad, who faced additional concerns about discrimination, hate crimes, and being stranded from home (Zhai & Du, 2020b) . In other cases, levels of reported anxiety were even higher. Balkhi et al. (2020) found that as much as 62.5% of residents of Karachi, Pakistan, experienced anxiety daily because of Covid-19, with an even higher percentage among respondents 35 years of age or older. These findings were similar among Pakistani medical students, where 69% believed that Covid-19 affected their social, mental, and psychological well-being (Noreen et al., 2020) .

While these consequences are beyond the scope of this contribution, they highlight an important consideration for educators operating during these moments of crises.

This study sought to document the curricular changes adopted by an undergraduate human anatomy laboratory course during the Covid-19 disruption, as well as to explore students' perceptions and performance during the change from face-to-face (F2F) instruction to ERT. As in many other schools, students at the University of Kansas (Lawrence, Kansas, USA; hereafter KU) attended F2F classes for approximately half of the first academic semester of 2020, and ERT for the remainder of the semester. To assess their perceptions of this experience, an optional Likert-style survey was distributed once classes concluded. Many studies in anatomy education have used Likert-style surveys to measure student and/or instructor preferences regarding online instruction (Petersson et al., 2009; Ogard, 2014; Mathiowetz et al., 2016; Srinivasan, 2020) , and there are abundant resources discussing the best ways to make this transition (Trelease, 2002; Rizzolo et al., 2010; Sugand et al., 2010; Trelease, 2016; Evans et al., 2020; Gewin, 2020) . Student feedback during such transitions is critical to understanding the challenges faced by students, as well as understanding effective study materials and course satisfaction (Gewin, 2020) . Much less is known about abrupt shifts to online education imposed by crises, and thus the transition period is a blind spot in understanding student response to dramatic course and curriculum re-structure. As the situation around Covid-19 and its slew of variants (He, 2021) continues to evolve globally, and as many universities worldwide consider re-opening their doors fully in the near future, understanding the effects of such changes is particularly pertinent.

The KU Anatomy laboratory course is atypical among undergraduate-level classes because it utilizes prosected human cadavers as the main teaching material, which are regarded as premier resources for anatomy education (Ellis, 2001; Patel & Moxham, 2006; Estai & Bunt, 2016) . The use of cadavers in anatomy education not only improves students' spatial reasoning and touch-mediated perceptions, but also fosters attitudes of ethics, empathy, teamwork, and professionalism (Rizzolo & Stewart, 2006) . Considering the myriad benefits of cadaver-based instruction, it is likely that students more highly value the knowledge and experience gained during the F2F phase than during the online-based ERT phase.

Although computer-based instruction is frequently lauded as a supplemental resource (Durosaro et al., 2008; McNulty et al., 2009; Tam et al., 2010; Jaffar, 2012; Green & Hughes, 2013; Green et al., 2014; Topping, 2014; Srinivasan, 2020) , students often prefer cadaverbased learning over computer-based alternatives (Azer & Eizenberg, 2007; Petersson et al., 2009; Kerby et al., 2011; Davis et al., 2014; Ogard, 2014; Attardi et al., 2016) . In addition, numerous studies have shown computer-based learning to be less effective than traditional methods when used as a substitute in anatomy (Khot et al., 2013; Preece et al., 2013; Saltarelli et al., 2014; Mathiowetz et al., 2016) , further suggesting that student performance and satisfaction may depreciate during the ERT phase. Other studies have reported that students experienced psychological distress after the onset of the Covid-19 pandemic (Cao et al., 2020; Li et al., 2020; Zhai & Du, 2020a) , affecting students' learning and performance (Vogel & Schwabe, 2016) .

To this end, the present study sought to address two central research questions: (1) how did students perceive this new teaching modality compared to the norm? and (2) how did the transition to ERT affect students' grades and study habits? The authors hypothesized (1) lower levels of satisfaction among students during the ERT phase when compared with the F2F phase and (2) lower examination grades during the ERT phase due to an increase of stress related to the transition to an online environment and isolation due to Covid-19 restrictions. Both of these negative expectations seem especially likely considering that the online learning experiences and environment during the ERT phase were undoubtedly different from those of a fully-formed, well-planned online anatomy course (Wright, 2012) .

The examined undergraduate course (Human anatomy observation laboratory) is a 200-level, two-credit-hour laboratory course that is separate from the lecture. It utilizes seven prosected human cadavers and follows a regional approach. The course is divided into five content units: (1) introduction, back, and central nervous system; (2) upper limb and pectoral girdle; (3) lower limb and pelvic girdle; (4) thorax and abdomen; and (5) head and neck. In addition to cadavers, which are dissected by junior and senior students as part of a 400-level biology course, the laboratory incorporates several ex situ prosections (i.e., prosected body parts removed from the cadavers) in all content units (Mattingly et al., 2021) . Learning is complemented with the use of plastic models and multimedia sources (for example, 75 labeled pictures and 43 video walkthroughs of content) presented through Blackboard Learn learning management system (Blackboard® Inc., Washington, DC).

To avoid confusion with the terms spring, summer, and fall between the Northern and Southern Hemispheres, and to facilitate comparisons with other institutions, the academic semesters in the spring (January-May), summer (June-July), and fall (August-December) are hereafter referred as to Semester I, II, and III, respectively. There is no semester IV (winter) offered for this course. Nine laboratory sections are offered during Semesters I and III, each consisting of a maximum of 25 students and meeting for two two-hour sessions per week. A single laboratory section is offered in Semester II. A graduate teaching assistant (GTA-either a master's or Ph.D. student) and three undergraduate teaching assistants (UTA) instruct each laboratory section. At the beginning of each class, the GTA provides brief instructions, and then students rotate among four or five teaching stations, which are each facilitated by one instructor (UTA or GTA). About four students work with each cadaver at a given time, and they spend approximately 20 minutes per teaching station. Throughout the semester (Semesters I or III), 23 regular laboratory sessions are scheduled, with five of them devoted to reviewing material prior to an examination using a mock test. In addition, 17 weekend review sessions are available to students, each two hours long, led by one GTA and three or four UTAs. Thus, students have access to the laboratory about 5.3 hours per week.

Student learning is assessed during each content unit through practical summative assessments, two quizzes and one examination, all in short-answer format and in the form of spotters (Inuwa et al., 2011) . Quizzes consist of 10 questions each and are worth 15% of the course grade, while examinations consist of 60 questions, are timed (approximately 54 seconds per question), and are worth 75% of the final grade. Quizzes and examinations include questions on both in situ and ex situ prosections, which students are allowed to handle, as well as on plastic models. In addition, students must complete 19 formative assessments (three or four per content unit) delivered through Blackboard. These assessments are due on the day of each unit examination and together are worth 5% of the course grade. Each formative assessment consists of 10 multiplechoice questions, which students can take multiple times, with only the highest grade used in the final grade calculation. Participation in the laboratory accounts for the remaining 5% of the final grade.

Students use a laboratory manual developed by the director of the course (VHG). Other information regarding this course is available in Sparacino et al. (2018) .

Student perceptions and satisfactions on learning outcomes during the online transition were assessed by administering a post-course survey. The survey intended to document students' perceived changes in knowledge, confidence, satisfaction, and preparedness for their future career because of ERT. This survey was designed by the authors, created using Qualtrics© (Qualtrics, Provo, UT), pretested by two UTAs, and distributed via email to all students enrolled in the course. The survey was available from April 30 to May 7, 2020, and students were offered extra credit for its completion. It consisted of 26 questions that covered demographic aspects, student study practices during both F2F and ERT phases, and students' attitudes about the course change (Appendix 1). Using multiple choice, multiple answer, and short answers, questions 1-8 gathered demographic data and students' experience taking the course as well as online courses. Questions 9-15 assessed students' study practices on both F2F and ERT phases through multiple choice and multiple answer questions.

Using a seven-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = somewhat disagree, 4 = neither agree nor disagree, 5 = somewhat agree, 6 = agree, and 7 = strongly agree), questions 16-24 assessed students' overall levels of confidence in their ability to learn anatomy, the effectiveness of the learning materials, and their satisfaction with the course before and after the online transition. The last two questions, 25 and 26, were open-ended and sought to understand students' challenges and benefits of the ERT. For the sake of clarity, the survey questions referred to the F2F and ERT phases as "pre-" and "post-Covid-19" or as "inperson" and "online".

The survey received 219 responses, 14 of which were incomplete and discarded. An additional 22 (10%) duplicate responses were received, wherein the same student completed the survey more than once. This was due to a software-user error, in which the survey was emailed to the students on three separate occasions, inadvertently allowing them to complete the survey three times. However, these duplicates were identified and removed from the total responses.

Hence, 183 out of 197 students enrolled in the course completed the survey.

To explore changes in students' performance due to the ERT transition, the grades from the summative examinations of each content unit were compared during the F2F phase (examinations 1 and 2) and ERT phase (examinations 3-5). In addition, the grades from these examinations were compared with those of Semesters I-III of 2018 and 2019, as well as all semesters after the Covid-19 disruption (Semesters II and III of 2020, and Semester I of 2021). Other metrics for student performance, namely the ten in-class quizzes, online homework, and participation were excluded because they were either unsuitable factors for analysis or remained stagnant throughout the transition of the course (e.g., online homework material).

Unless indicated otherwise, graphical and statistical analyses were performed using R computing software, version 1.3.1093 (R To identify the potential main challenges and benefits of the ERT experience, as well as their relative importance among students, responses from open-ended Questions 25 and 26 were analyzed using qualitative content analysis. Qualitative data, i.e., student written responses, were first classified based on characteristic themes (Kuckartz, 2019) . Then, the responses from each student were reanalyzed, coded, and categorized in accordance with these themes and listed in the same order as they were mentioned. This process was done manually given the small sample size of this study. The dataset was then analyzed using Anthropac software, version 4.97, (Analytic Technologies, Lexington, KY) to calculate a Smith's Salience Index (S), which ranks responses based on the frequency and order of mentioned (Borgatti, 1996) . Resulting S values are bounded (between 0 and 1), with higher values indicating higher consensus and importance among participant's responses. To visualize these responses, a word cloud was created using the R Word Cloud statistical package, version 2.5, (R Foundation for Statistical Computing, Vienna, Austria).

To assess for differences among examination grades during Semester I of 2020, a Generalized Linear Mixed Effect Model with normal distribution was used. Examination served as a fixed factor and laboratory section and student as random factors. A Generalized Linear Model with normal distribution was used to assess the effect of content unit examination, semester, and year on the grades. In this model, examination, semester, and year were used as factors.

These models were implemented using the lm4 and glmmTMB packages in R (Bates et al., 2015; Brooks et al., 2017) and assessed the significance of fixed effects using a Type II Wald χ 2 test with the car package (Fox & Weisberg, 2019) . The lsmeans package (Lenht, 2016) was used to conduct multiple pairwise comparisons with Bonferroni adjustment. Finally, a goodness-of-fit test was used to compare the weekly number of hours studying between the F2F and ERT phases and the proportion of students within each final course letter grade between the spring 2020 semester and the previous six semesters combined. Average values are given with standard error and sample size.

Nearly half of the 197 students enrolled in the course were sophomores in the pre-nursing program. Exercise science, applied behavioral science, and biology were the next most-prevalent majors. The vast majority of students were females (83% female; 16% male; 1% non-binary) between 18 and 26 years of age. Most (90%) students self-identified as white, non-Hispanic or Latino. Similarly, a greater number of students (91%) were enrolled in the course for the first time, and many (74%) had various degrees of experience with online courses (Table 1) .

Beginning the week of March 23 (week 10 of Semester I of 2020), following a one-week suspension due to Covid-19, all in-person courses at the University of Kansas transitioned to an ERT format for social-distancing purposes. This entailed shifting all current instructional materials to Blackboard. Fortunately, some resources, such as images, videos, Microsoft PowerPoint ® presentations, version 2105 (Microsoft Corp., WA), and online practice quizzes, were already available on Blackboard prior to the start of the Covid-19 pandemic, thereby reducing the difficulty of the transition. To maintain the "near peer" teaching model (Evans & Cuffe, 2009 ), the UTA's created an additional 45 instructional videos (each ranging from 1:43-13:26 minutes long) pertaining to Units 3, 4, and 5 that were either tested only (Unit 3) or entirely covered (Units 4 and 5) during the ERT phase. These videos were created with the Kaltura capture application (Kaltura Inc., New York, USA) and were subsequently posted to Blackboard. Videos included previously developed photographs and illustrations, as well as images from commercial software used in other courses (Anatomy and Physiology 

Blackboard. Due to the timing of the shift to ERT, the content delivery and assessment for Unit 3 was a hybrid; most of the material for this unit was taught F2F (utilizing cadavers), but the summative assessments were conducted online. Overall, the number and type of summative assessments remained the same during the ERT phase (two quizzes and one examination per unit). However, the questions therein differed in both type (identification or conceptual) and format (short answer, multiple choice, etc.) from those contained in the F2F assessments. This primarily involved the addition of multiplechoice questions during online assessments (Table 2) . Because the students learned the Unit 3 content mostly during the F2F phase, the online examination for this unit consisted only of short answer questions, thereby resembling the traditional practical spot test of the course. In contrast, examinations for Units 4 and 5 contained a mixture of multiple choice, multiple answer, ordering, and true or false questions. The unit quizzes were merged into one larger quiz, which also served as a mock examination. Each mock examination consisted of 40 questions and students were allowed multiple attempts, except for that of Unit 3 in which students were allowed only two attempts. In all mock examinations, students had a time limit of two hours (3 minutes per question). The formative assessments, which were already online through Blackboard, remained the same. Each examination was available to the students for two to three days and they had three hours to complete it (3 minutes per question). Two attempts were permitted, with only the highest grade recorded. To discourage academic dishonesty, questions were taken randomly from question pools, and these presented to students one at a time and in different order. The extended time per question in comparison to the F2F examination was chosen to minimize internet issues, as some students reported low internet connections that prevented them to see images.

TA B L E 1 Characteristics of students enrolled in the undergraduate human anatomy laboratory course at The University of Kansas

Frequency a n (%) 

All survey items using Likert scales were significantly correlated (P < 

The transition to the online format significantly shifted the time invested by the students learning the course material (Chi-square test, χ 2 = 1187.3, df = 4, P < 0.001). Most students spent more time studying for the summative examinations during the F2F phase than during the ERT phase. For example, many students reported that during the F2F phase they invested more than six hours each week to prepare for the class, whereas during the ERT phase most students spent fewer than six hours (Figure 1 ).

The students' reported reliance on class materials and other resources was largely similar between the F2F and ERT phases (Table 3) , though some resources were not available to students during both phases. For example, online discussion sessions were only offered to students during the ERT phase, and bone models were only available to students during the F2F phase. The only additional study material that either did not transition over or was not offered during the F2F was Zoom office hours, though these were rarely used by students (most students (87.4%) never attended these optional, synchronous discussion sessions). Some study habits, such as supplemental instruction sessions and study groups, did not transition well, with high attendance during the F2F phase, but far less use during ERT phase (Table 3) . Total number of questions created in the question pools. c Questions related to structures that students are required to identify. d Questions that students are required to provide an explanation for, and not necessarily to visually identify. For example, functions, origins and insertions of muscles, and knowledge on blood flow through the heart. classmates and instructors during the F2F format helped them to learn better anatomy (Q18), while very few disagreed or strongly disagreed ( Figure 2; Appendix 3) .

When asked directly which tools students used to compensate for the lack of cadavers during the ERT phase (Q15), a large number of students (n = 93), determined that no tools could adequately compensate for no longer having access to the cadavers.

Students equally responded that YouTube (n = 79) and pre-existing Blackboard materials (n = 102) compensated for this. The remaining category 'other' yielded 25 responses, and included resources such as: Biodigital®, online resources (e.g., pictures of cadavers), other anatomy textbooks, Quizlet (Quizlet.com), anatomy simulation phone applications, and Anatomy and Physiology Revealed. Some students included written responses: 'I feel like I am not learning even close to as much', '… Nothing compared to the learning that occurred in lab', and 'there are not many tools that adequately compensate …'

Most students perceived a greater learning confidence (Q19, 20), better knowledge gained (Q21) and felt better prepared for their future career (Q23) with the knowledge gained during the F2F phase than during the ERT. Similarly, students were more satisfied with learning anatomy during the F2F than ERT (Q22). By contrast, responses were mixed regarding whether student grades obtained during the F2F portion better reflected their true anatomical knowledge (Q24) (Figure 2 ; Appendix 3).

When asked which skillsets students had learned that they otherwise would not have because of ERT, most students (n = 152) reported Zoom as the most prominent learned skillset followed by other online resources (Figure 3 ). Students also listed better time management, self-motivation, and scheduling self 'class' time as valuable skills learned due to ERT.

The thematic characterization of students' responses from the open-ended questions regarding the challenges and benefits of ERT, resulted in 14 themes for challenges and 11 for benefits (Table 4, Figure 4 ). According to the Salience index, the most important challenges were those related to the loss of access to cadaver-based learning. Namely, the difficulty to mentally translate and visualize F I G U R E 1 Self-reported time invested by undergraduate anatomy students learning the course material during the faceto-face (F2F) and emergency remote teaching (ERT) phases. Data pooled across nine laboratory sections (n = 183) Among the most important perceived benefits, students recognized the flexibility in taking examinations, the ability to learn how to manage their own time, and the ability to develop their own learning strategies (Table 4, Figure 4 ). For example, one student commented: "Learning at my own pace, having better time management, learning at different times, being able to take a break in between some of the content and come back to it later when it gets too overwhelming".

During Semester I of 2020, grades from summative examinations differed before and after the disruption (χ 2 = 111,721, df = 4, P < 

Few studies have yet reported the specific curricular changes that occurred in anatomy courses during the Covid-19 disruption, but

the few examples indicate that a wide range of approaches were adopted. In the undergraduate course examined, content delivery during ERT included both asynchronous pre-recorded concept videos, discussion boards, and synchronous video discussion sessions.

Summative assessments consisted of online spotter examinations with a mixture of short answers, multiple choice, ordering, and true and false questions. Thus, this course falls within the wide range of mitigation strategies adopted by anatomy instructors and institutions after the onset of the pandemic, which ranged from entirely synchronous to asynchronous sessions, and from multiple choice assessments to open-book examinations with pass-fail grading (Longhurst et al., 2020; Pather et al., 2020) . In the course examined, several pre-existing videos available to students during the F2F course also facilitated ERT. Given the large size of the course (197 students distributed among 9 laboratory sections), these preexisting videos, as well as other newly created videos, were the basis for the asynchronous delivery of content. Synchronous discussion sessions were optional and poorly attended.

Student assessment has been recognized as one of the most challenging aspects to address during the ERT shift and multiple formats were incorporated during this transition (Evans et al., 2020) .

The variation in assessment format in the examined course was the result of differences in its content delivery. For example, the Unit 3 examination, despite being administered online, attempted to replicate the spotter test (short answer questions), because students learned the Unit 3 material during the F2F portion of the course.

However, the examinations for the remaining two units, which students learned entirely during the ERT phase, consisted mostly of multiple-choice questions, with a small number of other question F I G U R E 4 Word cloud displaying (A) self-perceived challenges; and (B) benefits of the emergency remote teaching by undergraduate anatomy students F I G U R E 5 Academic performance of students enrolled in an undergraduate human anatomy course at The University of Kansas during face-to-face (Units 1 and 2) and emergency remote teaching phases (remaining Units) in the Semester I of 2020 types included for variation (Table 2 ). In all cases, questions were taken from a pool and students were allowed to practice them in a "mock examination", as is customary in the F2F course. Although multiple choice questions may not be the most appropriate way to assess students' anatomical learning and understanding (Evans et al., 2020) , this temporary solution was adopted by many institutions during the ERT (Longhurst et al., 2020; Pather et al., 2020) , thus highlighting the utility of this format during contingency measurements in moments of crises.

Results from the survey indicate that students were generally dissatisfied with the quality of instruction during the ERT phase of the course, as more than 80% of respondents in some form of agreement indicated that they were more satisfied with the F2F phase than the ERT phase ( Figure 2 ; average Likert scale 5.95-6.27; see

Appendix 3). Thus, these results support the hypothesis that students would experience lower levels of satisfaction during the ERT when compared to the F2F phase. While this pattern is consistent with previous studies that have indicated student preferences of cadaver-based learning (or other physical, F2F methods) in anatomy over computer-based methods (Azer & Eizenberg, 2007; Petersson et al., 2009; Kerby et al., 2011; Davis et al., 2014; Ogard, 2014; Attardi et al., 2016; Cuschieri & Calleja Agius, 2020; Banovac et al., 2021) , the near unanimity of its consensus is somewhat unexpected.

This could be due to the students' self-perceived learning, the results of which nearly parallel those of overall satisfaction. Despite increases in student grades, roughly three out of every four students reported less confidence in their ability to learn, less knowledge gained, and less preparedness for the future during the ERT duration ( Figure 2 ). These results agree with previous studies demonstrating that computer-based learning is not as efficient as traditional methods to learn anatomy (Khot et al., 2013; Preece et al., 2013; Saltarelli et al., 2014; Mathiowetz et al., 2016) . They also reinforce the notion that cadaver laboratories offer a "hidden curriculum", wherein students learn aspects of professionalism and ethics (i.e., "preparedness for the future") through their interactions with cadavers (Bamber et al., 2014; Sándor et al., 2015; Karunakaran et al., 2017; Evans et al., 2018; Kumar Gosh & Kumar, 2019) . These principles are likely not as easily transmitted through computer-based learning, which the students of this course may have inadvertently alluded to in their responses.

Even so, there were some perceived benefits of the shift to ERT among students. Most notably, they responded that increased flexibility in studying and taking examinations was the greatest advantage of ERT. A reduction in time constraints in computer-based learning has been reported previously regarding online anatomy education (Attardi et al., 2016; Mathiowetz et al., 2016) , though some instructors faced the opposite scenario of increased time concerns due to Covid-19 (Longhurst et al., 2020) . Students in this course reported other advantages of ERT like time management skills, learning at their own pace, and more study time, all of which further emphasize the perceived benefits of reducing time constraints on students. Interestingly, one of the most common themes to emerge from studies detailing anatomy students' perceptions of ERT is that a plurality of students worldwide almost invariably prefer the use of pre-recorded lecture videos as an instructional resource above any other method (Cuschieri & Calleja Agius, 2020; Gupta & Pandey, 2020; Kim et al., 2020; Banovac et al., 2021; Yoo et al., 2021) . This seems to apply mainly to didactic components of anatomy courses, however, and does not necessarily extend to cadaver-based dissection (Banovac et al., 2021) .

The most frequent challenge cited by students during ERT was difficulty with mentally visualizing the cadavers. Previous studies have reported similar complaints among students (Attardi et al., 2016; Srinivasan, 2020) , and a key focus of online anatomy education is the 3D visualization of anatomical structures (Estai & Bunt, 2016) . Regrettably, students in this course did not have access to advanced 3D visualization software (other than those which are freely available online) due to the abruptness of the online transition, and thus may have struggled more with cadaver visualization as a result. Another intriguing challenge cited by students during ERT was a feeling of isolation, and lack of access to an instructor/ TA, the latter of which is commonly reported (Attardi et al., 2016; Longhurst et al., 2020) . These responses were given even though instructors were available through synchronous discussion sessions, as well as by email and through specially requested appointments. Further, dividing students into smaller chatrooms ("channels") within these media, each with an assigned instructor/TA, can also promote feelings of connectedness and engagement. Some of the suggestions listed above were implemented in the examined course during the remaining semesters of 2020 and Semester I of 2021. Although the impact of these changes to the course on the students' engagement were not measured, some experiences highlight the benefits of using such tools in online education (Mejia, 2020; Poston et al., 2020) .

Examination grades improved during ERT, resulting in high final course grades relative to previous semesters ( Figure 5 ; Appendices 6-9). Thus, the hypothesis that lower examination grades were expected during the ERT phase was not supported. This increase is likely due to at least four reasons: first, the grades from the examinations of Units 1 and 2 were lower than in previous semesters, which might have resulted from a new grading practice. Historically, GTA's graded examinations manually, but beginning Semester I of 2020, the first two examinations were graded using Gradescope® (Turnitin LLC., Berkeley, CA), a Web App software that uses artificial intelligence and allows for more consistent, faster, and less biased grading (Singh et al., 2017) . Unlike manual grading, Gradescope grades one question for all students across sections at once and thus it reduces grading bias among laboratory sections, in addition to grading fatigue (Singh et al., 2017) . Second, students were able to take the examinations twice and had three minutes per question during the ERT phase. In contrast, students had only one opportunity and less than a minute per question during F2F. Third, examinations during the ERT phase included primarily multiple-choice questions, except for that of Unit 3 that consisted of short answers only. These results agree with some studies demonstrating that students obtain higher grades in multiple choice tests than in short answer tests (Mujeeb et al., 2010; Pepple et al., 2010) , although this might depend on the students' college level (Adamu et al., 2018) . Fourth, because of the sudden change to an online format, examinations could not be proctored using a remote proctoring software, as would have been the case for a well-planned online course. Although examinations were timed and questions were taken at random from question pools each time to reduce cheating during ERT, these measures may not have been effective in completely removing student use of online resources or the course laboratory manual during examinations. It is interesting to note that when the Proctorio software was implemented in Semester I of 2021, the examination grades were significantly lower than those observed during the ERT phase and Semester III of 2020, and thus more like grades before the Covid-19

disruption. Although the impact of implementing Proctorio was not addressed in the course, there is evidence indicating that this software deters students from cheating and leads to grades from online examinations like those taken in person (Lewis, 2020) .

Although students' examination performance appears to be similar between an anatomy course delivered online and F2F (Inuwa et al., 2012) , available information indicates that course grades significantly increased during the ERT in a wide range of classes, at least in some Spanish academic institutions (Gonzalez et al., 2020; Iglesias-Pradas et al., 2021) . However, it is unclear whether this is a result from improvements in students' learning strategies due to confinement, structural changes made to examination type, or possible misconduct. Interestingly, the students themselves were unsure of the influence of F2F compared with ERT on the changes witnessed in their examination grades. The responses to the survey question 24 ("my exam grades better reflected my true knowledge of anatomy in the in-person, cadaver-based portion of the class …";

Figure 2) were mixed and the average response score for that question was 4.8 ± 1.77 (Neither agree nor disagree to somewhat agree), thus indicating students' uncertainty regarding the accuracy of their grades. Undoubtedly, further studies are needed to determine if grade improvement was a general pattern during the disruption, and whether this pattern correlates to students' self-perceived learning.

This study documents the curricular modifications of a single undergraduate anatomy course during the Covid-19 pandemic, assesses student perceptions and performance during the transitional F2Fto-ERT period, and further documents performance over sustained course-history. Undergraduate anatomy courses vary significantly in their approach, teaching materials and personnel, resources, and use of cadavers (Sparacino et al., 2018) , therefore it is likely that students' perceptions and performances may also vary. The variability in teaching practices of anatomy and other types of education is best given due consideration when making comparisons with the students' performance and perceptions in this undergraduate course.

It should also be noted that student attitudes and perceptions were asked during a time of great uncertainty, when Covid-19 was unresolved and remained a constant threat to personal and professional lives. There may be large differences in what people reported during this time, and what they perceive months or years later, after receiving their final grade, moving forward in their career path, and persisting through the continuation of the Covid-19 pandemic. Thus, the current views of these participants should not be the sole factor of long-term or extrapolated decision making. These results should instead be taken as a contribution to the evidence upon which decisions regarding education and best learning practices are made.

Finally, the order and wording of some questions using Likert scales might skew participant responses (McFarland, 1981; Malhotra, 2008; Garbarski et al., 2015) . However, the survey had two questions (19 and 20) with reverse wording (Appendix 1) to reduce the effects of response styles or tendency of respondents to answer items regardless of their content. In addition, Kendall's Tau b correlation coefficients were significant (P < 0.01) among survey items using Likert scales and thus indicating acceptable internal reliability among the survey items (Appendix 2).

This study revealed that, despite examination grades improving during ERT, students reported lower levels of learning, confidence, and engagement with the course materials. Grades remained abnormally high during the subsequent online-only semesters (Semesters II and III of 2020), but reverted toward the norm following the implementation of Proctorio during Semester I of 2021. Like other instructors and institutions challenged to promptly change their in-person classrooms to online, the teaching and examination methods of this undergraduate anatomy course fell within the range of approaches adopted by other instructors and institutions worldwide. Despite earning higher grades, students highlighted several learning challenges during ERT resulting from the loss of cadaver-based anatomy education. These included the loss of hands-on, tactile, and sensorial training, in addition to the social and teamwork interactions students develop through peer-to-peer and instructor-to-peer mentorship in an interactive laboratory setting. To circumvent challenges faced in future iterations of ERT, and in online anatomy courses more generally, the authors suggest mandatory synchronous video sessions, peer-to-peer interaction and feedback, and the use of accessible, informal means of communication, such as Slack or Microsoft Teams. These findings provide context for the expectations of students and instructors experiencing an academic learning environment challenged with the effects of a pandemic, economic uncertainty, and future uncharted disruptions.

This study helps to prepare students and instructors to develop resilient teaching and learning practices and create effective classrooms whether they be in-person, online, or both.

The authors wish to thank Kensi Peitz for assistance with data entry, Amy Comfort, Dyan Morgan, and anonymous reviewers and editors for comments and suggestions that significantly improved this manuscript. The National Science Foundation's REU program (DBI 1560389) provided support to V.H.G.

The authors declare no conflict of interest.

The University of Kansas' Human Research Protection Program and Internal Review Board (IRB) reviewed and approved this study on April 27, 2020 (IRB #00145687). All study participants were asked for consent prior to collection of data and were informed that the survey was anonymous and voluntary. Respondents were informed that the survey results and data were to be used for research purposes. To ensure privacy, no IP addresses were linked with completed surveys, thus the results were analyzed anonymously.

Victor H. Gonzalez https://orcid.org/0000-0002-4146-1634

Differences in multiple choice and short answer question performance among medical and dental trainees as reflected in their anatomy course

COVID-19 and medical education

Mixed methods student evaluation of an online systemic human anatomy course with laboratory

Do we need dissection in an integrated problem based learning medical course? Perceptions of first and second-year students

Psychological and behavioral response to the coronavirus (COVID-19) pandemic

Medical student attitudes to the autopsy and its utility in medical education: A brief qualitative study at one UK medical school

The anatomy lesson of the SARS-CoV-2 pandemic: Irreplaceable tradition (cadaver work) and new didactics of digital technology

Fitting linear mixedeffects models using lme4

Anthropac software, version 4.97. Analytic Technologies

Resuming cadaver dissection during a pandemic

Schools, skills, and learning: The impact of COVID-19 on education. VoxEU-CEPR's Policy Portal

glmmTMB balances speed and flexibility among packages for zero-inflated generalized linear mixed modeling

The psychological impact of the COVID-19 epidemic on college students in China

Gross anatomy education in China during the Covid-19 pandemic: A national survey

Spotlight on the shift to remote anatomical teaching during Covid-19 pandemic: Perspectives and experiences from the University of Malta

Human anatomy: Let the students tell us how to teach

Use of knowledge-sharing web-based portal in gross and microscopic anatomy

Teaching in the dissecting room

Best teaching practices in anatomy education: A critical review

Going virtual to support anatomy education: A STOPGAP in the midst of the Covid-19 pandemic

Near peer teaching in anatomy: An approach for deeper learning

Human skills for human[istic] anatomy: An emphasis on nontraditional discipline-independent skills

The impact of COVID-19 on medical education

An R Companion to Applied Regression

The impact of the Covid-19 pandemic on current anatomy education and future careers: A student's perspective

The effects of response option order and question order on self-rated health

Five tips for moving teaching online as COVID-19 takes hold

Influence of Covid-19 confinement in students' performance in higher education

Participation in asynchronous online discussion forums does improve student learning of gross anatomy

Student outcomes associated with use of asynchronous online discussion forums in gross anatomy teaching

Disruption of anatomy dissection practical in COVID-19 pandemic: Challenges, problems and solutions

An analysis of anatomy education before and during Covid

Rapid evolution of the COVID-19 pandemic calls for a unified public health response

The difference between emergency remote teaching and online learning

Emergency remote teaching and students' academic performance in higher education during the COVID-19 pandemic: A case study

Anatomy "steeplachase" online: Necessity sometimes is the catalyst for innovation

Anatomy practical examinations: How does student performance of computerized evaluation compare with the traditional format?

Impact of COVID-19 on dental education in the United States

YouTube: An emerging tool in anatomy education

A preliminary survey of professionalism teaching practices in anatomy education among Indian medical colleges

The relationships between learning outcomes and methods of teaching anatomy as perceived by medical students

The relative effectiveness of computer-based and traditional resources for education in anatomy

How medical education survives and evolves during Covid-19: Our experiences and future direction

Compendium for Early Career Researchers in Mathematics Education

Building professionalism in human dissection room as a component of hidden curriculum delivery: A systematic review of good practice

Human anatomy education and management of anatomic specimens during and after COVID-19 pandemic: Ethical, legal and biosafety aspects

Least-squares means: The R package lsmeans

Chemistry assessments through the sudden implementation of online instruction

Psychological distress among health professional students during the COVID-19 outbreak

Strength, weakness, opportunity, threat (SWOT) analysis of the adaptations to anatomical education in the United Kingdom and Republic of Ireland in response to the Covid-19 Pandemic

Completion time and response order effects in web surveys

Comparison of a gross anatomy laboratory to online anatomy software for teaching anatomy

Differences between attached and detached cadaveric prosections on students' identification ability during practical examinations

Using voicethread as a discussion platform to enhance student engagement in a hospitality management online course

Effects of question order on survey responses

Evaluation of computeraided instruction in a gross anatomy course: A six-year study

COVID-19: An insight into SARS-CoV-2 pandemic originated at Wuhan City in Hubei Province of China

Comparative assessment of multiple choice questions versus short essay questions in pharmacology examinations

Knowledge, attitudes, and practices against the growing threat of COVID-19 among medical students of Pakistan

Outcomes related to a multimodal human anatomy course with decreased cadaver dissection in a doctor of physical therapy curriculum

Challenges in Teaching human anatomy to students with intellectual disabilities during the Covid-19 pandemic

Using Microsoft Teams to enhance engagement and learning with any class: It's fun and easy

Impact of SARS-CoV-2 virus pandemic on the future of cadaveric dissection anatomical teaching

Attitudes of professional anatomists to curricular change

Forced disruption of anatomy education in Australia and New Zealand: An acute response to the Covid-19 pandemic

A comparison of student performance in multiple-choice and long essay questions in the MBBS state I physiology examination at the University of the West Indies (Mona Campus)

Web-based interactive 3D visualization as a tool for improved anatomy learning

Let's get physical": Advantages of a physical model over 3D computer models and textbooks in learning imaging anatomy

Stigmatization and prejudice during the COVID-19 pandemic

Dead body management in times of Covid-19 and its potential impact on the availability of cadavers for medical education in India

Design, Implementation, and Evaluation of an Innovative Anatomy Course

Should we continue teaching anatomy by dissection when …?

Medical student education in the time of COVID-19

Human cadavers vs. multimedia simulation: A study of student learning in anatomy

The effects of dissection-room experiences and related coping strategies among Hungarian medical students

Anatomy & Physiology Revealed 3.0: An Interactive Cadaver Dissection Experience

Teaching and Learning at a Distance: Foundations of Distance Education

Cadaverless anatomy: Darkness in the times of pandemic Covid-19

Gradescope: A fast, flexible, and fair system for scalable assessment of handwritten work

Medical student mobilization during a crisis: Lessons from a COVID-19 medical student response team

A quantitative analysis of four undergraduate human anatomy laboratory curricula: Approaches, identified structures, concepts, and thematic emphases

Medical students' perceptions and an anatomy teacher's personal experience using an e-learning platform for tutorials during the Covid-19 crisis

The anatomy of anatomy: A review for its modernization

Evaluation of a computer program ('disect') to consolidate anatomy knowledge: A randomised-controlled trial

Gross anatomy videos: Student satisfaction, usage, and effect on student performance in a condensed curriculum

Anatomical informatics: Millennial perspectives on a newer frontier

From chalkboard, slides, and paper to e-learning: How computing technologies have transformed anatomical sciences education

United Nations Educational, Scientific and Cultural Organization. COVID-19 educational disruption and response

Learning and memory under stress: Implications for the classroom

WHO. 2020. World Health Organization. Coronavirus disease 2019 (COVID-19): Situation report -72

Student perceptions of an upper-level, undergraduate human anatomy laboratory course without cadavers

Adaptations in anatomy education during COVID-19

Addressing collegiate mental health amid COVID-19 pandemic

Mental health care for international Chinese students affected by the COVID-19 outbreak