key: cord-0768022-nakr9vsi
authors: Vielma, Karina; Brey, Eric M.
title: Using Evaluative Data to Assess Virtual Learning Experiences for Students During COVID-19
date: 2020-09-02
journal: Biomed Eng Education
DOI: 10.1007/s43683-020-00027-8
sha: ef84eb056e1979132234cbd20a6e3050b1487106
doc_id: 768022
cord_uid: nakr9vsi

nan

Our diverse, Hispanic-serving institution (HSI) consists of approximately 64% underrepresented minorities, 45% first generation college students, 14% military affiliated students, and 39% transfer students. During a typical semester, nontraditional students may face demanding challenges outside of their academic responsibilities including full time employment, long commutes, and family obligations. During an emergency or disruptive situation, such as the COVID-19 pandemic, the effects of these challenges may be magnified. Faculty members transferring curriculum online in a short period of time need to understand the best mechanisms for supporting student learning and well-being.

During the 2020 Spring Break week, the university made the decision to migrate courses online and encourage students to move off campus, if possible. Students, as well as university faculty, were informed that an extra week without instruction would be allotted for faculty to prepare their courses for online instruction. While faculty were provided support and training for online course delivery, individual faculty members were allowed significant flexibility in determining their online course structure. For example, asynchronous course delivery was encouraged but not required. For the transition, faculty also had access to a variety of conferencing software for course instruction, but one uniform platform was not required. Selecting Pass/Fail grading was available for a course only if both the faculty member and the student selected this option.

These flexible options resulted in a broad range of course structures over approximately two months of online content delivery. While our Department organized multiple sessions for online learning strategies and best practices, faculty were able to choose their preferred structures. As a result of these varied options, the Department sought to evaluate student perceptions of online course delivery, identify challenges faced during the transition to online learning, and determine student preferences of best practices for remote learning at an HSI with a large, majority-minority, first-generation student population.

A survey was developed to collect information from students in the Department, which houses two engineering programs-Biomedical Engineering and Chemical Engineering. The process was reviewed and granted an exemption from the university Institutional Review Board, and the anonymous survey was implemented after completion of the semester. The primary goal of the survey was to identify student perceptions related to the effectiveness of educational activities during the online course delivery. One hundred and seventy students completed surveys (49% Departmental response rate). The students were asked to evaluate only the courses they enrolled in resulting in 352 total course evaluations (an average of 2.1 courses evaluated per student). Twenty-one (21) Biomedical Engineering courses and ten (10) Chemical Engineering courses were included in the survey. Students' responses to the following four open-ended questions were analyzed inductively, and emergent themes were defined and reviewed. 1, 2 The themes and number of  responses per theme are shown in Tables 1, 2, 3, and 4 .

What was the biggest challenge you faced during the online course delivery?What aspect of the online course content did you feel was the most effective?What aspect of the online course content did you feel was the least effective?How could our department and faculty improve online course delivery if required in the future?

The data in Tables 1, 2, 3, and 4 present the most common themes. Themes with the number of student responses ‡ 5 are presented in the Tables.

In general, students responding to the qualitative questions provided constructive feedback on the online teaching environment and the challenges encountered. Specific themes that were consistent across the survey responses are discussed below.

In an attempt to recognize the challenges that students faced when moving back home, the university encouraged asynchronous delivery of course lectures. Students expressed an appreciation for recorded content and the flexibility it provided, particularly the ability to re-watch videos as needed as a means to improve comprehension. However, a significant number of students discussed the additional need for synchronous content. The synchronous content brought some feeling of normality to the classes and allowed for questions and faculty interactions-contributing to the critical social component of learning. 3, 4 One student expressed the importance of the time relating to both their own questions but also learning from other students:

Sometimes I would be fuzzy about something but either did not have a direct question to ask or it was difficult to formulate a question and be understood. During those times, it would have been helpful to hear other student's questions and comments.

And another stated:

Live lectures keep us more motivated to focus. Students expressed a particular appreciation for courses that had pre-recorded lectures followed by an interactive component that focused on student questions, problem-solving, or more in-depth discussion. Interestingly, this style identified by the students is generally consistent with a ''flipped classroom'' approach, which takes the lectures to be watched prior to class meeting, and the classroom space is used for solving problems and asking ques-tions, providing more student-instructor interaction. 5 This learning environment helps students develop self-regulation as it pertains to their learning that helps them develop metacognition skills transferable to diverse problem-solving settings. 6, 7 Additionally, self-guided learning and the ability to return to recorded lectures as needed helps students reach understanding at their own pace, which gives high benefit to struggling students, rather than being 

Similar to having options for viewing lectures, students appreciated faculty flexibility and accommodations for completing assignments and exams. In general, students expressed concern with courses where the content available online had time restrictions or where assignment deadlines were strictly fixed. More broadly, students appreciated instructors who were accommodating as well as those that explicitly acknowledged the situation and expressed empathy.

Although not explicitly stated, students appeared to identify empathetic faculty members based on their flexibility, availability, and public display of acknowledgement of the challenging situation. Students praised faculty members who worked with them to address the complexities of the transition to online classes. They appreciated compassionate interactions with faculty when they felt their instructors authentically cared about their circumstances. 9,10 Faculty can exhibit empathy by ''listening'', thinking ahead of ways to provide support for their students, and acknowledging uncomfortable feelings and challenging situations. 11 Now, more than ever, the need to train ethical and responsive engineers becomes an imperative goal which can be modeled through an ethic of care. 12 Students requested more frequent interactions with faculty in general, including ''low-stakes'' assignments, virtual office hours, and reminders about deadlines. It appears that students saw the value of in-person classes as multifaceted. While online office hours were appreciated, students more often identified the positive impact of rapid response via email or readily available communications via online discussion groups (e.g. Slack).

Student comments also pointed to the overall challenges and burden of transitioning to distance learning, moving off campus, and the general disruption of the pandemic. Technically, students were challenged to learn to use multiple new software platforms. They also reported having difficulty communicating and receiving quick responses for help with these systems. A feeling of being ''overwhelmed'' after the transition to online learning was a common response. This may have been attributed to the stress of the pandemic and possibly a result of a type of grief over the unexpected loss of face-to-face academic interactions.

Many of the students were impacted by returning home. This move sometimes resulted in technical issues, including poor internet connectivity. Many lacked proper privacy in their family's home to study or participate in virtual lectures, at times requiring the shared use of a computer with multiple family members. In addition, being home meant some students needed to assist in the care for younger siblings or relatives as well as helping with household duties. Some also felt the financial strain of moving away from campus and losing critical employment opportunities.

Students also experienced a decrease in their motivation and focus to complete their classwork. When asked, ''What was the biggest challenge you faced during the online course delivery?'', one student commented:

Motivation. I'm the type of person that has to go to the library or any other place but my home to focus on studying.

Students' who identified perceived challenges linked to self-motivation were extracted and could be clustered into four main areas highlighted in Fig. 1 .

More than a third of students who responded that self-motivation was the biggest challenge faced perceived the home environment as not conducive to learning, including distracting and not ''academicfriendly'', lacking privacy, or demanding their attention. Another 28% of students commented that the online learning environment was related to their challenge with self-motivation, citing the lack of focus to view videos online, attend lectures, or spend the majority of their time on the computer. Twenty percent of students pointed to their own personal struggles with lack of concentration, optimism, commitment, and balance, including time-management skills during the sudden transition. A final theme identified as general academic captures students' perceived challenges completing academic activities such as group projects, assignments, and attending lectures.

In general, their responses indicated the need for greater socio-emotional support during an unexpected transition along with attention and acknowledgement about the challenges. The small actions proved empathetic and necessary for students to cope with such a marked change.

Perhaps one of the largest impacts on the student learning experience was the lack of access to advanced facilities for laboratories and senior design projects. Facilities include 3D printing equipment, materials characterization tools, advanced microscopy, and a pilot plant scale distillation column. Hands-on experiences with this equipment is important for training engineers and is not possible with at home lab experiments. Options for students included videos of experiments, research papers and study design exercises, and experimental data provided for analysis and interpretation. However, students identified the limitations of these approaches and lamented the loss of the hands-on experience. Students felt that this type of learning was best done in the physical space with the equipment designed to provide this experience.

A final common theme centered around the challenges related to group projects. Students found difficulties coordinating schedules, meeting online with group members, and collaborating with all members in the group. Interestingly, this seems like one of the more important skills that could be developed during online course delivery. Engineering work is increasingly global in scale. While this requires cultural understanding, it also requires an ability to collaborate effectively remotely. Students struggled to successfully collaborate in team settings through virtual platforms. Future developments could focus on empowering students to learn best practices for remote collaboration, including setting clear and shared expectations, collaboratively creating artifacts on a shared platform, and collectively and individually reflecting on their contributions. 13 This may require early guidance from an instructor or teaching assistant, but direction to promote successful group work through online methods was needed.

The transition to online teaching and learning was not expected and perhaps due to this, created unique challenges at a minority serving institution. Students and faculty managed multiple stressors during the transition to distance learning despite limited time for training and transitioning. While student input is not the only information to consider, much of what students identified is consistent with education research and learning theory. This information can provide insights into practices that may be used to enhance the online learning experiences in the future for engineering students. Some options to consider for virtual courses include, but are not limited to, the following:

(1) Multiple modes of learning Provide students with both asynchronous and synchronous modes of learning. Recorded live lectures should be housed online for students to readily access as needed after the session ends. Providing students with additional synchronous opportunities for participation helps students connect more easily as needed. These opportunities can come during synchronous lectures, online forums, small group remote meetings, or interactions with faculty during virtual office hours or online discussion groups. (2) Flexibility and empathy Faculty should listen, observe, and acknowledge the challenges that students may be facing in challenging circumstances. Students build trust when faculty show care and ''feel with [them]'' through respectful and empathetic interactions as well as when they are ''prepared to care'' for students. 14, 15 Providing flexible opportunities for contributing to the course aid in students' access to knowledge and show attention, thought, and care for the diverse experiences that students may face in an online learning environment. These options can be tailored to student's specific needs but should be communicated at the start of the semester/course, if possible. (3) Communication Set clear objectives, deadlines, and expectations for engagement from the beginning of the course. Whenever possible, use technology to automate course news and upcoming deadline reminders. Course management platforms also have mechanisms for sending personal emails to students identified for a specific characteristic i.e. missing an assignment or falling below a pre-determined course average. Students appreciate communication and transparency along with readily available responses to questions, but this may not always be fair or practical for faculty. Therefore, instructors should manage expectations by com-municating a timeline and mechanisms by which students will receive feedback. (4) Group Projects Group projects are critical to many aspects of engineering education. Faculty can be proactive in training students to effectively participate on virtual teams by leading student groups, for example, to use shared platforms where artifacts can be collected, viewed, and edited synchronously and asynchronously by all group members. Additionally, faculty can help groups identify project goals and milestones spread throughout the semester helping students reflect on their progress and gauge the project status at any given point in time. Therefore, the course should include instruction and guidance on developing working groups through online project management. 13 Student feedback is an important tool in evaluating our approach but not the only input of importance. Input should be received from all stakeholders (faculty, teaching assistants, administration, etc.), and the impact on all parties should be considered. It is critical that, as we reflect on student input, we recognize the potential risk for our faculty. Converting classes to online delivery, ensuring both recorded lectures and multiple learning resources are produced, communicating rapidly, reminding students of deadlines, and hosting small group meetings would be a challenge particularly if these elements need to be completed for all courses in a short period of time. One student addressed the incredible work of a faculty member, I don't know where he found the time to do all of these but he did and I am thankful for it.

It can take an incredible effort for faculty who, while committed to supporting students, also have other obligations. We must recognize these challenges, provide support for faculty to deliver the content, and assist with maintaining their well-being. Only then will we have the capacity to support our diverse students in their learning.

Fundamentals of qualitative research: understanding qualitative research

Naturalistic inquiry

Critical inquiry in a text-based environment: computer conferencing in higher education

Integrating the intangibles into asynchronous online instruction: strategies for improving interaction and social presence

The flipped classroom: the full picture

Best practices for flipping the college classroom

Investigating self-regulation and motivation: historical background, methodological developments, and future prospects

Flip your classroom: reach every student in every class every day

Authentic teachers: student criteria perceiving authenticity of teachers

Subtractive schooling: U.S.-Mexican youth and the politics of caring

Authenticity and empathy in education. Theory and Research in Education

Ethics, ethos' and the professions: some lessons from engineering

Online group work patterns: how to promote a successful collaboration

Complexity in caring and empathy

The caring relation in teaching

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations