key: cord-0850764-wcs8ol95 authors: Lengetti, Evelyn; Cantrell, Mary Ann; DellaCroce, Nicole; Diewald, Lisa; Mensinger, Janell L.; Shenkman, Rebecca title: Learning environment and evidence among professionals and students satisfaction (LEAPS), experienced during the COVID-19 pandemic date: 2021-07-27 journal: Teach Learn Nurs DOI: 10.1016/j.teln.2021.07.004 sha: fa2f07d2efd2857eb1d09308545e4a2f8d8dc6af doc_id: 850764 cord_uid: wcs8ol95 The COVID-19 pandemic required schools to transition courses to an online platform. This shift to Emergency Remote Teaching (ERT) created gaps in the literature about its impact on students. The purpose of this study was to test the relationship between learner and instructional attributes and learner satisfaction with ERT. A modified version of the Student Satisfaction Survey assessed learner and instructional attributes and learner satisfaction among a convenience sample of 12 graduate and 83 undergraduate nursing students. Open-ended questions assessed students’ responses to their satisfaction with ERT. Multiple regression analysis was used to test associations of learner and instructional attributes with student satisfaction. Overall satisfaction with ERT was neutral with a mean of 2.76 on a 1 to 5 scale; students rated instructional attributes higher with a mean of 3.64. Instructional engagement/technology use (single factor) and learner technology competence were associated with student satisfaction, beta = 0.93(0.09), p<.001; beta = 0.24(0.09), p = .008, respectively. Between-class technology use and prior experience with online courses were not associated with student satisfaction, beta = -0.08(0.09), p = .379, beta = 0.26(0.15), p = .079, respectively. Qualitative findings revealed faculty engagement was a major determinant in learner satisfaction with ERT. Supporting faculty competence for the use of technology may increase learner satisfaction with ERT. The COVID-19 pandemic and need for a nationwide quarantine mandated that higher education institutions implement an Emergency Remote Teaching (ERT) plan and transition courses taught in a traditional in-person learning environment to an online delivery. ERT is a temporary shift of instructional delivery to fully remote teaching, which differs substantially from courses initially designed and taught using a distance learning platform (Hodges et al., 2020) . The mandate to quarantine due to the COVID-19 pandemic was immediate making it very difficult for faculty to transition in-person courses to an online format. Faculty were forced to transition current courses to an online format using personal knowledge and experience for online teaching as well as request support from university experts of online teaching strategies. This sudden shift to ERT created gaps in the higher education literature as to the impact of ERT among learners' satisfaction enrolled in undergraduate and graduate curricula. Satisfaction and learning outcomes among all learners, especially the impact of demographic characteristics and instructional modalities, inclusive of faculty teaching skills, related to satisfaction. This shift in teaching modalities from in-person to online learning created a unique opportunity to measure learner satisfaction in a population that initially did not choose to participate in online learning. Students' perceptions of and satisfaction in a course can be impacted by many factors including instructor characteristics and student experience. Since student satisfaction is included as a program quality measure in nursing education program accreditation and online education (CCNE, 2018; Joosten et al., 2021) , these predictors of satisfaction were evaluated. The purpose of this study was to explore undergraduate and graduate nursing students' satisfaction with ERT implemented during the COVID-19 pandemic as measured by a modified version of the Student Satisfaction Survey (SSS) (DeBrough, 2003 Teaching and Learning in Nursing journal homepage: www.journals.elsevier.com/ teaching-and-learning-in-nursing A review of the literature that specifically examined learner satisfaction with ERT during COVID-19 yielded eight studies. Examination of learner satisfaction topics with the ERT during COVID-19 included learners' overall satisfaction, as well as satisfaction with online learning platforms, level of instructor interaction, and quality of course content. Most of the published higher education literature revealed positive learner satisfaction with ERT during COVID-19. Alqurshi (2020) examined 700 pharmacy students and instructors' overall satisfaction with ERT during COVID-19 across 18 colleges in Saudi Arabia and stated that half of the colleges reported scores ranging between 3 and 5 (very satisfied) on a 5-point Likert-type scale. A large-scale global study of 30,382 higher education students explored how student satisfaction was impacted with the COVID-19 pandemic within different aspects of their lives, including the academic learning environment (Aristovnik et al., 2020) . Overall, 57.6% of students were satisfied or very satisfied with the teaching staff during the COVID-19 pandemic (Aristovnik et al., 2020) . Likewise, the findings from a sample of 83 Omani students reported that 77% were satisfied with instructor interaction, and 84% reported satisfaction with the quality of materials and methods of ERT (Osman, 2020) . A study conducted by Srinivasan (2020) examined the opinions of 16 university anatomy students in Singapore regarding the use of the ZOOM platform during the switch to online learning during COVID-19 and reported that most students were satisfied (87.5%) with their ERT learning session in ZOOM, with a mean of 4.0 § SD 0.78. Openended responses showed a desire for more interaction using methods such as Poll Everywhere (PollEv) for instructor-student connection and implementing real-time quizzes to assess immediate learning. The one exception to these positive findings was those from a national survey conducted by Means and Neisler (2020) who assessed the experiences and student satisfaction levels among 1,008 undergraduate college students with ERT during the COVID-19 pandemic. Only 19% of students rated their overall course experience as "very satisfied" after the transition to online classes as compared to 51% of students who rated their course experience as "very satisfied" before the COVID mandated ERT (Means & Neisler, 2020) . The study findings by Alqurshi (2020) may provide a partial explanation to the outcomes reported by Means and Neisler (2020) . Alqurshi (2020) reported a negative correlation between limited studentÀteacher interaction (r = -.24; p < .01) and perceived clarity in assignment instructions (r = -.161; p < .01) and student satisfaction. Two studies examined the relationship between socio-demographic and geographic characteristics and learner satisfaction with ERT during COVID-19. According to Aristovnik et al. (2020) , males were more confident with their computer skills (p < .001) in online learning platforms. Chung et al. (2020) further examined the relationship between demographic factors and the higher education student's experience, readiness to learn, and preference in ERT during COVID-19. The sample consisted of 399 students across two online learning courses in Malaysia to examine self-directed learning, learner control, motivation for learning, computer/internet self-efficacy and online communication self-efficacy and to assess overall learning satisfaction, experience, and intention to continue online learning in the next semester. Both gender and program levels were shown to have a significant effect on online learning satisfaction (p < .05). Females were more satisfied than males (p = .02) and degree students were more satisfied than diploma students (p = .014). Both females (p = .021) and degree students (p = .004) also had better overall online learning experiences. Of note, there is an absence of studies in the higher education literature that have examined age or generational differences regrading learner satisfaction with ERT implementation during COVID-19. Social Cognitive Theory (SCT) by (Bandura, 1986) was the theoretical framework that guided this study. SCT is a widely known model for understanding and predicting human behavior and identifying methods in which behavior can be changed (Bandura, 1986) . This theory proposed a reciprocal model in which cognitive factors, environmental factors, and human behavior interact (Wood & Bandura, 1989) . This theory posits that students learn when the environment is social, where students interact with each other and the teacher, and the exchange is energetic and lively. Considering the traditional in-person learning environment most often takes on the attributes of a dynamic learning environment and one that promotes the constructs of the SCT, the purpose of this study was to examine how ERT influenced one's attitudes in terms of learner satisfaction when exposed to change in social and physical environment in the online setting. The primary research question for this study was: When learning in an Emergency Remote Teaching environment during the COVID-19 pandemic, is there an association between the multiple instructor and learner attributes of the Student Satisfaction Survey and overall student satisfaction with their courses? This study used an online survey platform to collect data on learner demographics, instructional attributes, learner attribute and learner satisfaction. Data were collected from undergraduate and graduate nursing students whose courses were transitioned to an online learning ERT platform due to COVID 19 pandemic state-mandated restrictions. All potential participants were asked to take part in this study by the Principal Investigator (PI) via an email describing the details of the study, risks, and benefits to the participants with a link to the consent and survey. An email was sent to a total of 629 undergraduates and 49 graduate nursing student enrolled at a medium sized private university located in the mid-Atlantic region of the Country. Data were collected in June and July 2020. The study received IRB approval from the institution where the study occurred. A modified version (approved by the instrument author) of Student Satisfaction Survey (SSS), developed by DeBrough, (2003) , was used to collect data on predictors of learner satisfaction specifically in academic courses that pivoted to ERT for the COVID-19 pandemic. The original SSS was a 59-item assessment (including demographic questions) that consisted of eight subscales separated into two groups-learner and instructor attributes. For this study, to update the tool to reflect current technology practices, several subscales of the SSS were modified or deleted, and demographics were analyzed separately. The final revised tool included 34 items and five subscales rated on a five-point Likert scale (1 = Very Poor to 5 = Very Good). There were three learner attributes (including technology competence, previous experience with online coursework, and betweenclass use of technology), and two instructor attributes (including instructional engagement and use of technology). Student satisfaction was a composite score of overall satisfaction and was determined as an average score from two questions (including "Overall, how satisfied are you with ERT?" and "Compared to a conventional inperson course, ERT was. . ."), also rated on a five-point Likert scale (1 = Extremely dissatisfied to 5 = Extremely satisfied; and 1 = Much worse to 5 = Much better, respectively). Exploratory factor analysis (EFA, principal components) with a varimax rotation was performed to confirm the factor structure of the SSS given the modifications made for the present study. EFA yielded one instructional subscale and two learner attribute subscales that explained 50% of the overall variance in the measure. Two items were dropped from the scale due to poor fit with the factors on which they loaded. Another item, that did not load with any other items ("Prior to COVID-19, how many exclusive online courses have you taken?"), was kept as a separate predictive factor in the model because it was deemed to be a theoretically important part of the model predicting satisfaction with ERT. Because of the non-normality of the distribution, we dichotomized the variable into "having prior online course experience" (n = 69) versus "have no prior online course experience" (n = 26). The instructor attributes made up a single subscale that combined the items measuring the instructor's engagement with the class and their skills using the technology for ERT to its fullest. This subscale consisted of 25 items and had a high internal consistency (Cronbach's a = .96). The two learner subscales 1) technology competence and 2) between-class technology use (frequency of email, ZOOM, Blackboard and FAX for course related work), to each contained three items and had poor to adequate internal consistency ratings, partially due to the limited number of scale items (Cronbach's a for technology competence = .60 and Cronbach's a for between-class technology use = .49). Students were also asked six open ended questions to express what they liked and what they would change or improve about using Blackboard, ZOOM and Virtual Sim Ò while learning remotely during the pandemic. Permission was obtained by the author to revise the SSS to reflect current technology to include Blackboard, ZOOM and Virtual Sim Ò . To answer our primary research question, multiple linear regression analysis was used to determine whether learner and instructor attributes (i.e., subscales derived from the SSS) were related to student satisfaction with ERT, while controlling for student age. Means and standard deviations were calculated and intercorrelations of all study variables were performed to detect multicollinearity problems between predictors in the model. Histograms and normal Q-Q plots of the standardized residuals were used to detect multivariate normality and potential outliers in the regression model. The normality assumption was deemed adequate, however there were three outlier datapoints. Cook's distance, df fit statistics, and analysis of leverage (i.e., the degree to which the outliers influenced the model) showed low concern of bias surrounding the influence of the datapoints on the conclusions of the regression model. To confirm this, sensitivity analyses were conducted removing the three outliers, and results of the model were unchanged. Missing data was not a factor in the analysis given that the survey utilized a forced-choice response system on the SSS survey items and age. All student responses to the open-ended questions were completed using AtlasTi8 following the content analysis method described by Elo et al. (2014) . Data were initially coded by three members of the study team to identify themes that emerged inductively from individual responses. The study team members shared individual themes, discussed rationale for themes and addressed any conflicts. Final themes were then identified for each open-ended question. Descriptive statistics for sample demographics are provided in Table 1 . Means, standard deviations, and intercorrelations of major study variables are shown in Table 2 . To assess overall satisfaction, an average score of 2.79 (SD = 0.09) was calculated based on learner's responses to two questions rated on a Likert scale of 1-5. Most of the specific responses for overall satisfaction with ERT were between a score of 2 (dissatisfied) and 3 (just ok). When asked how ERT compared to in-person courses, responses were between 2 (worse) and 3 (about the same). The multiple regression analysis is presented in Table 3 . The model's overall F-value (5, 89) = 24.66, p < .01. The model's adjusted Rsquare was .56, suggesting that 56% of the variation in student satisfaction was explained by the five independent variables (instructor attributes, learner technology competence, learner between-class technology use, learner online course experience, and age). Instructor attributes around creating an engaging learning environment and using technology to its fullest were strongly positively associated with student satisfaction (beta = 0.933, 95% CI = 0.752, 1.115). Learner self-ratings of technology competence (ability to use the ERT platforms) were also significantly positively associated with student satisfaction (beta = 0.241, 95% CI = 0.064, 0.41). Learner online course experience had a non-significant but marginally positive association with student satisfaction (beta = 0.256, 95% CI = -0.031, 0.549). Learner between-class use of technology (beta = -0.078, 95% CI = -0.255, 0.098) and age (beta = -0.003, 95% CI = -0.019, 0.013) had no unique association with student satisfaction scores. Many of the participants responded to the 6 open-ended questions, which were analyzed for themes and provided valuable information to further describe the quantitative data. The question which addressed what students "liked" about ZOOM received the most responses (92%) and what was "liked" about Virtual Sim Ò received the least (40%). Table 4 lists the themes and response percentages for all 6 open-ended questions. The final question of the SSS invited participants to share additional comments. Fifteen participants (16%) provided comments and overall, expressed gratitude for the time and effort made by the faculty teaching during the COVID-19 pandemic. Student comments included "Understandably the level of organization and preparedness improved as the weeks went on," and "The quality of Emergency Remote Teaching was great, what it (sic) affected me was my internet connection and the difficulty to deal with children at home". Many expressed concern for how teaching would occur during the fall semester of 2020 with an even a greater desire to return to real patient clinical practice experiences and in-person classes. The findings of this study provide some support to those of Alqurshi (2020), Aristovnik et al. (2020) , and Srinivasan (2020) who reported a higher degree of student satisfaction with ERT during COVID-19 than what was reported in this study. The combined satisfaction index of this study showed the students in this learning community were relatively neutral (or just below neutral) in their satisfaction with ERT, with a mean student satisfaction score of only 2.79 ( §0.09). Of note, most of the study participants were traditional undergraduate students (n = 60%) who attend the University for its reputation as an in-person community-oriented teaching environment. Nursing classes for this group of participants are almost exclusively in-person and how they were accustomed to learning pre-pandemic. This may be a contributor to the just below neutral overall satisfaction score. The open-ended questions provided valuable information about student satisfaction and provided more details for why participants may have rated the overall satisfaction questions below neutral. Students "liked" the ease, accessibility, and convenience of Black-Board, ZOOM, and Virtual Sim Ò . Students found BlackBoard selfpaced, flexible, and promoted autonomous learning in that it was easy to submit assignments, access recorded lectures and participate in discussion boards anytime, anywhere. The organization of the content was both a "like" and a need for "change". When material was well organized, easy to find all in one place, it was listed as a like. The opposite was true when material was disorganized, old, and hard to navigate. The synchronous engagement that ZOOM promoted was, by far, a vast "like" among participants. The ability to see classmates and faculty, ask questions and have a face-to-face discussion made their distance learning resemble an in-person experience. While all participants reported that they understood the reason why all classes Table 4 Themes (With Definitions) and Response Results to Open-Ended Questions. Question #1: List the things you like about taking a course taught in Blackboard (N = 70) EASE OF USE/ACCESSIBILITY: accessible on many different types of devices; easy to submit assignments; course content organized and easy to find all in one place; familiar platform) (79%) AUTONOMOUS LEARNING: self-paced flexible learning; access to recorded learning tools and discussion board anytime anywhere (21%) Questions #2: List the things you would like to change or improve about taking a course taught in Blackboard (N = 64) COURSE CONTENT ORGANIZATION: Hard to navigate, disorganized; hard to find assignments; use it in a consistent way (36%) PROMOTE ENGAGEMENT: incorporate more student-student interaction besides just a written response; improve access to faculty and other student; more engaging learning activities (34%) TECHNOLOGY FUNCTION/RELIABILITY: enhance product/app reliability and use on multiple browsers, computers systems, and devices (phone); improve video and audio quality (30%) Question #3: List the things you like about taking a course taught via Virtual Sim (N = 38) REALISTIC: scenarios matched clinical patient conditions, provided a variety of nurse/patient experiences; best alternative to in person clinical time (42%) AUTONOMOUS LEARNING: instructor accessibility; no fear of error; ability to repeat (32%) EASE OF USE/CONVENIENCE: ability to learn from my own home, felt more comfortable; flexible, easy to fit in schedule (26%) Question #4: List the things you would like to change or improve about taking a course taught via Virtual Sim (N = 39) REALISM: scenarios lacked variety of clinical experiences; improvement necessary to provide a real nurse/patient experience; improve clinical details (depth and breadth) of patient condition to promote critical thinking and match student level of experience (basic skills vs advanced clinical competence) (54%) PROMOTE ENGAGEMENT: use interactive features to engage and promote group discussion between students and faculty; improve faculty familiarity and organization of scenarios with v-sim product to use all features and promote engagement (36%) TECHNICAL FEATURES: enhance product reliability; use product that is easier to navigate and functions with multiple browsers and computers systems; enhance software to increase speed and program performance (10%) Question #5: List the things you like about taking a course taught via ZOOM (N = 87) SYNCHRONOUS ENGAGEMENT: ability to see classmates and faculty real-time; able to ask questions, and face to face interactions and discussion with faculty and students (32%) ZOOM FEATURE: able to see the class, share screen, polls, chat box for questions; many tools like the classroom such as raise hand option, recorded lectures, and breakout rooms (29%) COMFORT/CONVENIENCE: ability to learn better in the comfort of my own home; not commuting to campus (26%) EASE OF USE: easy, and simple to access; minimal technical issues (13%) Question #6: List the things you would like to change or improve about taking a course taught via ZOOM (N = 73) INSTRUCTOR/STUDENT KNOWLEDGE OF ZOOM BEST PRACTICE: improve professor training and organization; knowledge of ZOOM etiquette; decrease ZOOM fatigue/length of class; minimize distractions (47%) TECHNOLOGY FUNCTION/RELIABILITY: recognition of time zone differences, improve audio and visual quality, connectivity reliability to minimize screen freezing and shorten content loading time, streamline exam protocol (34%) SYNCHRONOUS ENGAGEMENT: more interactive and participatory opportunities; improve personal connection with professors & classmates; increase class discussions and ability to ask questions (19%) needed to move online, ZOOM by far provided the best experience. What they "liked" about ZOOM was evident by statements such as "I liked being able to see and talk with my professor live, as well as feel like we were having the next best thing to in-person class" and "Next best alternate to being in person, very convenient to participate and ask questions in class." Although Virtual Sim Ò was used by fewer students, this platform matched clinical patient experiences in an environment that felt safe to make a mistake without affecting a patient. Concerns for technology reliability, engagement, and faculty knowledge of use of each platform were reported as elements of the experience that needed be "changed." The strong positive relationship between students' satisfaction with ERT during COVID-19 with their professors' degree of engagement and personal connection relates to those of Alqurshi (2020) and shows the critical role of the faculty play in times of change and crisis. While Alqurshi reported a negative correlation between student satisfaction and limited studentÀteacher interaction their results and the findings of this current study both highlight the importance of instructor engagement with students' satisfaction. Relatedly, the qualitative findings suggest that students relied on professors to create a sense of belonging among the students and the faculty. Being part of a greater whole, while being recognized as an individual, may reflect students' perceived sense of social isolation during the state-mandated quarantine. In addition, the institution in which these students were enrolled is mission-driven in which community is a prominent part of this institution's culture. Seeking a sense of community with ERT dependent on faculty engagement may explain this finding. Despite the characteristics and culture of the institution where this study occurred, as well as the COVID-19 imposed need for ERT, education as a practice, requires a strong degree of faculty presence and effective student engagement strategies. Raina and Khatri (2015) posited that faculty engagement is a significant predictor of enhanced student learning. This conclusion is supported by the findings of Cantrell and Farer (2019) who found that disengaged faculty impact undergraduates' degree of learning. Faculty should receive training on how to implement an effective online course, so they are equipped and competent to best utilize technology so that students are engaged, and learning is enhanced. The results also find that for students, issues such as interest, motivation, and satisfaction are tied to their perception of successful knowledge acquisition within the context of instructor engagement, resourcefulness, and competence with online teaching tools. This study included a larger number of undergraduate students than graduate. This occurred by design; the recruitment email was sent to 677 students of which only 49 were graduate level students. The total response rate was 22% (N = 95) of which the majority, 87.3%, were ungraduated and only 12.7% graduate. Academic year was not a demographic question asked on the survey. Further exploration of academic year may have provided additional information about the expectations and satisfaction for new students, freshmen, as compared to upper classmen. Our sample was also limited in its diversity. Respondents were quite young (mean age = 24 § 8.3) and mostly consisted of female-identifying (91%) non-Hispanic white students (78%). Exploration of a more diverse group may have gleaned different outcomes, especially with respect to our finding that age was not associated with student satisfaction in the regression model. In addition, students received this survey at the end of the semester when the university also conducted a student's satisfaction survey. Many students expressing fatigue with technology which may have impacted the return rate. Overall, the ability of faculty to create an engaging learning environment that uses technology to its fullest has the greatest influence on student satisfaction with online learning. Students sought guidance, expertise, and reassurance from faculty while learning via ERT. As online teaching due to the COVID-19 pandemic is likely to continue, faculty have a responsibility to provide students a high quality "product" that includes faculty who are competent and prepared to teach in an online environment. This includes ensuring course content is available and organized, demonstrating competence with the use of technology and being accessible outside of class time À similar to the expectations of in-person courses. While students' satisfaction with ERT during COVID-19 was their own degree of technology competence, the main attribute driver of their satisfaction was their perceived degree of faculty engagement. The ability of faculty to create an appealing online learning environment that uses technology to its fullest had the greatest influence on student satisfaction with ERT. These findings were supported by both the qualitative and quantitative data. The implication of these findings suggest that higher education faculty should focus pedagogical competence in cultivating skill in those teaching-learning strategies that promote student engagement in online learning environments. Investigating the impact of COVID-19 lockdown on pharmaceutical education in Saudi Arabia-A call for a remote teaching contingency strategy Impacts of the COVID-19 pandemic on life of higher education students: A global perspective Social foundations of thought and action: A social cognitive theory Millennial nursing students' experiences in a traditional classroom setting Online learning readiness among university students in Malaysia amidst COVID-19 Predictors of student satisfaction in distance-delivered. Graduate nursing courses: what matters most? Qualitative content analysis: a focus on trustworthiness The difference between emergency remote teaching and online learning Planning for a Blended Future: A Research-Driven Guide for Educators Suddenly online: A national survey of undergraduates during the COVID-19 pandemic Global impact of COVID-19 on education systems: the emergency remote teaching at Sultan Qaboos University Faculty engagement in higher education Medical students' perceptions and an anatomy teacher's personal experience using an e-Learning platform for tutorials during the Covid-19 crisis Social cognitive theory of organizational management The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.