key: cord-0709748-c97thcfa authors: Cygan, Heide; Bejster, Mallory; Tribbia, Carly; Vondracek, Hugh title: Impact of COVID‐19 on public health nursing student learning outcomes date: 2021-10-06 journal: Public Health Nurs DOI: 10.1111/phn.12978 sha: b8f32843c1989d504c209c6cfb453e1612e6913d doc_id: 709748 cord_uid: c97thcfa BACKGROUND: The COVID‐19 pandemic has highlighted the importance of a strong, effective public health nursing workforce while also requiring public health nursing faculty to adapt teaching strategies as courses transitioned online. It is essential to understand how the pandemic‐enforced transition from face‐to‐face to remote learning impacts student outcomes. The purpose of this paper is to compare student learning outcomes in a pre‐licensure public health nursing course before, during, and after the transition to remote learning. METHODS: Descriptive statistics were computed for assignments, exams, and final course grades for three terms (Fall 2019, Spring 2020 and Fall 2020). RESULTS: Analysis showed statistically significant differences between terms for assignments and exams but not the final course grade. However, these differences were driven by small standard deviations rather than differences between mean scores demonstrating that there was actual little difference in student learning outcomes across terms. CONCLUSIONS: Authors suggest strategies to support consistent academic outcomes and future research needed understand student learning outcomes during the pandemic; ultimately building the public health nursing workforce necessary to address the current and future public health crises. As the COVID-19 pandemic emerged in the United States in early 2020, nursing programs across the country were forced to rethink nursing education. This public health crisis simultaneously elevated the need for a strong public health nursing workforce while challenging faculty who traditionally use face-to-face (F2F) teaching to develop this workforce. During the COVID-19 pandemic public health nurses serve on the frontlines in a variety of roles, highlighting their importance. Public health nurses are leading mobile strike teams to inves-tigate and contain the spread of COVID-19, interpreting guidance from the Centers for Disease Control and Prevention to best serve underserved populations, and delivering patient-specific education on infection control measures such as isolation and quarantine (Edmonds et al, 2020) . At the same time, public health nursing faculty swiftly implemented teaching strategies to maintain student, faculty, staff and patient safety while continuing to prepare future nurses to enter the workforce Dewart et al., 2020; Morin, 2020; Tomietto et al., 2020) . graded assignments and three examinations. All graded assignments and examinations are in alignment with course objectives. All assignments and examinations, other than the final cumulative exam, can be considered formative evaluation methods as they allow students to identify areas for continued learning. Learning evaluation criteria are described in Table 1 . The public health nursing course is typically delivered F2F once per week in an active, learning classroom setting. The CDC immunization module and discussion board activity were always completed asynchronously online outside of F2F class. However, other activities and all three exams were conducted in a F2F environment. The course was last taught 100% F2F in Fall 2019. In Spring 2020, this course started just 1 day before a novel coronavirus was identified as the cause of respiratory illness and death in China (Holshue et al., 2020) . In the United States, the first case of this new virus was identified on January 20, 2020 (Holshue et al., 2020) . Starting on January 27, 2020 (week four of the term), faculty began incorporating emerging knowledge about the novel coronavirus into course content. This was done mainly during a 10-min section of class titled PHN in the Media, during which faculty would highlight a news story related to the new virus. Further, course faculty used the emerging public health crisis as a learning opportunity, relating the current events to course content whenever possible. The emerging pandemic was discussed in class in relation to public health concepts such as the social determinants of health, communicable disease and ethics. By early March 2020, the virus was given the name COVID-19 and public health officials began to issue guidance in response to the growing number of cases in the United States (Centers for Disease Control and Prevention [CDC], 2020). Local public health officials issued guidance limiting the number of people allowed to gather in one place (City of Chicago, 2020). On March 7, 2020, all non-essential large group gatherings over 50 people at the university were cancelled. However, based on the University guidance, classes continued as they were considered essential. On March 9, 2020, when there were just 647 total cases of COVID-19 in the United States (CDC, 2020), we held our last live class of the year (week nine of the term). On March 11, 2020 the university cancelled all live classes and asked faculty to make immediate arrangements to continue classes remotely for the foreseeable future. This meant that while the course began F2F during the Spring 2020 term, due to the emerging COVID-19 pandemic the course transitioned to 100% remote during week 10 of the 15-week course. The course remained 100% remote for the Fall 2020 term. Faculty were consistent across all three terms. Table 2 shows the transition in course delivery format through-out the three terms. Details about this transition and student perceptions of the transition can be found in 2021. Data was collected from the online learning management system for all learning evaluation criterion. All raw scores for each learning evaluation criterion across three terms: Fall 2019, Spring 2020, and Fall 2020, were converted into percentages to permit comparison. Standard descriptive statistics (mean, standard deviation, and range) were computed for assignments, exams, and final course grades for each term and compared between the three terms. Means were compared using a one-way ANOVA and graphical analysis to identify differences between terms. This project was reviewed by the authors' Institutional Review Board and approved as a quality improvement project. All data are reported in aggregate to protect student privacy. The one-way ANOVA analysis shows that statistically significant differences exist between terms for all assignments (except the CDC module for which it could not be computed as all scores were identical across F I G U R E 1 Assignment scores and final course grade by term the three terms) and exams but not the final course grade. Table 3 includes data for all assignments, exams, and final grades across the three terms. For the assignments, these differences appear to be driven by small standard deviations rather than large differences between mean score. This is evident by the largest difference of any pair being only 2.04 percentage points between the poster presentation in Fall 2019 and Fall 2020, as can be seen on Figure 1 . Differences on exam scores between terms are caused by greater variability in Fall 2020, as can be seen in Figure 2 . Fall 2019 and Spring 2020 exams were never more than 1.41 percentage points different. There was no difference between the scores on the CDC Immuniza- The low score for the peer review ranged from 70 to 90, with the high score for each term remaining at 100. Figure 1 includes graphical analysis of assignment scores by term. The purpose of this paper was to compare student learning outcomes in a pre-licensure public health nursing course over three academic terms, before and during the COVID-19 pandemic. It is important to note that in Fall 2019, the term preceding the COVID-19 pandemic, the course was taught 100% F2F. In the middle of Spring 2020, the course transitioned from F2F to remote learning, and in Fall 2020 the course was taught 100% remote. Nonetheless, based on our evaluation, there was little difference in student learning outcomes across the three terms. As noted in the results section, all changes in assignment and examination scores were within a few percentage points. Even so, rationale for these changes should be considered. Discussion board grades increased slightly. Asynchronous discussion board activities are an established method for increasing nursing student knowledge in didactic courses (Hudson, 2014 ). An increase in discussion board scores over time for this course may be a result of students becoming more familiar with this type of online learning activity. As the COVID-19 pandemic persisted this type of learning activity became used more frequently in courses than it was used during F2F instruction. Further, this activity facilitated interaction with peers which has been noted as important for nursing students during the pandemic (de Tantillo & Christopher, 2020). Moving forward, nursing faculty should consider the use of asynchronous discussion board activities to facilitate student learning and peer engagement. Synthesis presentation grades also increased. One reason for this may be that students were provided with multiple platforms to coordinate group work during the pandemic. Prior to the pandemic, students were expected to organize F2F group meetings. However, once this course transitioned to the remote environment, faculty set up virtual workspaces for students, allowing them flexibility in how they met as a group to prepare for the presentation. Flexibility has been identified as a facilitator of learning during the pandemic de Tantillo & Christopher, 2020; Goni-Fuste et al, 2021) . Authors also identified a drop in exam 2 scores during Spring 2020. This is not surprising considering this exam was administered in week 10 of the course; the first week this class was taught remote. Students were required to use an online, live, exam proctoring service of which they were not familiar. Exam anxiety is often cited as a stressor and barrier to learning for nursing students (Quinn & Peters, 2017) . While there is limited literature available to explore the impact of online exam proctoring on nursing students (Castano et al., 2021) , onlineproctoring was noted as anxiety inducing for college students at large during the pandemic (Elsalem et al., 2020) . Further, general anxiety for nursing students related to COVID-19 is well documented (Savitsky et al., 2020) . Combined, these factors may be the reason for a decrease in exam 2 scores. In order to better understand the reasons for changes in student learning outcomes across the terms, authors suggest a qualitative deep-dive into the student experience exploring rationale, both academic and personal, that may impact learning outcomes. Authors acknowledge that evaluating assignment and exam scores only considers student learning outcomes in a didactic course and may not be transferable to the clinical setting. Authors suggest an evaluation of clinical skills to determine if students were also successful in meeting clinical objectives consistently during the pandemic to more fully evaluate overall student learning. This paper describes the evaluation of student outcomes in one public health nursing didactic course. Results may not be generalizable to other universities, programs or courses. Nonetheless, considering the limited availability of published literature specifically exploring student academic outcomes during COVID-19, this paper offers unique insight and suggests areas for future exploration. Continued development of the public health nursing workforce is essential to protect the health of the public during and beyond the COVID-19 pandemic. Results of this project demonstrate that although theCOVID-19 pandemic presented a variety of challenges for public health nursing students and faculty over the past year, student learning outcomes remained consistent. Moving forward, nursing educators must continue to evaluate student outcomes in a variety of ways to assess how course adaptations may affect student learning outcomes, especially as more courses begin transitioning back to the traditional F2F format. Further, looking beyond academic outcomes and exploring the overall impact of the pandemic on public health nursing students and new graduate nurses will be essential to ensuring the long-term stability of the public health nursing workforce. Considerations for COVID-19 preparedness and response in U.S. schools of nursing Faculty perspectives on transitioning public health nursing clinical to virtual in response to COVID-19 Implementing remotely proctored testing in nursing education Latest Guidance From the classroom and into the virtual world: Faculty response to COVID-19 Transforming graduate nursing education during an era of social distancing: Tools from the field Nursing education in a pandemic: Academic challenges in response to COVID-19 A call to action for public health nurses during the COVID-19 pandemic Stress and behavioral changes with remote E-exams during the covid-19 pandemic: A cross-sectional study among undergraduates of medical sciences COVID-19 remote learning transition in spring 2020: Class structures, student perceptions, and inequality in college courses Experiences and needs of nursing students during pandemic outbreaks: A systematic overview of the literature First Case of 2019 Novel Coronavirus in the United States Teaching nursing concepts through an online discussion board Nursing education after COVID-19: Same or different Strategies to reduce nursing student test anxiety: A literature review Anxiety and coping strategies among nursing students during the covid-19 pandemic Nursing Education: Challenges and perspectives in a COVID-19 age. Formazioneinfermieristica: Sfide e prospettivealla luce del COVID-19 Impact of COVID-19 on Public Health Nursing Student Learning Outcomes Authors do not have any conflicts of interest to disclose. The data that support the findings of this study are available from the corresponding author upon reasonable request.