key: cord-0715163-7tqvh4ix authors: Banjo-Ogunnowo, Susan M.; Chisholm, LeAnn J. title: Virtual Versus Traditional Learning During COVID-19: Quantitative Comparison of Outcomes for Two Articulating ADN Cohorts date: 2022-02-24 journal: Teach Learn Nurs DOI: 10.1016/j.teln.2022.02.002 sha: 4bae275331b2fd9c1c683991874ebecef10ef15a doc_id: 715163 cord_uid: 7tqvh4ix AIM: : The study compared outcomes between articulating nursing student cohorts virtual learning during COVID-19 and traditional learning prior to COVID-19. BACKGROUND: : Clinical reasoning is vital to developing clinical nursing judgment. During the COVID-19 pandemic, nursing schools replaced traditional classroom and in-hospital clinical with virtual classroom and clinical learning. DESIGN: : A quantitative retrospective cohort study compared outcomes of the nursing students virtual learning during COVID-19 (n=18) with traditional learning methods before COVID-19 (n=14) at one college in Texas. METHOD: : Outcomes of the study were measured using a standardized Health Education Systems Inc. (HESI®) specialty exam, HESI® end of program (Exit) exam, and HESI® clinical judgment subscales. Independent-samples t-tests and Chi square were conducted to compare outcomes from virtual and traditional learning. RESULTS: : No statistically significant difference was observed between the traditional group (M=752.93, SD=148.88) and virtual learning group (M=761.72, SD=152.11); t(30) =-.16, p =.87 was identified on the HESI® Maternal pediatric Specialty exam. No statistically significant differences were observed on the HESI® Exit exam or clinical judgment subscales; no relationship between the type of learning and NCLEX-RN® pass rates. CONCLUSION: : The study findings are encouraging, yet limited, warranting further investigation of virtual learning. Background: Clinical reasoning is vital to developing clinical nursing judgment. During the COVID-19 pandemic, nursing schools replaced traditional classroom and in-hospital clinical with virtual classroom and clinical learning. Design: A quantitative retrospective cohort study compared outcomes of the nursing students virtual learning during with traditional learning methods before at one college in Texas. Method: Outcomes of the study were measured using a standardized Health Education Systems Inc. (HESI ® ) specialty exam, HESI ® end of program (Exit) exam, and HESI ® clinical judgment subscales. Independent-samples t-tests and Chi square were conducted to compare outcomes from virtual and traditional learning. Results: No statistically significant difference was observed between the traditional group (M=752.93, SD=148.88) and virtual learning group (M=761.72, SD=152.11); t(30) =-.16, p =.87 was identified on the HESI ® Maternal pediatric Specialty exam. No statistically significant differences were observed on the HESI ® Exit exam or clinical judgment subscales; no relationship between the type of learning and NCLEX-RN ® pass rates. The study findings are encouraging, yet limited, warranting further investigation of virtual learning. The global COVID-19 pandemic provided an opportunity for nursing educators and nursing students to demonstrate resilience in the face of adversity. Traditional methods of learning, such as large lecture halls, on-campus simulation, and clinical placements were limited and, in some cases, prohibited by state guidelines to prevent the spread of COVID-19 (National Council of State Boards of Nursing, 2020). For many schools, virtual learning was the only option to on-campus instruction. Universities are equipped with online learning management systems and methods for virtual conferencing, which were quickly utilized for course delivery. However, nursing programs also teach using clinical skills labs, simulations, and direct patient care experiences which are challenging to replicate virtually. The global COVID-19 pandemic triggered many clinical sites to suspend clinical agreements with nursing schools (Dewart et al., 2020) . Across the United States, clinical agencies allowing student participation in patient care experienced a shortage of personal protective equipment; therefore, clinical agencies limited student participation to non-COVID-19 patients or suspended clinical affiliation agreements (Fogg et al., 2020; Shea & Rovera, 2021) . As the rates of COVID-19 cases in the hospital increased, the availability of non-COVID patients for student interaction was limited; thereby, limiting clinical opportunities for nursing students. An alternative to in-hospital clinical and on-site high-fidelity simulation is virtual simulation (Fogg et al., 2020; Pleasant, 2020) . A survey conducted in 2017 by Wolters Kluwer Health and the National League for Nursing (NLN, 2018) indicated nurse educators were leading in the adoption of new technologies. Virtual simulation (VS) is one rapidly emerging technology which involves manipulating a simulated environment via computer (Padilha, Machado, Ribeiro, & Ramos, 2018) . The Wolters Kluwer Health and NLN (2018) survey report shows 65% of nursing programs use virtual simulation in addition to many other technologies and nearly half are expected to use virtual reality within five years. Based on evidence obtained through the landmark National Council of State Boards of Nursing (NCSBN) study (Hayden et al., 2014), up to 50% of clinical practice hours may be safely replaced with various modes of simulation while continuing to achieve good outcomes. However, little is known about the outcomes of virtual learning and the impact on clinical reasoning and judgment in nursing as compared with traditional classroom and clinical education. The authors will describe one school's standardized testing outcomes from traditional classroom and clinical approaches (pre-COVID) and the outcomes of virtual classroom and virtual simulation experiences during COVID-19. The purpose of this study was to compare the critical thinking, clinical reasoning, and clinical judgment of articulating Associate Degree Nursing (ADN) students in traditional and virtual learning experiences as measured by Health Education Systems Inc. (HESI ® ) specialty and HESI ® end of program exams. Tanner's model "Thinking like a nurse" (2006), derived from research studies about the way nurses' think when doing patient care, was the theoretical framework for this study. The development of sound clinical judgment ability is essential to making vital clinical decisions in nursing and not just the application of scientific nursing process that includes assessment, diagnosis, planning, intervention, and evaluation (Modic, 2013) . Clinical judgment is "an interpretation or conclusion about a patient's needs, concerns, or health problems, and/or the decision to take action (or not), use or modify standard approaches, or improvise new ones as deemed appropriate by the patient's response" (Tanner, 2006, p. 204) . The four phases of Tanner's clinical judgment model are noticing, interpreting, responding, and reflecting (Tanner, 2006) . Noticing is a function of prior knowledge, contextual knowledge, and knowing a patient, based on experience through clinical or practical knowledge of similar patients. The Interpreting process includes critical thinking, analytical, intuitive, or narrative reasoning (Modic, 2013) through which the nurse interprets the meaning of assessment data and selects relevant nursing diagnoses. In the responding phase, the nurse's reasoning leads to implementation of the appropriate nursing action. By reflecting on the patient's response to nursing interventions, the nurse adjusts actions as needed, reflects on own actions and the clinical knowledge derived from the experience which further develops clinical judgment skills (Tanner, 2006) . Simulation requires use of one or more typologies to promote, improve, or validate a participant's progression from novice to expert (International Nursing Association for Clinical Simulation and Learning's [INACSL] Standards Committee ab , 2021), such as the use of virtual reality and virtual patients. Virtual reality is the "recreation of reality depicted on a computer screen" (McGovern, 1994 (McGovern, , p. 1054 . The COVID-19 pandemic limited in person instruction and patient care experiences (Fogg et al., 2020; NCSBN, 2020; Shea & Rovera, 2021 ) for many nursing students; therefore, alternate methods to achieve learning outcomes were needed. Highfidelity simulations have been used effectively to replace up to 50% of the clinical experience without adverse effects on learning outcomes (Curl et al., 2016; Hayden et al., 2014) . Virtual simulations may be a plausible alternative to high-fidelity simulations on campus and clinical experiences in light of the pandemic; however, data supporting the efficacy of replacing clinical and high-fidelity simulation experiences with virtual simulations is limited. Whether simulations are conducted virtually or in person, debriefing and evaluation of learning is essential (INCSL Standards Committee, 2016ab). The Virtual Patient Lasater Clinical Judgment Rubric (vpLCJR), adapted from Lasater (2007), has been successfully utilized in assessing clinical reasoning and judgment skills of nursing students during virtual simulations (Georg et al., 2019) . Debriefing of virtual simulations may be facilitated by the virtual simulation computer program (Goldsworthy et al., 2019) ; however, debriefing methodologies are not always reported (Redmond et al., 2020) and consequently identified as an overall gap in the literature (Foronda et al., 2020) . A quantitative retrospective cohort design was utilized to answer the research question: pandemic from May to July 2020. Standardized HESI ® exam raw scores and conversion scores for the HESI ® Admission Assessment (A2), HESI ® Maternal-pediatric Specialty exam, and HESI ® end of program (Exit) exam were used to examine student learning outcomes. According to Elsevier Incorporated (2020), the HESI ® raw score reflects application of the HESI ® Predictability Model (HPM) to the composite score. The HPM is highly accurate in predicting National Council Licensure Exam (NCLEX) success (Elsevier Incorporated, 2020). Currently, the KR20 for HESI ® Specialty Exams is 0.92 -0.98 and 0.90 for the HESI® RN Exit Exam indicating good reliability for these established exams (Riley & Gouveia, 2022) . Raw HESI ® scores range from 0 to 1500 and are used to predict NCLEX success, while the HESI ® conversion score is the weighted composite percentage score that considers the average difficulty of the exam and the average difficulty of the test items that were answered correctly. The HESI ® A2 measures proficiency in reading comprehension, math, and English grammar. The HESI ® Maternal-pediatric (Specialty) exam administered at the end of the Maternal-pediatric course is a standardized exam designed by HESI ®. The HESI ® Exit exam was administered to students at the end of the program to prepare students for the NCLEX ® Institutional Review Board (IRB) approval was obtained from the Principal Investigator's university prior to data collection. Informed consent was not obtained due to the retrospective nature of data collection. Data were coded for confidentiality and maintained in a secure location. Data were retrieved from student files as each exam was part of the student's record and usual educational process. The HESI ® A2 exam results (administered prior to nursing program admission) were used to assess for baseline differences between groups. The HESI ® Specialty exam (administered at the end of the Maternal-pediatric course) was utilized to assess for differences immediately after the course, and the HESI ® Exit exam (administered after all nursing courses were complete) was used to analyze differences at the end of the program. Data analysis was completed using the Statistical Package for Social Sciences (SPSS) version 24. Descriptive statistics (percentage, mean, and standard deviation), Chi square, and independent-samples t-tests were used to assess for differences between groups. Descriptive data regarding age, gender, and ethnicity were not collected in order to maintain confidentiality of participants in the small sample. Data were collected regarding HESI ® A2, HESI ® Specialty, HESI ® Exit, and NLN Clinical Judgment, NLN Nursing Practice, Clinical Judgment/Clinical Decision Making/Critical thinking subscales for the HESI ® Specialty and HESI ® Exit exams. Mean scores and standard deviations are provided for Group 1 (traditional) and Group 2 (virtual) in Table 1 . An independent-samples t-test was calculated comparing the mean HESI ® A2 raw scores for Group 1 (traditional) and Group 2 (virtual). No significant difference was found for the raw scores (t(30) = 0.726, p > .05) or conversion scores (t(30) = 1.11, p > .05) indicating there was no statistically significant difference between groups at baseline. In order to test for differences based on the type of learning methodology, independentsamples t-tests were conducted on the HESI ® Specialty exam and the HESI ® Exit exam. No statistically significant differences were observed between the traditional and virtual learning groups on the HESI ® Specialty exam, the NLN clinical judgment subscale, NLN nursing practice subscale, or the Clinical Judgment/Clinical Decision Making/Critical Thinking subscale after the Maternal-pediatric course (See Table 2 ). Although scores for the Virtual group were slightly higher, the independent-samples t-test to compare the raw scores on the HESI ® Maternalpediatric specialty exam did not detect a significant difference in raw scores between the traditional group (M=752.93, SD=148.89) and virtual group (M=766.00, SD=145.43), conversion scores, or subscales (see Table 3 ). Higher scores were observed on the HESI ® Exit exam for Group 1 (traditional group) as reported in Table 4 . Independent-samples t-tests were conducted to compare raw scores, conversion scores, and subscales on the HESI ® Exit exam between the traditional and virtual groups. No statistically significant differences were detected between raw scores for the traditional group (M=896.79, SD=148.5) and the virtual group (M=859.11, SD=105.73) on the HESI ® Exit exam [t(30)=4.91, p =0.430], conversion scores, or the clinical judgment subscales (see Table 5 ). Differences in NCLEX-RN ® pass rates were also examined. Chi square was used to assess the relationship between NCLEX-RN ® pass rates and the type of learning: Group 1 (traditional) and Group 2 (virtual). The results indicate there is no relationship between the type of learning and NCLEX-RN ® pass rates (X 2 = 1.339, df = 1, p = .512). During Spring 2020, nursing schools experienced challenges obtaining permission for students to attend clinical experiences in affiliated clinical sites due to the COVID-19 pandemic. A retrospective analysis was conducted to compare HESI ® A2 and HESI ® Maternal-pediatric standardized exam results for the Fall 2019 traditional learning (Group 1) and Spring 2020 virtual learning (Group 2) cohorts. No significant differences were detected on the raw or conversion scores for the HESI ® A2 baseline assessment prior to nursing program admission. These data indicate the groups were not significantly different prior to the Maternal-pediatric course in which different teaching and learning methodologies (traditional vs. virtual) were utilized. The HESI ® Exit exam was utilized to measure end of program learning outcomes. The HESI ® Exit exam raw and conversion scores were similar between the two groups with the traditional group scoring higher than the virtual group; however, the difference was not significant. Two additional modules followed the Maternal-pediatric course. Students in the virtual learning group (Group 2) continued virtual learning during these modules. Although the difference is not statistically significant, the cause of lower scores on the HESI ® Exit exam need further investigation especially since scores for the virtual group were slightly higher on the Maternal pediatric specialty exam. Differences may be related to virtual learning in the Maternal-pediatric course, continued virtual learning in the two subsequent modules, or other covariates, namely the COVID-19 pandemic. Five factors contribute to nurses' clinical judgments. (Tanner, 2006) . Based on the data in this study, clinical judgments of both groups are similar; however, it is quite possible that the clinical judgment subscales on the HESI ® exams are not sensitive enough to detect a difference (Polit & Beck, 2021) . Virtual learning during the Spring 2020 (pandemic) semester was analyzed and outcomes were compared with those of students in the Spring 2019 (pre-pandemic) semester; however, many additional covariates may have impacted HESI ® exam results. The unexpected global pandemic is one covariate which needs to be considered as a potential for adversely impacting student exam performance. In addition, methods for debriefing virtual simulations and clinical experiences were not analyzed. Debriefing is a critical activity for learning in simulation-based education (Palaganas, Fey, & Simon, 2016) , and although debriefing did occur for virtual simulations and post-conferences did occur for clinical experiences, the debriefing methodology was not analyzed. It is possible that variations in debriefing strategies and faculty debriefing skills may have impacted the findings. Overall, there were no differences in the HESI ® clinical judgment, clinical decision making, critical thinking subscale scores between the traditional and virtual learning groups in this study; indicating the groups learned and retained critical thinking, reasoning, and judgment skills equally. Results are limited to the comparison of two articulating cohorts at one school in Texas. Assessment of clinical reasoning using the vpLCJR for virtual simulations and the Lasater Clinical Judgment Rubric for on campus high-fidelity simulations would provide an additional measure of clinical judgment, strengthen the study, and limit covariates due to differences in faculty debriefing strategies. The ultimate goal of nursing education is safe nursing practice. The NCLEX-RN ® exam is an assessment of basic safety and clinical judgment necessary to enter professional nursing practice. Results of the Chi square calculation indicate there is no relationship between the type of learning (traditional or virtual) and NCLEX-RN ® pass rates, indicating both groups were equally likely to attain this essential benchmark. The study is limited to LVN articulating nursing students from one Health Care institution in Southeast Texas region. Due to small sample size and the limited sample population studied, results cannot be generalized to other populations of nursing students. Demographic data such as age, gender, and ethnicity for this group were not analyzed, so further analysis based on demographics was not possible. In addition, although all groups utilized simulation, variances in simulation facilitation and debriefing were not explored and may potentially account for some of the difference between groups. Although the HESI ® standardized exams demonstrate reliability and internal consistency, the results of this study are limited to differences HESI ® can detect. There may be other measures which are more sensitive and have the ability to detect differences. No significant differences were observed for traditional and virtual learners on NLN Nursing Judgment, NLN Nursing Practice, and the Clinical Judgment, Clinical Decision Making, and Critical thinking subscales of the HESI ® Specialty or HESI ® Exit exam; therefore, nursing institutions not already using virtual simulation learning technology may consider its usage. However, differences in the overall HESI ® Exit exam raw and conversion scores need further investigation. Virtual simulations may be a viable solution when traditional on-site educational practices are not possible such as those experienced during the global coronavirus pandemic ; although further investigation of prolonged substitution of traditional methods with virtual learning is needed. Future research will need to focus on evaluating the effectiveness of virtual learning in non-articulating associate and baccalaureate nursing programs. Nurse educators must evaluate the effectiveness of virtual educational tools and determine the impact on learning outcomes, specifically on skill performance, clinical reasoning, and clinical judgment. Multi-site research studies should be considered to evaluate the impact of virtual learning on student retention, end of program student learning outcomes, long-term exposure to virtual learning and success on the NCLEX-RN ® licensure exam. A more rigorous study design including randomization of control and intervention groups is also recommended to control for confounding variables. The future of traditional clinical learning is unpredictable in view of the current global pandemic and measures to promote continued clinically based learning need to be fully examined. Since the COVID-19 global pandemic is an ongoing crisis which may forever alter the way education is delivered throughout the world (Dewart, Corcoran, Thirsk, & Petrovic, 2020) , it is important to analyze the long-term impact of educational interventions in order to address these unprecedented conditions. Authors have no conflict of interest. Effectiveness of Integrated Simulation and Clinical Experiences Compared to Traditional Clinical Experiences for Nursing Students Nursing education in a pandemic: Academic challenges in response to COVID-19 HESI ® exams scoring explanation Transitioning from direct care to virtual clinical experiences during the COVID-19 pandemic Virtual simulation in nursing education: a systematic review spanning Psychometric properties of the virtual patient version of the Lasater Clinical Judgment Rubric How does simulation impact building competency and confidence in recognition and response to the adult and pediatric deteriorating patient among undergraduate nursing students? The NCSBN national simulation study: A longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education Healthcare Simulation Standards of Best Practice TM The Debriefing Process Healthcare Simulation Standards of Best Practice TM Evaluation of Learning and Performance Kaplan i-Human Patients. (2020). i-Human patients company overview Virtual patient simulations in health professions education: Systematic review and meta-analysis by the digital health education collaboration Clinical judgment development: using simulation to create an assessment rubric Applications of virtual reality to surgery Model of Clinical Judgment Applied to Preceptorship: Part 1 Tanner's Model of Clinical Judgment Applied to Preceptorship: Part 1 Changes in education for nursing programs during COVID-19 65% of nursing education programs adopting virtual simulation Clinical virtual simulation in nursing education Structured debriefing in simulation-based education. American Association of Critical Care Nurses Advanced Critical Care Simulation learning system (SLS) as an alternative clinical experience Nursing research: Generating and assessing evidence for nursing practice Increasing competence in wound care: a cross-sectional study to evaluate use of a virtual patient by undergraduate student nurses White paper: 2022 scientific evidence for Elsevier HESI ® RN and HESI ® APRN Exams Preparing for the COVID-19 pandemic and its impact on a nursing simulation curriculum Thinking like a nurse: A research-based model of clinical judgment in nursing 65% of nursing education programs adopting virtual simulation The authors would like to acknowledge Dr. Eileen Deges Curl and Dr.Ruthie Robinson for proofreading, editorial suggestions, and recommendations for improvement of this work.This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.