key: cord-0079276-1l3ct8eo authors: Tseng, Li-Ping; Hou, Tung-Hsu; Huang, Li-Ping; Ou, Yang-Kun title: The Effect of Nursing Internships on the Effectiveness of Implementing Information Technology Teaching date: 2022-05-13 journal: Front Public Health DOI: 10.3389/fpubh.2022.893199 sha: 88118c1f0172a77a00aa7c04da1581df3e325d55 doc_id: 79276 cord_uid: 1l3ct8eo In nursing education, the diversity of clinical scenarios is complex and dynamic, and it is a challenge for nursing students to learn this clinical knowledge in conventional teaching mechanisms. However, integrating information technology into teaching can promote diversity of learning environment. This study aimed to investigate the effectiveness of teaching mechanisms which combined clinical simulation scenario and Information Technology Integrated Instruction. This study applied innovative experimental teaching in medical–surgical and critical care nursing courses for nursing students in their fourth or fifth year at a five-year junior college. Instructional systems design was combined with clinical simulation scenarios and technology to develop multidimensional teaching strategies. Fifth-year students' overall OSCE mean score was considerably higher than that of fourth-year students; specifically, their scores on basic life support and care for subdural hemorrhage varied substantially. Compared with fourth-year students, fifth-year students performed considerably greater on situational awareness. The results of the present study can be used to develop nursing core competencies and improve the clinical care competency of nursing personnel. Objective structured clinical examination (OSCE) is essentially a method for assessing learning outcomes by measuring clinical nursing ability and competence in clinical practice (1, 2) . To conduct OSCE, a series of assessment stations is established, various health problems are simulated, and students' behavior and interaction with standardized patients are video recorded. Watson et al. (2) defined OSCE as an examination in which "students demonstrate their competence under a variety of simulated conditions." OSCE is effective for presenting learning outcomes and assessing students' clinical performance after graduation (1) . Situational awareness (SA) describes individuals' perception, understanding, and subsequent predictions of what will happen in the surrounding environment. Specifically, people with high SA are aware of their surroundings and use that information to devise necessary plans or make decisions (3, 4). Projection is the highest level of SA and is achieved through personal understanding and the interpretation of perceptual clues as well as the expectation of future results (5, 6) . Nursing care is a dynamic process, and Tower et al. (7) identified clinical decision-making as a paramount skill for nurses when caring for patients. High SA helps nursing personnel make appropriate clinical decisions vital to patient care outcomes and safety. SA training for nursing personnel should be improved to help them achieve the highest level of SA and provide high-quality nursing care. The rapid development of information technology and prevalence of the internet have rendered information skills essential to student competitiveness (8) . Innovative teaching methods such as online education, massive open online courses (MOOCs), and small private online courses (SPOCs) have been developed and employ various interactive educational software programs, high-fidelity equipment, teaching platforms, and sharing platforms that differ considerably from conventional teaching methods (9) . The application of information integration has also changed how researchers examine learning methods (10, 11) . The integration of technology into nursing education for the cultivation of professional nursing skills warrants further discussion. This study applied innovative experimental teaching in medical-surgical and critical care nursing courses for nursing students in their fourth or fifth year at a 5-year junior college. Instructional systems design was combined with clinical simulation scenarios and technology to develop multidimensional teaching strategies. The flipped classroom approach and lesson plans based on clinical scenario simulations were applied to establish a reliable and valid OSCE scale for assessing nursing students' learning effectiveness, SA, and satisfaction with the proposed teaching methods. The findings may serve as a reference and highlight valuable implications for the development and application of future nursing education programs. The objectives of this study are as follows: (1) To establish clinical simulation scenario lesson plans for medical and surgical nursing and integrate it with an OSCE, and then to verify the reliability and validity of the OSCE Checklist. (2) Explore the difference between fourth-and fifth-year students in terms of OSCE scores following their participation in innovative teaching courses. (3) Explore the difference between fourth-and fifth-year students in SA performance following their participation in innovative teaching courses. (4) Explore fourth-and fifth-year students' satisfaction with the innovative teaching courses. In the national examination, medical and surgical nursing is a core competency course and a major subject. This course cultivates students' professional and practical capacity in medical and surgical nursing-applying basic biomedical and scientific knowledge and nursing expertise they have learned; assessing and analyzing patients' physical, mental, spiritual, and sociocultural responses; suitably providing nursing measures; and accurately implementing and applying related techniques to clinical practices. This research implements innovative teaching methods, with teaching strategies including lectures, flipped classrooms, group discussions, information-technology based strategies, simulated scenarios and OSCE to cover the following eight topics of nursing care: body fluid and electrolyte imbalance, respiratory system, cardiovascular system, basic and advanced life support, shock, sepsis, and multiple organ failure, endocrine disorders, nervous system and urinary system. There were two stages to develop course and design clinical simulation scenario lesson plans in the present study. Stage 1: The medical-surgical and critical nursing courses integration. The course development team, which constituted by nursing instructors, clinical nursing experts, and a clinical physician, convened several teaching discussion meetings to integrate teaching and learning mechanisms aimed at fourth-year students of a 5-year nursing program. They also incorporated scenario simulation with ITII (Information Technology Integrated Instruction) as the teaching strategy and conducted flipped classroom concepts and a revised OSCE to establish a critical nursing course. Traditional OSCE is usually conducted with rotations among multiple stations, while the three OSCE subjects tested in this study among 126 students, due to constraints in time, space and examiner availability were conducted by the examiners scoring the video recordings of students performing the tasks in the skills lab. The revised OSCE was more effective and fulfills its purpose than the traditional. Stage 2: Scenario simulation teaching and clinical scenario teaching templates with the ADDIE model development. (1) Scenario simulation teaching and clinical scenario teaching templates development The present study introduced scenario simulation teaching into the lesson plans by referencing the Human Patient Simulation Scenario Development Patient Case Template (HPSSDPCT), the Template of Events for Applied and Critical Healthcare Simulation (TEACH Sim) (12) and the ADDIE teaching design model which included analysis, design, development, implementation, and evaluation. The learning priorities, important events, and target responses were set by KSA (knowledge, skills, and attitude) which involved cognitive, psychomotor and affective domains. (2) The clinical scenario simulation lesson plans design The five topics was developed by course development team, included percutaneous transluminal coronary angioplasty (PTCA) and stent placement care for patients with acute myocardial infarction (AMI), basic life support (BLS) and operation of automated external defibrillator (AED), subdural hemorrhage (SDH) care, applications of the advanced Apollo Simulator in care for patients with septic shock, and acute respiratory distress syndrome (ARDS) care. They select the these five topics based on the importance and feasibility of the major teaching contents, and integrate the knowledge of other teaching contents into them as much as possible.Then, the final version was formed by conducting two pilot tests. Depending on the topic scenario, each plan was taught for 30-90 min. OSCE scores were used to evaluate student performance in this study and the test of face validity, content validity, and criterion validity were conduct after the first draft of the OSCE checklists. (1) The First Draft of the Checklists Because septic shock care and ARDS care require the use of advanced simulators and are therefore limited by equipment and this study's OSCE feasibility, these two scenarios were mainly taught and excluded from the OSCEs. In the end, the PTCA care for patients with AMI, BLS and AED operation, and SDH care were included in OSCE checklists. Checklist items were based on the actions in the process that students must complete; each checklist item was graded in 0 to 2 points (2 = fully completed, 1 = partially completed, 0 = not completed). ( The seventeen clinical experts and three fifth-year nursing students were invited to review the first draft of the three clinical scenario checklists and this study used the content validity index (CVI) as the measurement, each item of the evaluation survey was assigned 1-4 points on a 4-point Likert scale, where 1 = very appropriate, 2 = appropriate, 3 = inappropriate, and 4 = very inappropriate. The items were assessed based on expert appraisal, and the statistical method was based on the number of appropriate/very appropriate items divided by the total number of items; if the result was >0.80, the checklist was indicated to have a favorable validity index (13) . The expert evaluations and opinions were organized into a summary table. Items that received 3 or 4 points were retained; items that received only 1 or 2 points or were determined to be unclear or badly worded were either revised or deleted after the expert opinions were discussed by the course development team. The CVI of PTCA care for patients with AMI was 0.981; 26 items were retained (four items were revised, one item was deleted). The CVI of BLS and AED was 0.987; 25 items were retained (three items were revised). The CVI of SDH care was 0.981; 13 items were retained (one item was revised). The OSCE checklists of all three simulation scenarios were graded based on a full score of 100 points. ( (7) for the applications of SA levels in medicine: the perception of cues from the setting, including patient information, electronic medical charts and information from other clinicians is important for processing of information and subsequent decision-making for the organization, rationalization and ordering of usuable data to construct an understanding of the present situation in the system to predict one or more possible occurrence of events to formulate appropriate responses and predict the next step. The first level (SA1) is perception level which included basic patient identification, interpretations, and skills operations. The second level (SA2) is comprehension level which included further treatment following observation, decisions and treatments based on SA1. The last level (SA3) is projection level which included proposal of precautions and health education, decisions and treatment based on SA2. According to course progress and clinical scenario topics, seven information technology tools inclueded SPOCs, Zuvio, MOODLE teaching platform, LINE APP, High-fidelity wireless simulaton, Little Anne QCPR and AED trainer, were introduced in this study. The student satisfaction survey was a structured survey modified with permission from a twenty-item survey by Tsai (14) with items relating to simulated clinical scenario, SPOCs and Zuvio rated by a five-point Likert scale to be submitted by nursing students anonymously for quantitative anlalyses. In total, 61 fourth-year and 65 fifth-year nursing students were recruited from a 5-year junior college. SPSS Statistics 23.0 was used for data compiling and analysis. Applied statistical methods consisted of descriptive statistics, independent sample t-tests, and Pearson's correlation coefficients. This section describes the reliability analysis of the scoring scales for the three OSCE dimensions, the effectiveness of the teaching methods according to clinical scenario simulation and technology integration, and student satisfaction with the innovative teaching methods. The consistency of the three OSCE checklists for the AMI, BLS, and SDH lesson plans were verified using Cronbach's α; the results were 0.61 for the AMI checklist, 0.80 for the BLS checklist, and 0.76 for the SDH checklist. The BLS and SDH checklists achieved high reliability. Correlation Between Three Items of OSCE Score and Students' Scores on the Summative Critical Care Nursing Assessment The correlation between summative evaluations of fourth and fifth year students and the revised OSCE scores, and to utilize the summative scores as performance indices for the OSCE subjects. The showed the results of fourth-year students showed that significant correlation between their summative critical care nursing assessment and revised OSCE score for AMI [r(61) Table 1) . The fifth-year students had completed a year-long clinical nursing internship, whereas the fourth-year students were preparing for the internship after the completion of the prior semester. The results of the independent sample t-test for the two groups are presented in Table 2 . The results revealed that the scores of fourth-and fifth-year students on the summative critical care nursing assessment [t(61.65) = −1.22, p = 0.225] differed non-significantly. The slightly higher mean score for fifth-year students indicated that internship experience did not affect student performance in critical care nursing.The results showed that two groups of students differed significantly in overall revised OSCE scores [t(61.65) = −8.34, p < 0.001], and the fifth-year students had a total mean score 33 points higher than the fourth-year students did ( Table 2) . The following results were obtained ( Compared with fourth-year students, fifth-year students scored 9.59 and 22.05 points higher for mean BLS and mean SDH, respectively. Despite the non-significant difference in AMI, fifthyear students still had a higher mean score for this dimension than fourth-year students did. All participants were assessed for SA1, SA2, and SA3 on the AMI, BLS, and SDH in the OSCE. The following results were obtained ( In total, 20 SPOCs were established in this study, and students were able to preview lessons through the flipped classroom approach. The shortest and longest SPOCs, which were entitled "Bonus: Virtual Reality for BLS and AMI" and "BLS, " were 7 min and 34 s and 44 min and 25 s long, respectively. Most SPOCs were between 20 and 30 min, with a mean duration of 20 min and 10 s. The SPOC completion rate was calculated by the number of fourth or fifth year students online for the duration of the class divided by the total number of fourth or fifth year students recruited for this study, multiplied by one hundred percent.The difference of SPOC completion rates between fourth-and fifth-year students was non-significant (fourth year: 77.5%; fifth year: 75.8%). For the courses "Respiratory System and Review, " "Bonus: Virtual Reality for BLS and AMI, " and "Medical-Surgical Nursing for Cerebrovascular Diseases, " students had higher completion rate than fourth-year students by 10-30%. For the courses "Endocrine System and Function, " "Physiology of the Nervous System, " "Abdominal Physical Assessment and Common Diseases, " and "Gastrointestinal Bleed Care, " the completion rate for fourth-year students was higher than that for fifth-year students by more than 20%. The SPOC completion rate statistics are presented in Table 4 . The overall satisfaction scale used in the current study covered three dimensions, namely scenario-based simulation teaching, SPOCs, and the interactive educational software program Zuvio, and comprised 18 items. The overall satisfaction scale yielded a Cronbach's alpha of 0.984, indicating high reliability. The fourth-and fifth-year students varied in terms of items they were satisfied with and in satisfaction with the three dimensions covered by the scale. The following results were obtained ( Table 5) : Our results showed no significant difference in satisfaction with scenariobased simulation teaching, SPOCs, or the interactive educational software program Zuvio, while the mean and standard deviation in fourth year students were higher than fifth year students. This result was likely caused by differences in group background and because fourth-year students were preparing for their internship and thus responded more positively to innovative teaching. The fifth-year students had already completed 1 year of clinical internship and were preparing for national entrance examinations; therefore, they were more committed to preparing for mock examinations in school than they were committed to studying for elective courses. The results of this study exhibited that after participating in innovative teaching, the fourth-and fifth-year students exhibited significant differences in overall revised OSCE scores; accordingly, internship experience and other factors contributed to higher overall revised OSCE scores. Subsequently, the differences in scores between the two groups in the three OSCE dimensions were analyzed; significant differences were observed in the BLS and SDH scores. The mean BLS score of the fifthyear students was 9.59 points higher than that of the fourthyear students, and the two groups exhibited a considerable 22.05-point difference in SDH scores. Accordingly, internship experience and other factors related to learning experience contributed to higher scores on the three OSCE dimensions. According to the internal discussion of the course development group, SDH was the most difficult to learn, BLS required basic knowledge and skills, and AMI was of medium level of difficulty. The varying difficulty levels across OSCE might have caused the two groups to exhibit non-significant differences in the AMI dimension. After participating in innovative teaching, the fourth-and fifth-year students exhibited significant differences in AMI SA1; BLS SA1, SA2, and SA3; and SDH SA3. Moreover, the fifthyear students had higher scores than the fourth-year students. Specifically, the two groups exhibited a considerable 21.29point difference in SDH SA3. The high difficulty level of SDH might have caused a marked difference in scores between the two groups. Accordingly, internship experience and factors related to learning experience (e.g., higher seniority) greatly influenced the SA3 score for SDH, the most difficult subject. Bambini (15) proposed that the complexity of the simulation should be built according to the learner's level and match their competency. To construct the scenario lesson plans, which approximate clinical practices and complexity, the lesson planner should consider students' competency and complexity of materials simultaneously. Furthermore, the level of difficulty in the curriculum is related to the revised OSCE checklist, as seen in the significant difference between fourth and fifth year students in the total OSCE score and in SA 3. Recommendations for future development of the OSCE checklist based on simulation curricula include formulating program year-and subject-appropriate test items paired to the level of difficulty with SA 1, SA 2 and SA 3 in order to increase student-centered KSA. In the satisfaction survey, fourth-year students had significantly higher scores than fifth-year students on three OSCE dimensions and on overall satisfaction, which may be considered the overall evaluation of the proposed innovative teaching method. The different learning plans students had may have affected the evaluation results. The fourth-year students, who were preparing for their internships, exhibited high satisfaction with innovative teaching, especially in scenario-based simulation. Fourth-year students had an overall satisfaction rate of 96.6% for SPOCs, which was higher than that of fifth-year students by 7.2%. The different learning plans students had may have resulted in the considerable difference between the two groups in terms of SPOC satisfaction rate. Specifically, fourth-year students were preparing for their internships, whereas fifth-year students were preparing for exams. Different learning goals between the two groups might have affected their satisfaction levels. Another possible factor influencing satisfaction is whether the SPOC recordings were attractive to the students. The overall optimization of online courses in the future may yield superior results. In terms of platform operation, both groups found Ewant easy to use and helpful for reviewing physical assessments, basic medicine, and medical-surgical nursing; both groups of students were highly satisfied with the application of SPOCs to autonomous learning and the use of the flipped classroom approach for course preview and review. Both the fourthand fifth-year students had satisfaction rates of 98 and 89.5% toward Zuvio, respectively, and such high satisfaction may be attributable to the software's diverse functions. The fourth-year students had more experience using Zuvio, whereas the fifthyear students-who had just returned to school after a yearlong internship-had little experience using it in class, despite its simple user interface. Whether usage frequency affected satisfaction requires further discussion. Because of the increase in confirmed COVID-19 cases, numerous schools stopped offering onsite education and switched to remote learning (10) . Previous literature has already reported the advantages of replacing MOOCs with SPOCs because there were some challenges of MOOCs (9, (16) (17) (18) . The findings of this study demonstrated that SPOCs support the effectiveness of small-scale blended learning, and that this learning mechanism enhances the comprehensive and in-depth learning experiences for students while providing instructors with flexible and feasible teaching models simultaneously. Such model will provide a convenient way for instructors to understand students' learning needs and behaviors through learning hours, achievement rates, and formative (on-going feedback between students and instructors) and summative (tallying of test scores) evaluations. However, this study is an action research through a practicum course and we could not utilize only one mode of instruction while withholding others to protect the students' rights and interests. Future studies to compare between different modes of instructions such as ones in this study may be conducted in elective or non-core courses. Another limitation of the present study was that the students in the experimental group cannot prevent from Hawthorne effect since the research process cannot adopt a double-blind design in practical. The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s. The studies involving human participants were reviewed and The predictors of nursing students' clinical competency in pre-internship objective structured clinical examination: the roles of exam anxiety and academic success Clinical competence assessment in nursing: a systematic review of the literature Safety at the Sharp End: A Guide to Non-Technical Skills Toward a theory of situation awareness in dynamic systems The future of onboarding: implementation of immersive virtual reality for nursing clinical assessment training Situation awareness and the decision-making processes of final-year nursing students The effectiveness of ICT-enhanced learning on raising intercultural competencies and class interaction in a hospitality course MOOCs, an innovative alternative to teach first aid and emergency treatment: a practical study Guidelines for online assessment in emergency remote teaching during the COVID-19 pandemic Factors that influence teachers' adoption and integration of ICT in teaching/learning process The Template of Events for Applied and Critical Healthcare Simulation (TEACH Sim): a tool for systematic simulation scenario design Measurement in Nursing Research Evaluation of "comprehensive nursing and skill performance" curriculum in a four-year technology college nursing department with clinical-scenario case method Writing a simulation scenario: a step-by-step guide What knowledge is available on massive open online courses in nursing and academic healthcare sciences education? A rapid review Creating a 'choose your topic'massive open online course: an innovative and flexible approach to delivering injury prevention education Who can benefit more from massive open online courses? A prospective cohort study The authors are very grateful for the generous support from St. Martin De Porres Hospital and Chung-Jen Junior College of Nursing, Health Sciences and Management, as well as the assistance and co-operation of all the participants involved in the experiment. We would like to thank reviewers and the editor for their comments. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.