key: cord-0993898-79whwzm8
authors: Akbari, Fakhridokht; Nasiri, Morteza; Rashidi, Neda; Zonoori, Sahar; Amirmohseni, Leila; Eslami, Jamshid; Torabizadeh, Camellia; Havaeji, Fahimeh Sadat; Bigdeli Shamloo, Marzieh Beigom; Paim, Crislaine Pires Padilha; Naghibeiranvand, Mehran; Asadi, Masoomeh
title: Comparison of the effects of virtual training by serious game and lecture on operating room novices’ knowledge and performance about surgical instruments setup: a multi-center, two-arm study
date: 2022-04-11
journal: BMC Med Educ
DOI: 10.1186/s12909-022-03351-5
sha: fb0f3fd74212599fe44501a896e3eb5fc273dda8
doc_id: 993898
cord_uid: 79whwzm8

BACKGROUND: Game-based training is increasingly implemented in different nursing fields, as it allows students to learn experientially, with the flexibility to regulate their training based on their personal progresses and abilities. This study aimed to compare the effects of virtual training by the “Playing with Surgical Instruments (PlaSurIn)” game and the lecture on the surgical instruments setup knowledge and performance of Operating Room (OR) novices. METHODS: This study was conducted on 51 s-semester undergraduate OR technology students taking the course “An Introduction to Surgical Instruments and Equipment.” An additional virtual training session was held via a learning management system using two different methods. The students of the Game Training Group (GTG, n = 27) played individually with the “PlaSurIn” game during a week, while the students of the Lecture Training Group (LTG, n = 24) received the lecture-based training during a week. To measure knowledge, all the students participated in a theoretical test with 10 multiple-choice questions before and immediately after the training. They also participated in an Objective Structured Clinical Examination (OSCE) after the training, and their performance was evaluated by the remained time for setup completion and the scores, errors, and bonuses. RESULTS: The mean score of the theoretical test was significantly higher in the GTG than in the LTG after the training (p = 0.040). Additionally, the GTG participants had higher scores (p = 0.016), fewer errors (p = 0.001), and higher bonuses (p = 0.011) compared to the LTG ones. The remained time for setup completion was also significantly longer in the GTG than in the LTG (p < 0.001). CONCLUSION: Virtual training by “PlaSurIn” was superior to the lecture-based method for the enhancement of surgical instruments setup knowledge and performance amongst OR novices.

Akbari et al. BMC Medical Education (2022) 22:268 Background A complex issue in perioperative nursing training is the way novices develop their technical or procedural skills, improve their performance, and acquire their theoretical or practical knowledge [1] . Most perioperative nursing training occurs through the clinical apprenticeship methods in surgical wards and Operating Rooms (ORs) [2] . In these learning environments, trainees experience high levels of stress due to being subjected to time pressures by the OR personnel, which makes them susceptible to clinical errors [3] . In addition, OR trainees face challenges regarding traditional training due to the inherent nature of perioperative nursing, increasing demands to improve patients' safety, limitations in OR resources, and restrictions in perioperative trainers' work hours [1] [2] [3] . Accordingly, it is necessary to complement or change the traditional training methods to access the required perioperative nursing competencies among OR novices.

Game-based training, as an alternative method for traditional training, has received increasing attention in different nursing fields [4, 5] . Serious games developed in perioperative nursing allow trainees to acquire knowledge and develop skills in a safe and relaxed virtual learning environment [6, 7] . These games create a fun and positive interactive learning experience for trainees and foster their collaboration, communication, and critical thinking by assisting them to maintain their engagement in training, decrease their fear of unknown and environmental stress, and prevent their clinical errors, ultimately improving the quality and safety of patient care [4, 8] .

Among different games developed in perioperative fields, considerable attention has been devoted to games in the field of surgical instrumentation. In this regard, different types of games have been introduced in nursing and medical training such as "Playing with Tweezers" [9] , "Play and Learn for Surgeons" [10] , "PeriopSim ™ instrument trainer" [11] , and "Nintendo Wii U ™ " [12] [13] [14] . Recently, a serious game called "Playing with Surgical Instruments (PlaSurIn)" was developed for OR novices to set up basic surgical instruments on the Mayo stand or a back table at the commencement of minor general surgeries [15] . The "PlaSurIn" is an English computerbased game adapted from a Portuguese game called the "Playing with Tweezers" through a rigorous modification approach and validation process [9, 15] . The "PlaSurIn" has a "main menu" containing three main options on the right side, namely "virtual mode, " "learning mode, " and "assessment mode" (Fig. 1 ). In the "virtual mode, " trainees can view the setup of 35 instruments in the designed table, six instrument classifications (i.e., retractors, hemostats, others, graspers, cutters and dissectors, and needle holders), and each instrument's information and image when clicking on the instrument (Fig. 2) . In the "learning mode, " trainees set up instruments on the designed table according to the "virtual mode" as many times as needed and can request tips if necessary. In the "assessment mode, " trainees recreate the instruments setup according to the platform of the "learning mode" in seven minutes. They can also see their obtained scores and the remained time for setup completion at the end of the "assessment mode. " Moreover, they receive a bonus at the end of the "assessment mode" in case they place all the instruments of a classification in their correct positions on the designed table. Trainees must reach the appropriate level of 70% regarding the correct positions of the instruments in each attempt; otherwise, they should repeat the game whenever they wish to promote their performance (Fig. 3 ) [15] .

In a recent systematic review, games used in health professions were reported to be effective as a training method. Many studies also indicated that games were more effective in improving the trainees' knowledge and performance [16] . Although the "PlaSurIn" game has been validated, the effects of playing with this game compared to other training methods on learning outcomes have not been evaluated [15] . Considering the importance of comparative studies for drawing evidence-based conclusions regarding the superiority of serious games over other types of training strategies, the present study aims to compare the effects of virtual training by the "PlaSurIn" game and the lecture on surgical instruments setup knowledge and performance amongst secondsemester undergraduate OR technology students. We assumed that virtual training by the "PlaSurIn" game is more effective than virtual training by the lecture-based method in enhancing surgical instruments setup knowledge and performance.

The present study was conducted at Abadan University of Medical Sciences (AUMS) and Shiraz University of Medical Sciences (SUMS). Undergraduate OR technology students were invited during the academic year 2020-2021 to participate in the study. The inclusion criteria were 1) taking the course "An Introduction to Surgical Instruments and Equipment, " 2) studying in the second semester, and 3) having access to a personal computer. The students were excluded if they had any experience of participating in the surgical instruments setup courses or playing serious games. Totally, 51 students were invited. All these students met the inclusion criteria and willingly accepted to participate in the study. To avoid sharing information between the students of two groups, the conditions were randomly assigned to universities by coin tossing. To this end, 24 students who studied at AUMS were allocated to the Lecture Training Group (LTG), and 27 students who studied at SUMS were assigned to the Game Training Group (GTG).

According to the program of the Ministry of Health and Medical Education in Iran, the course "An Introduction to Surgical Instruments and Equipment" contains two theoretical credits, which is equal to 16 two-hour sessions. Since surgical instruments setup is trained by clinical apprenticeship methods, less attention is paid to these topics in this course. Accordingly, an additional session was conducted to train the students how to set up basic surgical instruments on the Mayo stand or a back table in minor general surgeries.

The course was presented in 16 theoretical sessions in the second semester by the corresponding professor, and the same course plan, lesson plan, training approach, and evaluation method were implemented for the students of the two groups. Moreover, the curriculum was the same between groups during the study and also before the study. In two groups, the same topics regarding the names and applications of surgical equipment and instruments were trained via a Learning Management System (LMS). This virtual training method has been routinely performed at most universities in Iran during the Coronavirus Disease 2019 (COVID-19) pandemic. For each session, a two-hour lecture was presented and the students were provided with educational resources (i.e., multimedia, textbooks, and podcasts), assignments, and self-exams with Multiple-Choice Questions (MCQs). At the end of the semester, the final theoretical test with MCQs was conducted through an online synchronous system for the students of two groups under the same conditions.

Four weeks after the final theoretical test, an additional session was conducted to train students how to set up basic surgical instruments. This session was held for both groups during a week by an experienced professor who was unaware of the study objectives using either game-based or lecture-based training. The students of both groups were informed that top-performing students in the additional session would receive prizes to ensure their maximum performance.

The lecture-based training was performed by using the LMS in the same approach as the theoretical sessions. To this end, a two-hour online session based on lecture and Microsoft PowerPoint presentation was held on the first day of the additional session, and the students were trained the principles of surgical instruments setup in minor general surgeries similar to the platform established in the "virtual mode" of the "PlaSurIn" game. The students were also given the last edition of the book "Surgical Mayo Setups" [17] and a recorded video containing the instructions presented in the online session and they were allowed to review them as much as they wished during a week. Moreover, some assignments and selfexams were presented and the students were requested to do them individually during the week. In addition, the students were allowed to send messages to their professor when needed through the "messages module. " Afterwards, all the functions provided via the LMS, except for the "messages module, " were deactivated. At the end of the week, no message was received from the students.

Game-based training was defined as playing with the "PlaSurIn" game. First, a video clip was presented in the LMS to instruct the students how to play the game. Then, the game Uniform Resource Locator (URL) address was presented and the students were provided with unique usernames and passwords to enter the game. They were requested to play the game individually on the computer whenever they wished to do so during a week. Afterwards, the game URL address and the related video clip were deactivated. The students were also asked to send messages to their professor through the "messages module" in order to resolve their problems about the game. In total, three students reported their problems during the week, which were mainly due to invalid usernames or passwords. Other functions of LMS (i.e., online session, assignments, and self-exams) were not used for the students in the GTG.

To evaluate the homogeneity of the groups in terms of the demographic and educational data, the following information was recorded: age, gender, marital status, previous semester's Grade Point Average (GPA, average grade received for all the courses a student took in the previous semester), number of credits taken in the current semester, and the final theoretical test score of the course "An Introduction to Surgical Instruments and Equipment. "

To assess the students' knowledge of the basic surgical instruments setup on the Mayo stand or a back table at the commencement of minor general surgeries, a theoretical test was performed. This test consisted of 10 MCQs (e.g., item No. 1: on which part of the table do you place a cutter or dissector instrument) developed by five researchers (instructors of surgical instruments). The qualitative content validity of the test was approved by 10 OR faculty members and its qualitative face validity was satisfactory as determined by 10 students with similar characteristics to those of the target population. In order to evaluate the internal consistency and stability of the test, a test-retest analysis was done with a 12-day interval. In doing so, 30 OR technology students who were not supposed to participate in the main analysis took part in the test, and satisfactory results were obtained by Kuder-Richardson 20 (KR20, r = 0.76) and Intra-Class Correlation (ICC, r = 0.81) [18] . The theoretical test was performed for the students of two groups in 15 min on the first and last days of the additional session through an online synchronous system under the same conditions. To decrease the possibility for random answering, correct answers were displayed at the end of the test and the order of questions was set randomly. The students' scores were recorded by a blinded professor. It is worth mentioning that scores one and zero were assigned to correct and wrong answers, respectively. Therefore, the total score range was from 0 to 10, with higher scores indicating greater knowledge of the surgical instruments setup.

The students' performance on the surgical instruments setup on the Mayo stand was evaluated by an Objective Structured Clinical Examination (OSCE) one week after the end of the additional session. The OSCE was performed in a single station in the corresponding university under the same conditions. Accordingly, the students were asked to set up instruments on the Mayo stand within seven minutes. This time was considered based on the time allocated in the "assessment mode" of the "PlaS-urIn" game. Also, to ensure that the time allocated for OSCE was adequate, a pilot OSCE was run before the real OSCE for four second-semester OR technology students, who were not included in the main analysis. During the OSCE, a blinded rater recorded the students' skills using an OSCE form, which addressed the four items of time, score, error, and bonus. In this regard, time, score, and bonus were considered based on the variables presented in the "PlaSurIn" game [15] , while error was regarded as a new variable in the current study. Time was considered as the remained time for setup completion. Score was defined as the total score obtained out of 100 by summing up the instrument's classification scores calculated by multiplying the number of instruments in a classification by the importance score of that classification, as follows: 1) retractors: 2 × 2.5 = 5, 2) hemostats: 6 × 2.75 = 16.5, 3) others: 10 × 2.75 = 27.5, 4) graspers: 7 × 3 = 21, 5) cutters and dissectors: 7 × 3 = 21, and 6) needle holders: 3 × 3 = 9. Error was considered the total number of errors in setting up the instruments in their correct positions, which could range from 0 to 35 (Fig. 2) . Finally, bonus was defined as the correct positioning of all the instruments in a classification. Considering the six classifications, the number of bonuses could range from zero to six.

The qualitative face validity and content validity of the OSCE form were approved by 10 OR faculty members. In addition, its inter-rater reliability was measured by calculating the correlation between the scores recorded by two trained OR faculty members with the same professional characteristics for 30 OR technology students (not participating in the main analysis). The ICC coefficients for the agreement between the faculty members were found to be 0.81-0.92, which signified the substantial agreement of the ratings [19] . To avoid bias, the rater who completed the OSCE form at each university was trained by the main researcher how to evaluate the students' performance. The ICC coefficients for the agreement between the raters were 0.86-0.91.

The study was approved by the Regional Research Ethics Committee of AUMS. All methods were performed in accordance with the relevant guidelines and regulations. The study was also reported based on Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) [20] .

The data were analyzed using the SPSS software, version 22 (SPSS, IBM © Corp., Armonk, NY, USA), and p < 0.05 was considered statistically significant. The normal distribution of the quantitative data was not confirmed by the Kolmogorov-Smirnov test. Therefore, the homogeneity of the study groups in terms of demographic and educational characteristics was assessed using Mann-Whitney U test and Fisher's exact test. Additionally, Mann-Whitney U and Wilcoxon signed-rank tests were used for within-group and between-group comparisons of the students' theoretical test scores. Mann-Whitney U test was also employed to compare the study groups concerning the OSCE score.

All the students in the two groups were single. The results also revealed no significant difference between the two groups in terms of other demographic and educational data (Table 1) .

The results showed no significant difference between the study groups regarding the mean score of the theoretical test at baseline (p = 0.306). However, the mean score obtained by the students in the GTG was significantly higher than that obtained by the students in the LTG after the training (p = 0.040) ( Table 2 ). The two groups' mean scores were significantly higher after the training compared to the baseline (p < 0.001 in GTG, p = 0.010 in LTG).

The remained time for setup completion was significantly longer in the GTG than in the LTG (p < 0.001). The scores and number of bonuses were also significantly higher in the GTG compared to the LTG (p = 0.016, p = 0.011). On the other hand, a significantly fewer number of errors were made by the students in the GTG compared to those in the LTG (p = 0.001) ( Table 2 ).

This study aimed to compare the effects of virtual training by the "PlaSurIn" game and lecture-based method on the surgical instruments setup knowledge and performance of OR novices. The results demonstrated that the remained time for the setup completion was significantly longer in the students trained by the "PlaSurIn" game in comparison to those trained by the lecture. In addition, the students in the GTG had fewer errors and obtained higher scores and bonuses compared to those in the LTG. The results also indicated that the students in the GTG gained higher theoretical test scores compared to those in the LTG after the training. Although both groups' theoretical test scores were higher after the intervention compared to the baseline, the increase was more prominent in the GTG. Hence, the results supported the hypothesis that using the "PlaSurIn" could enhance the surgical instruments setup knowledge and performance among OR novices.

The "PlaSurIn" has been introduced as the first international game of surgical instruments setup developed in the nursing field. Previous studies regarding the effects of game-based training in surgical instrumentation are limited to medical students and surgical residents [11] [12] [13] [14] 21] . The results of a recent study showed the effectiveness of "Play and Learn for Surgeons," a serious game in uterine artery ligation surgery, on the surgical instruments handling and knowledge of surgical instruments amongst obstetrics and gynecology residents [10] . The results of another study indicated that using the "Perio-pSim ™ instrument trainer" and "PeriopSim ™ for burr hole surgery" significantly increased the total score, saved time, and decreased errors in the identification of neurosurgical instruments during a simulated burr hole surgery procedure among neurosurgery residents [11] . Similarly, playing with "Nintendo Wii U ™ , " a laparoscopic video game, could improve inexperienced medical students' performance regarding laparoscopic instrument handling [12] [13] [14] . Although the findings of the reviewed studies are in line with those of the current study concerning the beneficial effects of serious games on surgical instrumentation skills, the comparison should be done with caution due to the differences in the populations studied, game objectives and designs, and methods used to evaluate the trainees' performance.

Based on the findings, it seems that the "PlaSurIn" game could be used as an acceptable alternative or complement to the traditional training methods to access the required knowledge and performance about surgical instruments setup among second-semester OR technology students. Given that the "PlaSurIn" game is free of cost and has an easy-to-use platform, it can be used in future training programs to enhance OR novices' surgical instruments setup knowledge and performance in a limited amount of time with reduced educational resources, especially when there is no access to routine training methods including the COVID-19 lockdown period. Moreover, the "PlaS-urIn" game is of practical use when perioperative trainers are under pressure due to restrictions in their work hours and a large number of students. The results of this study could be generalized to other universities with similar student demographics. Moreover, the "PlaSurIn" game could be used to train surgical instruments setup for other health-related students such as inexperienced nurses and surgical residents during their OR apprenticeship courses. However, further studies are required to evaluate the generalizability of the present findings to other health-related students. Accordingly, it is hoped that similar interventions will be implemented and tested amongst other novices, who are involved in setting up surgical instruments, to evaluate the generalizability of the present findings to other educational settings and participants.

To the best of our knowledge, this work was the first interventional study to evaluate an educational game on the surgical instruments setup in the nursing field. One major strength of this study was the evaluation of both knowledge and performance. In doing so, OSCE was used since it has been reported to be a well-validated evaluation approach [22] . This was, in fact, another strength of the current study since most previous studies evaluated trainees' performances using simulations such as the virtual reality. Additionally, most previous studies on this topic did not assess the validity and reliability of their measures. In the current study, however, a group of OR faculty members and students confirmed the face and content validity of the theoretical test and the OSCE form. The reliability of these tools was confirmed, as well. Moreover, the OSCE raters were instructed regarding the assessment tool and their agreement was satisfactory.

The study findings should be interpreted with caution due to some limitations. Even though the students were recruited from two universities via census, the small sample size in each group can limit the generalizability of our findings. In addition, generalization of the present findings can be limited as group allocation was done based on the university, which might lead to selection bias due to the inadequate generation of a randomized sequence. Moreover, no pretest was performed for performance, as it could lead to a bias related to the use of the same OSCE. Likewise, this study did not measure whether the students were able to retain the knowledge learned from the two training methods. Finlay, the findings could be affected by inequality in the students' computer literacy. However, the students in two groups were instructed how to use the LMS so as to minimize this difference.

Training by the "PlaSurIn" game was more effective compared to the lecture-based method in the improvement of the surgical instruments setup knowledge and performance of OR novices. Thus, it seems that the "PlaSurIn" game can contribute to the development of OR novices' skills for surgical instruments setup. However, further long-term studies with larger sample sizes are recommended in this area. 

Professional development strategies to enhance nurses' knowledge and maintain safe practice

Nursing shortages in the OR: Solutions for new models of education

What factors within the peri-operative environment influence the training of scrub nurses?

Educational games developed for students in perioperative nursing: a systematic review and appraisal of the evidence

Learning experiences of game-based educational intervention in nursing students: A systematic mixed-studies review

Engaging perioperative learners using online game-based education

Using games to provide interactive perioperative education

Using a videogame to facilitate nursing and medical students' first visit to the operating theatre. A randomized controlled trial

Development of an educational game to set up surgical instruments on the mayo stand or back table: applied research in production technology

Play and learn for surgeons: a serious game to educate medical residents in uterine artery ligation surgery

Simulation-based training for burr hole surgery instrument recognition

Playing to your skills: a randomised controlled trial evaluating a dedicated video game for minimally invasive surgery

Impact of video game genre on surgical skills development: a feasibility study

Training surgical skills using nonsurgical tasks-can Nintendo Wii improve surgical performance?

Playing with Surgical Instruments (PlaSurIn)" game to train operating room novices how to set up basic surgical instruments: a validation study

Serious gaming and gamification education in health professions: systematic review

Surgical mayo setups. Spiral bound Version: Cengage Learning

Measurement in nursing and health research

The measurement of observer agreement for categorical data

Development and validation of the guideline for reporting evidencebased practice educational interventions and teaching (GREET)

Development of a serious game for laparoscopic suture training

How to measure the quality of the OSCE: a review of metrics -AMEE guide no. 49

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

The authors would like to thank the Vice-chancellors for Research Affairs of AUMS and SUMS. They would also like to appreciate the valuable assistance of all the students who took part in the study. Thanks also go to the instructors of the course "An Introduction to Surgical Instruments and Equipment" at the two involved universities. Finally and yet importantly, Ms. A. Keivanshekouh at the Research Consultation Center (RCC) of SUMS is acknowledged for her invaluable assistance in editing the manuscript.

This work was supported by AUMS (grant No. U98-601). The funding body had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Data and materials are available by contacting the corresponding author.

This study was approved by the Regional Research Ethics Committee of AUMS (approval No. IR.ABADANUMS.REC.1398.055). All the students were informed about the study objectives and their informed consent was obtained before the initiation of the study. 

Ready to submit your research Ready to submit your research ? Choose BMC and benefit from:? Choose BMC and benefit from:

No individual's data are included in this study.

The authors declare that they have no known competing financial interests or