key: cord-1026503-z434tlz1
authors: Feenstra, Tim M.; van Felius, L. Peter; Vriens, Menno R.; Stassen, Laurents P. S.; van Acker, Gijs J. D.; van Dijkum, Els J. M. Nieveen; Schijven, Marlies P.; Bonjer, H. Jaap
title: Does livestreaming surgical education have the future? Development and participant evaluation of a national COVID-19 livestreaming initiative
date: 2022-01-06
journal: Surg Endosc
DOI: 10.1007/s00464-021-08922-z
sha: a0a65141f484f0a16d1316a881961cc5c7254735
doc_id: 1026503
cord_uid: z434tlz1

BACKGROUNDS: COVID-19 related reduction of surgical procedures jeopardizes learning on the job of surgical residents. Many educators resorted to digital resources in the search for alternatives. However, these resources are often limited to the extent they offer resident-surgeon interaction like a joint surgical performance does. Here we present a roadmap of livestreaming surgical procedures, and evaluate how surgical livestreams on human cadavers address the unmet educational needs of surgical residents in our Dutch nationwide initiative. METHODS: Technical and organizational feasibility, and definition of outcome deliverables for the livestream series and per livestream were essential in livestream development. Faculty selected interventions, lecture contents, and participant preparations. Appropriate location, technical setup, and support were imperative for a stable, high-quality stream with integrated interaction, while maintaining digital privacy. A survey was sent to livestream participants to evaluate each livestream, and allow for constant improvement during the broadcasting of the series. Only surveys which were completed by surgical residents were included in the analysis of this study. RESULTS: Each livestream attracted 139–347 unique viewers and a total of 307 surveys were completed by participants (response rate of 23–38% per livestream). Sixty percent of surveys (n = 185) were completed by surgical residents. Livestreams were highly valued (appreciation 7.7 ± 1.1 and recommendation 8.6 ± 1.1), especially the live procedures combined with interaction and theoretical backgrounds. Criticized were technical difficulties and timing of the livestreams between 5 and 7 pm, which interfered with clinical duties. CONCLUSION: Livestreaming surgical procedures on human cadavers is a valid and valued solution to augment resident education. Digital privacy and a stable, high-quality interactive stream are essential, as are appropriate moderation and relevant lectures. While livestreaming cannot replace hands-on training in the operating room, it enables surgeon-resident interaction which is key in education—and missed in pre-recorded surgical procedures which are currently available online.

The COVID-19 pandemic has presented significant challenges to the medical community, and surgical practice needed to adjust to the new reality. Elective surgery was scaled down or postponed due to lack of personal protective equipment (PPE) and to enable sufficient capacity for COVID-19 patients on wards and intensive care units (ICUs), and surgical residents were relocated to COVID-19 units to assist in the care on the ward [1] . Consequently, training opportunities offered by clinical activities, especially by direct involvement in surgical procedures have been significantly reduced, with a negative impact on the residents' learning progress [2, 3] .

Surgical educators were challenged to provide alternatives for surgical training without increasing stress on hospital capacity, and while maintaining social and physical distance. As a result, many educators have resorted to digital education to augment traditional curricula. A pre-COVID-19 study from January 2020 concluded that surgical residents use and appreciate these digital resources [4] . Most platforms provide videos, lectures, tips & techniques, and e-learnings-but none of them offer the opportunity to virtually attend the surgery and interact with the surgeon.

The Dutch Surgical Society (NVvH, Nederlands Vereniging voor Heelkunde) and Amsterdam Skills Centre (ASC) identified the opportunity to offer interactive online education to Dutch surgical residents by livestreaming surgical procedures and technique on human cadavers in a nationwide initiative. The current surgical residency program contains a total of 488 residents, 135 first postgraduate year (PGY) residents-including urology, orthopedic, and plastic surgery residents in their general surgery rotation −, and 63-78 residents per PGY's 2-6. An online platform enabled interaction between surgeons, lecturers, and residents, thereby mimicking the interaction between surgeons and their residents during onsite live surgery. Five surgical key-procedures were chosen and supplemented by lectures of specialist faculty, the first of which aired on April 9th, 2020. This manuscript therefore describes and evaluates the achievements of last year and presents a roadmap to livestreaming surgical procedures and technique in resident education. By sharing our experiences, we aim to demonstrate the possibilities of livestreaming surgery for educational purposes, and to inspire others to adopt this promising digital resource in their curricula.

To maintain surgical education, the NVvH consulted with the ASC, which is affiliated with the Amsterdam UMC, to discuss the feasibility of the livestreams in March 2020.

After the concept was deemed technically feasible, the boards of the NVvH sub-associations (on gastro-intestinal-, oncologic-, vascular-, pulmonary-, and trauma-surgery) were approached to discuss the execution of the livestreams and contents (Fig. 1) . The NVvH prioritized residents in their first four postgraduate years (PGY's) as target audience for the first three livestreams. This, because a prolonged interruption of the education of these residents was particularly detrimental for their development and clinical activities [2, 5, 6] . To enable the highest attendance possible and to accommodate cooperating surgeons, the NVvH decided to broadcast all the livestreams after workhours at 5 pm. The livestreams duration was limited to 1.5-2 h to maintain optimal attention.

The boards of the respective sub-associations contacted their members to recruit faculty for the livestreams. These experts in their respective fields determined which surgical procedures were most appropriate based on the national surgical curriculum [7] , and whether it was feasible within the timeframe defined by the NVvH-the actual livestreaming would have to be effected within four weeks. Therefore the laparoscopic cholecystectomy, intramedullary tibial fixation, lower leg amputation, total thyroidectomy, and introduction to pulmonary surgery were chosen. With the selected surgical procedures as backbone, the faculty decided on specific learning objectives for participants and accompanying explanatory topics. Etiology, symptoms, and diagnosis were covered, as were anatomy, surgical approach, pathology, and prognosis. Next, the performing surgeons and speakers were elected and the lectures were composed. Each livestream brought new experiences that were used to adapt contents, goals, timeframe, and script to support the faculty.

Based on the selected surgical procedure and learning objectives, the faculty of each livestream established what elements to cover in the participant preparation. A combination of existing e-learnings, available video materials, and relevant guidelines was selected and made digitally available to participants for self-study in the weeks before the livestream. Additionally, a voluntary pretest was made available for participants to test their knowledge before participating in the livestreams. All Dutch 1-4 years surgical residents were then approached by the NVvH and the Association of Surgical Residents (Vereniging van Assistent-Geneeskundigen in de Heelkunde, VAGH) to register for the livestream by following a link and creating a login on the secure online platform. The preparation for the livestream was distributed by the ASC to all the registered residents.

The ASC is an existing high-level training center containing a wetlab where live surgical procedures on human cadavers can be performed for training health care workers. Pre-COVID this took place in a setting with faculty and trainees present at the operating tables. In accordance with national laws and regulations, all individuals voluntarily donated their body to science, and consent is obtained from all that their body will be used for scientific and educational purposes. In the Netherlands, such donations are made to anatomy departments of university medical centers. After death, the donated bodies are then transferred to the ASC. The ASC has its own ethical board containing the heads of the both anatomy departments of Amsterdam UMC, a professor in ethics, as well as several surgeons and legal experts who advice and decide on all matters concerning the ethical use of human tissue.

A custom high-performance computer was used to incorporate audio and video signals from the wetlabs and lectures in the conference rooms. Video and images were incorporated in the livestream with Open Broadcaster Software®, a free and open-source cross-platform streaming and recording program, and streamed using Vimeo (InterActiveCorp, New York City) streaming services. The Vimeo platform was chosen above others (such as Twitch and YouTube) because it offers unrestricted video availability to participants with a personal login, and allows the host to disable screen recording or easy downloading of the livestream. This functionality was essential to protect the privacy of the human cadavers, surgeons, and participants. Second, the platform was preferred above video conferencing services because it offers a dependable, high quality, and stable stream which incorporates videos, lectures, and images, and facilitates monitoring of these components. Furthermore, the platform provides a chat-and poll-function which were embedded into the livestream. Livestream hardware and software were extensively tested before the first stream, and constantly improved upon during the subsequent streams.

Participants of courses in the ASC can request free access to online elements of the course through the ASC website, to prepare and augment their learning experience. Consequently, individuals without registration and permission cannot participate in courses or access videos and other leaning materials. All livestreams were broadcast within the learning platform environment that could be accessed on any device with an internet browser.

There was a viewing room next to the studio that was built in one of the conference rooms. Two ASC staff members monitored and adjusted video-, audio-, and livestream quality, supervised contents and viewer interaction, and were in charge of time management. Two additional ASC staff members were standby for extra support and troubleshooting between lab and studio, if necessary. 

The chat-and poll-function enabled interaction between participants, lecturers, and surgeons (Fig. 2) . Designated faculty members moderated the input and output of these functions, while two ASC staff members provided technical and digital support. The designated faculty member(s) managed the questions and comments in the chat, and relayed them to the right person (i.e., the surgeon, lecture presenter) during the broadcast or answered the question themselves. Poll questions were devised in advance and integrated in the livestream at the appropriate timing.

One surgeon and resident performed the procedure with a resident, assisted by two wetlab managers. Overview of video and audio was achieved through three cameras, two microphones, and an integrated speaker system. The surgeons explained their actions during the surgery, sometimes pausing to elaborate on technique or anatomy. The moderator of the livestream interacted with the surgeons, asking for clarification or to discuss the next course of action.

One faculty member was designated as moderator of the livestream as a whole. They introduced the stream, faculty, and the lectures. The moderator or lecturers stood in front of a green screen when presenting, which enabled the simultaneous display of surgery and moderator using picture in picture. Figure 3 summarizes the technical setup of the livestreams and interaction flow of the livestreams.

All participants were asked to voluntarily fill out a survey after each of the livestreams. Depending on the surgical procedure, the survey consisted of 19-23 questions and combined quantitative and qualitative items to enable a complete overview of the participants' experience. Surveys that were completed by surgical residents of PGY's 1-6 were 

Each of the five livestreams attracted an audience ranging from 139 to 347 unique viewers. While the livestreams were created specifically for residents in training, also residents not in training and medical students viewed them as guests. Frequently, surgical staff allowed for group-viewing of the livestreams in lecture halls and staff rooms, while maintaining physical distance. The total reach of the livestreams is therefore not measurable, although it is certainly larger than the number of unique viewers recorded.

A total number of 307 surveys were completed by participants, thereby corresponding with a response of 23-38% per livestream. Sixty percent (n = 185) were residents in training, 19% (n = 59) were medical students, 19% (n = 59) had another function, did not report a function, or skipped most of the survey, and 1% (n = 4) were residents not in training. The vast majority of residents in training rated and recommended the livestreams with at least a seven on a scale form 1-10 (Fig. 4) , and the mean livestream rating and recommendation of the residents in training were 7.7 ± 1.1 and 8.6 ± 1.1, respectively. There was little variety in these numbers based on postgraduate year (PGY) or livestream number (Fig. 5a, b) , although the second livestream had the lowest appreciation due to technical issues.

Three out of four participants that gave a 6 or lower as livestream recommendation did so due to technical issues (n = 3) such as problems with logging in or an unstable connection-all in livestream two-while one resident suggested to a discussion covering the surgical technique and decision-making could have added value. Multiple residents, who scored 7 or 8, commented that the theoretical discussion was too extensively at the expense of the surgery (n = 6), that a start time 5 pm was too early (n = 5), and the difficulty level was too low for more senior residents. The group which graded the livestreams with a 9 or 10 did so because of the interaction between faculty and participants (n = 14), the overall educative value (n = 12), the combination of theory and practice (n = 9), the level of detail of the contents (n = 7), the performance of the faculty (n = 7), and the value of viewing a live surgery (n = 5). Several residents remarked that the offered interaction was of great value to the streams, both the interaction with the participants and the interaction between multiple surgeons. The first was appreciated for the engagement of the participants, while the latter was appreciated because a discussion between multiple surgeons from different hospitals seldom occurs and offered interesting points of view and different surgical techniques.

All components of the livestreams were predominantly rated as positive to strongly positive on a five point Likert-scale, with 'use of moderators' and 'balance between theory and practice' rated best (Fig. 6 ). In the qualitative analysis, residents specifically appreciated the possibility to follow the interaction between multiple surgeons and hear their different surgical approaches and techniques during surgery. Additionally, residents agreed that the theory should be used as background and be in support of the surgery, without overruling the practical execution. Time of day received the most negative and neutral ratings; in the qualitative analysis residents agreed that starting at 5 pm interfered with clinical duties.

Low ratings of livestreams were given because of technical issues, which especially interrupted the second livestream. Several participants were unable to watch the livestream midway onward due to a connectivity outage followed by problems logging in for a second time. Additionally, sound was delayed or absent during parts of streams two and four for some participants.

Livestreaming surgical procedure and technique on human cadavers in our nationwide initiative has proven to be useful, feasible, and very well appreciated by surgical residents. Although use of video in surgical education cannot substitute the learning experience of performing or assisting in real-life surgery, live video streaming does provide the experience of blended learning through the combination of surgical theory with practice, while enabling a two-way interaction between the audience and the performing surgical specialists. Easy access and a stable stream are obviously essential for a successful livestream. Time of day, duration of the livestream, and difficulty level of livestream content are important factors to keep in mind, and should be adjusted to the target audience.

Our large-scale evaluation of participants validates the positive livestreaming experiences as previously referred to in literature. Use of live video streaming for demonstration of surgical technique is a feasible and valuable addition to the surgical education palette according to both surgeons and residents, especially during the COVID pandemic [8] [9] [10] . There are several differences between aforementioned studies in literature and our study. First of all, we performed surgical procedures on human cadavers as opposed to the surgical procedures on patients as published by Jack et al., Faiz et al., and Yuen et al. [8] [9] [10] . Because previous authors all presented data from the USA, legislation on digital privacy and security is different. There is no globally accepted set of laws or regulation that apply to digitally livestreaming surgical procedures, especially not with regard to streams of cadaver surgery. Two authors examining the question of (digitally) sharing anatomical images agree that there is little guidance from in the literature regarding ethics and safety, although there is an increasing risk of unethical or misuse [11, 12] . In the ASC such an ethical board advices and decides on these matters, although (inter)national guidelines can support clinicians and educators now that education is being digitized. There are some key considerations to take into account which apply to both surgical procedures on patients and human cadavers. As shown in Table 1 , perioperative precautions are necessary to assure the anonymity of the patient or cadaver, while digital measures are essential to protect privacy of the patient/cadaver and faculty. While cadaver surgery is valuable as hands-on training form, we believe that there are also distinct advantages to livestreaming cadaver surgery over surgery on patients; less stress and cognitive load of the surgeon, opportunity for a more elaborate interaction between multiple surgeons and residents, expanding the surgery and increasing surgical time and time for educational purposes, and legal and ethical considerations with regard to possible postoperative complications. Especially during this COVID pandemic, livestreaming for educational purposes from this setting has the great advantage over live surgery on patients of not further compromising scarce OR resources in the clinical COVID setting, nor jeopardizing the safety in the hospital setting by bringing in more people than strictly necessary for patient care. Additionally; COVID might interfere with planned livestream surgery on patients. Bringing live streaming surgery out of the hospital therefore safeguards both clinical setting and education.

There are myriad options to watch and learn from online surgical videos-such as WebSurg, AIS, CSurgeries, MedTube, GIBLIB, and YouTube-which are clearly appreciated by trainees [4, 13] . There is little research to the quality of the videos, although most dedicated surgical education sites offer some kind of quality assurance, accreditation or peer review of uploaded videos-as opposed to Youtube® which is completely open source. Three studies assessed the reliability, quality, accuracy, and satisfaction with regard to videos of the critical view of safety, inguinal hernia repair, and sleeve gastrectomies on Youtube®, Websurg®, and GIBLIB® [14] [15] [16] . All authors agree that while the opportunity and satisfaction these videos offer is significant, their quality is often less than appropriate-even the videos which were reviewed and prepared by clinicians. Especially information with regard to pre-and postoperative processes is often limited, and there is no significant superiority of GIBLIB® and WebSurg® over YouTube®.

With the exception of Giblib, the aforementioned sites are free to use, although with the exception of Websurg and Csurgeries all others require an account to get access to the videos. While these videos offer a wide range of surgical procedures, they lack the live immersive interaction which differentiates livestreaming surgery. Residents indicated that this interaction was highly appreciated, and we believe that this interaction is essential to align the contents of surgical videos to the knowledge and questions of the audience, and transforms a passive movie-watching occasion into an active learning experience. Combining prerecorded videos with a live discussion could be a possible alternative. An advantage this application would be lower costs and a wider applicability without the need for advanced technology. To best mimic the interaction with the performing surgeon like in cadaver surgery, preferably the surgeon or surgical team that recorded the video takes part in the discussion. However, zooming in on a certain part of the procedure, literally or by expanding the surgery or surgical time-one of the key advantages of cadaver surgeries in our study-is not possible in this application of surgical videos. We and previous authors did not measure the impact of livestreams on the clinical performance of residents and efficacy, which limits our study and the available evidence [8] [9] [10] . Future studies should focus on the quantitative evaluation of the effects of video based resources and livestreams on resident performance. However, it should be taken into account that these resources are complementary to the indispensable hands-on training-not a replacement-and should be researched and evaluated accordingly. Moreover, a problem when evaluating complementary training resources such as videos and livestreams, especially in a nationwide setting, is the variability in previous exposure of residents, and the unworkability of withholding residents from further training during a trial. This makes quantitative evaluation in for instance a randomized trial nearly impossible and introduces bias and confounders in practically all types of quantitative research. Another limitation is the response rate to the surveys of only 23-38%. Also, we did not collect information whether the respondents were new participants or had joined a previous livestream, partly because of privacy regulations. This may have impacted the response rates of the surveys. Lastly, the surgeries performed were quite heterogeneous. The cadaveric model may be more or less effective for the given procedure. We did not evaluate this specific aspect. Future evaluations should focus on learning effects among different procedures.

The impact on clinical performance needs to be quantified to consolidate the role of livestreams in surgical education, and should therefore be subjected to future studies. The COVID-19 pandemic has made clear how vulnerable surgical resident training is due to the dependability on surgical caseload [2, 17] . Digital training resources such as livestreams, virtual/augmented/mixed reality, and simulators may become indispensable in providing sufficient surgical exposure to residents-although they will never replace the need for hands-on operating room experience. By combining the best of both the physical and digital world in blended learning, hybrid curricula can be established to future proof surgical education and to train proficient future surgeons.

Livestreaming of surgical procedures proved a feasible and very well appreciated augmentation of resident education during the COVID pandemic. In times of social distancing and limited surgical caseload, interactive, immersive livestreaming added value to traditional surgical training. Digital resources show great promise in providing blended learning environments that allow for sufficient learning experiences for residents that do not depend on clinical exposure.

SAGES and EAES recommendations for minimally invasive surgery during COVID-19 pandemic

The perspective of surgical residents on current and future training in light of the COVID-19 pandemic

Vascular surgery education during COVID-19 pandemic

EAES online educational resources: a survey of the membership of the European Association for Endoscopic Surgery (EAES)

Practical techniques to adapt surgical resident education to the COVID-19 era

Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review

NVvH (2020) Opleidingsplan heelkunde: structuur curriculum heelkunde voor reflectieve professionals (SCHERP)

Livestreaming surgery for medical student education-educational solutions in neurosurgery during the COVID-19 pandemic

Untethered and HIPAA-compliant Interactive Livestreaming of Surgery to Residents and Medical Students

Teaching Operative Surgery to Medical Students Using Live Streaming During COVID-19 Pandemic

Ethical issues surrounding the use of images from donated cadavers in the anatomical sciences

Social media guidelines for anatomists

Use of laparoscopic videos amongst surgical trainees in the United Kingdom

The educational quality of the critical view of safety in videos on youtube(R) versus specialized platforms: which is better? Critical view of safety in virtual resources

Comparison of new era's education platforms, YouTube(R) and WebSurg(R), in sleeve gastrectomy

Evaluation of the reliability, utility, and quality of information used in total extraperitoneal procedure for inguinal hernia repair videos shared on WebSurg

Slowdown of urology residents' learning curve during the COVID-19 emergency