key: cord-0885852-8k0vxkjs authors: Slater, Bethany J; Kashyap, Meghana V.; Calkins, Casey M.; Powell, David; Rothstein, David H.; Clifton, Matthew; Pandya, Samir title: Global dissemination of knowledge through virtual platforms: Reflections and recommendations from APSA/IPEG date: 2022-01-15 journal: J Pediatr Surg DOI: 10.1016/j.jpedsurg.2022.01.006 sha: 6d3640dd13613e536d43c6608e2fc86be4dab46f doc_id: 885852 cord_uid: 8k0vxkjs BACKGROUND: The COVID-19 pandemic forced the cancelation of conventional in-person academic conferences due to the risk of virus transmission and limited ability to travel. Both the American Pediatric Surgical Association (APSA) and International Pediatric Endosurgery Group (IPEG) converted to a virtual format for their 2020 annual meetings. The purpose of this article is to review the successful implementation of the APSA and IPEG virtual meetings and reflect upon lessons learned for future virtual conferences. METHODS: Logistics, structure, and attendance statistics were reviewed. Informal interviews were conducted with key stakeholders and the number of presenters and participants were analyzed. Finally, post-meeting attendee surveys were conducted to elicit feedback after both virtual meetings. RESULTS: The meetings were organized in different ways, with APSA spreading a mix of scientific and clinical educational content over several months and IPEG keeping the meeting compressed, similar to previous in-person versions. Both meetings were free and therefore attracted a high proportion of participants (720 for APSA and 834 for IPEG). The meetings were felt to be educationally appropriate by most, although timing and lack of Continuing Medical Education (CME) opportunities were detractors. Most attendees said they would be willing to pay fees similar to in-person amounts. IPEG compressed presentations into four 2-hour sessions spread over 4 weeks, but also made material available on-line through a proprietary application. There was a broad range of international attendees. IPEG attracted a larger percentage of non-members than did APSA (3:1 nonmember to member ratio). Both societies reported net losses, largely due to lost registration revenue and non-refundable costs from having to switch from an in-person meeting. CONCLUSIONS: The main advantage of the virtual meeting was increased participation while disadvantages included the lack of networking. The key lessons learned from the meetings include methods to increase interactivity, adjustments of technical logistics, and creation of enduring material. In the future, hybrid conferences will likely become more prevalent with advantages of both platforms. LEVEL-OF-EVIDENCE: Level V - Expert Opinion The first virtual medical society meetings were held in the mid-1990s with limited audio and video capabilities.(1, 2) Digital conferences have become more prevalent in the last decade. (3, 4) The year 2020 brought unprecedented challenges, with the SARS-CoV-2 pandemic forcing the cancellation of conventional, in-person academic conferences. Due to travel restrictions and concerns of viral transmission, medical societies cancelled, postponed, or converted to a virtual format their annual meetings. (5) (6) (7) (8) (9) Many surgical societies including the American Pediatric Surgical Association (APSA) and the International Pediatric Endosurgery Group (IPEG) were affected by the COVID-19 travel restrictions and lockdowns. Originally, in-person events had been planned in Orlando, Florida for APSA (May 13 -17, 2020) and Vienna, Austria for IPEG (June 10 -13, 2020). APSA reorganized its annual meeting into 46 specific virtual sessions spanning between May 14 -July 2, 2020. IPEG converted to a virtual format but reorganized the live content to span four 2-hour virtual sessions from September 11-28, 2020. The annual meetings of both APSA and IPEG are two of the largest meetings targeting pediatric surgeons. The goal of the APSA annual meeting is to cover the breadth of pediatric surgery and communicate the latest research findings, clinical discoveries, and trends that influence the day-to-day practice of pediatric surgery. For IPEG, the main goal is to discuss challenges of dissemination of minimally invasive surgery to international pediatric surgeons. The structure of the virtual meetings was chosen to align the meetings with these missions, and they were converted to a virtual format on short notice. To analyze and summarize lessons learned from this sudden change, as well as to construct a framework for the organization of future virtual conferences, we undertook a descriptive study of survey data and available analytics from APSA and IPEG's virtual annual meetings. Logistics, structure, and attendance numbers were reviewed from the APSA and IPEG virtual meetings. Informal interviews were conducted with key stakeholders who organized and led the virtual meetings from both organizations. In addition, the number of presenters and participants were analyzed. Performance evaluation of the video content was based on the number of impressions, plays, and embeds. Finally, post-meeting surveys were conducted to elicit feedback after both virtual meetings. This study was reviewed by the UTSW Human Research Protection Program (HRPP) and deemed not to require IRB approval or oversight under 45 CFR 46.102 The APSA leadership and program leads decided to utilize a proprietary teleconference platform, GlobalCastMD (Cleveland, OH). Presenters recorded and submitted their talks. The subsequent discussion sessions were also pre-recorded. All video recordings were then curated, edited, and finally published via the teleconference platform. The total time spent for this process was approximately 5-6 hours per session. While the scientific content was all pre-recorded, the educational content did include live discussion. The virtual platform was run off a frontline system that could switch between recordings, live sessions, and advertisements. A backup system was utilized in the event of technical failure. A "green room" was created utilizing commercially available software, ZOOM (Zoom Video Communications, Inc., San Jose, CA) in which presenters and moderators tested their audio and video quality. Weekly debriefing meetings were held following the sessions. During the early stages of the virtual sessions, it was noted that attendees were not logging into the conference on time, so a virtual tailgate was started. This entailed trivia questions through the chat room with prizes awarded for those who arrived on time and answered the questions correctly. It was intended as an educational way to introduce the upcoming session and engage the viewers electronically. There was also a drop off of viewers during breaks between sessions, so a halftime-show with a summary of the presentations was instituted to maintain engagement and transition to different topics. They offered an entertaining way to highlight important lessons imparted during the educational and scientific sessions. The digital nature of the conference allowed for this week-to-week flexibility. Meeting Structure APSA conducted meetings on Tuesdays and Thursdays over a two-month period. The meeting contributors comprised presenters of scientific abstracts (80%), invited speakers (10%), and educational experts in myriad subject areas organized by the committees (10%), with a total of 560 contributors. The Tuesday sessions were scientific forums with abstract presentation, while Thursday sessions focused on clinical education. Despite the option to withdraw from the virtual meeting, 96% of oral presentation and 84% of poster presentation authors accepted the invitation to present at the virtual session. Poster presenters recorded a two-minute summary with no discussion. Content from all sessions could either be viewed live or later from the website as the sessions could be accessed on demand at any time thereafter. The APSA leadership elected to not charge for meeting registration and opened the meeting to any provider with an interest in pediatric surgery. Nearly 4,000 individuals registered for the meeting although only 720 ultimately viewed content. Of those, 285 (40%) responded to a 21-question program evaluation. APSA members comprised half of the respondents, with international non-members making up the second largest group (28%). Most respondents were in active practice, with one third of respondents in their first 10 years of practice. For many respondents (40%), 2020 was their first time attending an APSA annual meeting (Supp. Fig. 1) . The top three reasons why survey responders attended the meeting included educational opportunities, affordability (zero cost to attend), and professional networking. However, timing and duration of the conference posed the greatest obstacle for attendance and participation. Of those who responded, a little over 50% would be willing to pay the same registration fees as for an in-person meeting (Fig. 1) . In addition, 90% of responders had to use personal time to view the sessions, and only 59% of respondents were satisfied with the length and format of the meeting. When asked to comment on the ideal duration, there was a wide range of responses from 1-3 days (most preferred) to being spread out over 4 weeks. In addition, there was no consensus on the ideal length of sessions per day with approximately half favoring between 1-2 hours per day and half preferring 4-5 hours per day. Technical challenges affected about onethird of respondents, and the absence of CME was viewed as a significant deterrent to 44% of the survey responders. CME was not offered due to the short timeline for conversion to a virtual meeting. Finally, while many of the sessions were rated as valuable, the digital poster session and exhibit halls were the features least utilized (Supp. Fig. 2-3) . Although post-tests were not employed to measure the educational impact of the virtual meeting, some of the survey questions assessed this. Eighty-nine % of responders felt that the virtual meeting achieved its educational goal. In addition, 74% stated that they learned new techniques or services that could be incorporated into their practice and 70% indicated that their medical knowledge improved as a result of attending the meeting. The main differences from the APSA 2019 survey results with similar questions were a larger proportion of APSA members attending in 2019 as compared to 2020 (66%), a smaller group of first-time attendance (13%), and a higher number of participants viewing the exhibit hall. Informal interviews were conducted with key leadership personnel for feedback about the meeting as well. The leadership were unanimously pleased with the meeting. The positive elements of the virtual format included the global outreach achieved, the speed by which the meeting was converted to a virtual format, the teamwork of the planning team, and the flexibility of viewing the meeting content. The IPEG leadership and program leads decided to proceed with a similar approach with some notable differences. First, content was trimmed to a core nucleus of scientific and technique driven material, and all expert panels were rescheduled for the next annual meeting. Second, scientific content was divided into three categories: poster, video, and podium. Poster authors were requested to convert their material to a digital format to be exhibited on the IPEG website. Technique videos were disseminated via a third-party educational app (StayCurrent: Pediatric Surgery) available on the Apple App Store (Cupertino, CA). Videos considered suitable for presentation at the meeting were then published via the mobile app and email blasts at a rate of three videos per week. A total of 13 weeks were required to disseminate these videos. The two-pronged distribution served dual purposes. First, by sharing the content via the app, it was made accessible to all subscribers to the app and not just IPEG members. The intent was to potentially capture and entice non-members to join the society. Second, by sharing the videos via a weekly email blast, members who did not have access to the app on their mobile devices could still see the content. For the podium presentations, authors prerecorded their presentations. Four two-hour sessions were broadcasted with the discussion sections held live and unedited. Zoom was used as the virtual meeting platform. Strict adherence to a timeline with limits on overall session length was essential to prevent the meeting from running past the intended length. The IPEG podium presentations consisted of four two-hour "virtual sessions" which were all pre-recorded and broadcast on a commercially available platform. For the IPEG virtual sessions, a large panel of diverse discussants facilitated questions and discussion live during the sessions. Additionally, most of the work presented included at least one of the authors present for the live discussion portion. These were unscripted and unedited to simulate an in-person meeting. The total number of IPEG registrants was 834 with a 3:1 nonmember to member ratio. The attendance rate per virtual session was 58% (range, 34-83%) and average duration per attendee was 126 minutes (range, 99-162 minutes). 94% of participants listened for greater than 5 minutes and 41% stayed for at least 75% of the session ( Table 1a) . The average number of registrants for live events was 541 (359-701). The average number of actual attendees was 226 (188-284). The live attendance increased in the last two sessions (335) as compared to the first two sessions (223). This may be due in part to the increased marketing by social media and email blasts after the second session. Participants registered from 123 different countries and Antarctica was the only continent from which there were no participants. Video content was viewed on average 117 times (1-390) with an average of 57 plays (0-223) and 57 embeds (integration of the video into social media or other web media (0-151) ( Table 1b) . It is important to note that there are significant cost considerations associated with annual conferences. For APSA, as with many national organizations, its financial liquidity is limited in the months preceding the annual meeting. It historically has operated at a net loss for the inperson meeting despite the registration fees. The approximate cost for digitizing the 2020 virtual meeting was $25,000 not including the time spent editing the prerecorded presentations and discussions. The additional cost of digitizing, loss of revenue due to free registration, and nonrefundable costs incurred during the planning of the in-person meeting, resulted in a significant financial loss for APSA. For IPEG, the cost of digitizing the meeting was limited to that of a professional Zoom account and the loss of registration fees. However, the virtual meeting also resulted in a loss for IPEG due to the costs already incurred from vendors for the in person meeting as well as management fees. The main impetus for virtual meetings in the past centered on the potential environmental benefits by decreasing carbon emissions from travel, as well as increased affordability and accessibility. (2, 10) In addition, removing the requisite travel for conferences may encourage a broader audience, both nationally and internationally, while minimizing the expenses associated with travel and lodging. There has also been a presumptive democratization of scientific discourse over the internet, but skepticism remains whether the in person aspects of social interaction can and should be replaced. (4) As with many conferences set to be held in 2020, both APSA and IPEG's annual meeting planning were well underway for an in-person event. As the pandemic spread, institutions, states, and countries imposed travel restrictions and it soon became clear that inperson meetings could not be held. The governing boards of each society met independently and decided to convert to a virtual meeting. APSA kept its original date which gave the meeting planners approximately four weeks to convert to a virtual meeting. IPEG decided to change to a later date, allowing the potential to keep the in-person meeting or have more time to plan for the conversion. For both meetings, the goal was to take advantage of current technologies and to deliver a product that met the quality standards of previous meetings. The two societies addressed the conversion with two different approaches. The greatest benefit of the virtual conference was gained through interfacing with an international audience. The global reach through the internet, combined with the free registration, attracted a wider viewing audience than would have been possible for an in-person meeting as evidenced by the 40% first time attendees at the APSA meeting. Virtual meetings allow for the dissemination of medical information. Students, residents, and even junior faculty also greatly benefited from the virtual format, allowing viewing without missing clinical or personal duties. In addition, because the scientific sessions were deployed to a single audience, presenters had greater exposure to attendees who may not have seen their presentations in a breakout room in an in-person meeting. The virtual environment and chat function facilitated better engagement by providing a less intimidating forum for participants to pose their comments and questions, as opposed to the typical paradigm of a microphone in front of a large audience. Due to the scientific sessions being prerecorded, discussions were more tailored and poignant. From a technical standpoint, virtual platforms do allow for breakout sessions, but this was not utilized for the main APSA or IPEG events. In addition, capitalizing on the ability of more trainees to attend, a student and resident session was also held during the APSA meeting. The other technologic advantage was that all sessions were recorded allowing for on-demand viewing later. There was a 10-fold higher number of viewers of the archived presentations than those able to watch live which has been described for other virtual meetings. Finally, the most significant advantage virtual platforms have over in-person meetings is flexibility. There are often limitations of sessions at in-person meetings due to both physical restraints of room availability and size as well as time constraints.(1) Technology is scalable and a virtual audience need not be capped. The APSA sessions were also shortened over the course of the meeting due to criticism about the length. IPEG increased the panel members during the sessions to allow for more interaction during the live sessions. The flexibility also allowed for impromptu live sessions. Due to the civil unrest that arose during the time of the APSA conference, the virtual format allowed for adjustment of the actual content to address this subject. Overall feedback on the virtual meetings was positive, but there were many opportunities for improvement (Supp. Table 1 ). The most consistent criticism of APSA's meeting was the timing, length, and duration of the sessions. Most felt that seven weeks was too long, and this was realized by falling attendance numbers. The virtual setting did allow for real time modifications, and sessions were shortened after initial feedback. However, these time constraints made it difficult to use discussion from the chat box for the live broadcast. In addition, the timing of both meetings was inconvenient for viewers on the West coast and abroad. This has been encountered in other virtual conferences as well. (8, 9) Given the lastminute reorganization, agendas for attendees were prepared on short notice and this prevented viewers from planning to attend with adequate forewarning. From a presenter perspective, prerecorded virtual sessions shortened the preparation time as submission deadlines for pre-recorded sessions were adjusted to reflect the need for reviewing and preparing content. The most consistent request after IPEG's meeting was dual language interpretation. As an international society, while the official language of the meeting is English, participants noted that the virtual format may provide an opportunity to provide real time interpretation. The main disadvantage of a virtual meeting is the loss of the ability for professional networking. Although an attempt to mitigate this was made with additions of a social lounge and chat function, there is an unquantifiable effect of direct contact between trainees, junior faculty, and leaders. A great deal of business is accomplished between sessions that is not defined under the strict definition of "business." This was manifested most notably in the lost benefit of live job interviews and discussions tailored to pediatric surgery fellows. In the past, both meetings provided a venue for pediatric surgeons and their families to gather once a year. To many, an annual meeting is diminished without in-person social gatherings and the allure of travel. Technical challenges are certainly more possible in the virtual setting. APSA's virtual platform ran on a frontline system that could switch between recordings, live content, and advertisements. Despite utilizing a backup system, one session had to be run off a mobile device. These behind-the-scenes obstacles were not noticeable for most of the conference, however, the first day of the APSA conference had the most significant technical failure. The polling function used up the server's capacity and caused 20% of viewers to encounter a system crash. Simultaneously, the support system malfunctioned, so attendees were unable to re-engage with the live session. The polling function was subsequently abandoned for the remainder of the conference to mitigate this technical risk. Other technical considerations, such as compatibility issues between ZOOM and GlobalCastMD needed to be addressed on a weekly basis given the relative lack of experience with this format. Finally, there were features of the virtual platform that were underutilized. Poster presentations were poorly viewed, and the consistent lack of live interaction was seen as a barrier. There were resources attached to sessions with little to no views by attendees. Noteworthy for future sponsorship considerations, the virtual exhibit hall feature was also not used by most attendees (Supp. Figure 3) . Committee meetings were not held at either meeting, though they could be in the future with breakout sessions. Additionally, the short timeline of conversion to virtual prohibited the approval of continued medical education credits for both meetings. There was some overlap between the planning committees of these two pediatric surgery organizations and as such there were similarities in the structure of the two virtual meetings. Both virtual conferences featured free registration. In addition, both associations designated moderators to manage the chat function in the virtual platforms. However, there were some lessons that were learned from APSA's meeting that were applied to IPEG's meeting. The main differences between the APSA and IPEG meetings centered on APSA utilizing more prerecorded content while IPEG displayed more live content. Although some of this design was planned initially, IPEG made the virtual sessions more interactive with larger panels after a fall off in viewership was noted during the APSA sessions. This translated into less work done upfront as well as more flexibility with increased engagement for the IPEG meeting. However, the increased potential for technical difficulties and bandwidth problems are an important consideration in this model. Fortunately, no major technical issues occurred. Finally, APSA used the GlobalCast platform and IPEG used the Zoom platform to conduct the meeting. Based upon the lessons learned from APSA and IPEG's first virtual meetings, several recommendations for implementing virtual conferences or virtual components during annual meetings were compiled ( Table 2) . Rubinger et al. have created a roadmap that identifies four key phases of the meeting cyclepre-planning, planning, accomplishing, response and engaging (PrePARE). (11) The primary elements in the pre-planning phase are defining the organizing committee, stakeholders, target audience, and type of meeting. Clear communication of the vision allows all team members to work towards the same final product. (5) The planning phase is the most cumbersome. Features specific to the virtual platform consist of ensuring speakers have highquality hardware, adequate internet connectivity, and assuring that presentations all have standardized templates. Timing considerations are the most difficult and should consider attendance for most of the audience. As new technologic innovations arise, virtual networking may be more consistent with in-person networking through gaming software and avatar environments where participants can walk around and engage in video conferences spontaneously. The next phase is accomplishing conference goals. This involves designation of hosts and moderators who can be active and engaging, and a sense of humor helps as was noted during the impromptu halftime and post-game shows at APSA's meeting and the larger panels at IPEG. Being prepared for unexpected disruptions is also of paramount importance. Backup sessions can be pre-recorded and substituted during large group events. Regular debriefing sessions should be held by organizers to make real-time changes. Technical preventative measures such as green rooms, random access codes, and close monitoring by administrators should limit hacking and provide quality control.(1) Finally, post-meeting materials such as handouts or recordings for delayed viewing, should be tailored to the target audience. Current technology and artificial intelligence allow for indexing content to make it more searchable and accessible both during and after the event. For global audiences, utilizing live and print translation services would ensure the content disseminated is understood by the wider audience.(1) The final phase is gauging the response to the virtual conference by post-meeting evaluations and engaging the target audience for future meetings by utilizing the survey results to improve upon successes and eliminate failures. In addition, polls could be utilized during the sessions to measure the educational impact of the virtual meeting. Other educational assessment tools could be integrated in the digital format. Hybrid approaches seem to bring the best of both worlds. This could involve small, conventional conferences locally with streaming, a scaled down in-person meeting for presenters with a combination of live streaming and recorded videos, or even asynchronous question and answer segments after dissemination of pre-recorded videos. Ultimately, after the success and global dissemination of content from the 2020 virtual meetings, it may prove difficult to return to a completely traditional in-person approach if broad satisfaction of members and potential attendees is considered. Unanswered challenges include how to best provide networking and career advancement opportunities in the virtual arena. Should traditional abstract submissions be converted to video submissions? This would create more work up front for presenters but could be accomplished if there is the ability to deidentify the video submissions. There will always be those who are reluctant or unable to use computers and newer technologies, as demonstrated by many individuals who utilize the call in feature of ZOOM.(1) Many virtual conferencing technologies are under development or have improved with the increased demand, which may provide a solution. (12) Another question is how to keep people engaged while watching a computer screen given "ZOOM fatigue". Increasing the interactivity and using gaming environments may be the answer in the future. The rising role of social media in content distribution may provide an added layer of engagement. APSA and IPEG were able to create successful virtual meetings. The main lessons learned from the meetings include methods to increase the interactivity, adjustments of technical logistics, and creation of enduring material. In the future, when travel restrictions are lifted, hybrid conferences will likely become more prevalent. With a combination of virtual and inperson sessions, the advantages of both modalities can be implemented, such as increased participation and enduring content with the ability to provide networking and social opportunities. Virtual congresses Virtual conferences First Virtual Live Conference in Healthcare The scientific importance of asking questions at meetings: why virtual debate is not enough Ad astra per aspera (Through Hardships to the Stars): Lessons Learned from the First National Virtual APDS Meeting Annual Virtual Meeting: One Small Step for the AOFAS Scientists discover upsides of virtual meetings Transitioning to Virtual: ACMT's 2020 Annual Scientific Meeting Use of the Internet to Facilitate an Annual Scientific Meeting: A Report of the First Virtual Chest Wall Injury Society Summit Virtual meetings: A critical step to address climate change Maximizing virtual meetings and conferences: a review of best practices Acknowledgements: Marina Petrulla, LeAnn Clark, Viera Ewell, Jaqueline Narvaez, John Waldhausen, Jay Vacanti, Todd Ponsky Funding Sources: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. pandemic has shifted society meetings to a virtual format, with many recent articles describing initial experiences. • This article highlights the conversation to a virtual annual meeting for two pediatric surgery societies, APSA and IPEG, and uses interviews, survey data and online metrics to provide a guide for future meetings utilizing a virtual platform to disseminate pediatric surgical education.