key: cord-0723726-wzspfkee
authors: Li, Charles H.; Rajamohan, Anandh G.; Acharya, Patricia T.; Liu, Chia-Shang J.; Patel, Vishal; Go, John L.; Kim, Paul E.; Acharya, Jay
title: Virtual Read-Out: Radiology Education for the 21st Century During the COVID-19 Pandemic
date: 2020-05-01
journal: Acad Radiol
DOI: 10.1016/j.acra.2020.04.028
sha: 0f18c1c763085743e7e37fbc366822dd36e3a65d
doc_id: 723726
cord_uid: wzspfkee

Technologic advances have resulted in the expansion of web-based conferencing and education. While historically video-conferencing has been used for didactic educational sessions, we present its novel use in virtual radiology read-outs in the face of the COVID-19 pandemic. Knowledge of key aspects of set-up, implementation, and possible pitfalls of video-conferencing technology in the application of virtual read-outs can help to improve the educational experience of radiology trainees and promote potential future distance learning and collaboration.

T he novel coronavirus pandemic (SARS-CoV-2 virus and subsequent respiratory disease COVID-19) has disrupted nearly every aspect of life across the world, and the full effect that it will have on the American healthcare system, economy, and education is still uncertain. Across the nation, college campuses have cancelled classes and administrators have scrambled to convert courses into an online format in a matter of days to weeks (1) . Although 74% of college campuses have an online learning management system, a recent survey of campus computing revealed that fewer than 15% of classes utilize these systems (2) . Some universities have adapted "e-learning ambassadors" to help professors' and instructors' transition to the new online learning environment (1) .

As part of the response to the COVID-19 health crisis, many medical specialties have sought to limit their exposure by using "telehealth" solutions to interact with their patients. With the technological advances in picture archiving and communication systems (PACS) and tele-radiology services, radiology has already been at the forefront of telemedicine. Prior studies have examined the use of tele-radiology to participate in virtual radiology rounds with pediatric providers (3) and intensive care units (4) , to teach ultrasound guided procedures remotely (5) , and to impact global health (6) . One key focus of the American College of Radiology Imaging 3.0 initiative is to add value by facilitating better patient care (7) . Virtual radiology has rapidly become a key tool to achieving that initiative.

Regarding radiology education, the American Board of Radiology has decided to postpone the May/June 2020 administration of the Core exam for the first time since the new format has been administered, in keeping with the Centers for Disease Control and Prevention guidelines to mitigate spread of disease by social distancing. Beyond the disruption to clinical work, protocols to limit coronavirus transmission among healthcare team members has resulted in significant disruptions to the educational workflow of many radiology training programs (8) .

In our radiology residency and fellowship programs, which serve the Los Angeles County hospital system, the COVID-19 response has required the programs to limit the physical number of trainees on clinical rotations during the day and to shift to a more robust call system, allowing residents to cover for those who may become exposed or sick. Residents have been asked to read studies remotely to reduce exposure even with a significantly decreased total volume of imaging cases. Many nonessential outpatient imaging and procedures have also been cancelled, further limiting educational opportunities. Finally, didactic and multidisciplinary conferences that have traditionally involved the assembly of multiple individuals in one room for lectures, case review, and discussion have also been affected. The combination of these changes has resulted in significant challenges to radiology trainee education.

In this paper, we discuss the steps we have taken to mitigate these obstacles to radiology trainee education, including the preservation of trainee exposure to clinical cases, teaching from supervising radiologists, and safe interaction with colleagues. We also review some of the potential features and tools that a videoconferencing service provides. Finally, we discuss the implementation of such a service to meet the changing needs of our department in the era of COVID-19.

In the last decade, several web-based video-conferencing platforms (VCPs) have emerged to deliver audio, video, and screensharing experience across various devices such as macOS, Windows, Android, and iOS (9À13). The platforms enable users to host webinars, virtual meetings, video demonstrations, videoconferences, and online training. Virtual meeting solutions have long been in use in the business sector with great reported impacts on employee productivity. More recently, there have been numerous applications in the medical community from patient education to multidisciplinary conferences (11, 14) . Specific to the field of radiology, VCPs have historically been applied primarily to didactic education (10), with virtual conference sessions commonly offered both at international and national conferences, such as the Radiological Society of North America, American Roentgen Ray Society, and American Society of Neuroradiology, as well as at local forums. Previous studies discussing the use of remote conference systems have shown positive reviews from both faculty, fellows, and residents, citing convenience and flexibility (12) . Moreover, the use of VCPs has not resulted in any quantifiable decrease in performance on either in-training or board examinations (12) . While webinars and other web-based teaching are not new to the realm of radiology education, its use as a daily clinical teaching tool in lieu of in-person real-time clinical sessions has not been addressed.

In this article, we describe our experience with Zoom Video Communications (San Jose, CA) software (15) , although it should be noted that most other platforms typically offer similar features of audio conferencing, video-conferencing, and screen sharing (11) .

For successful implementation in virtual read-out sessions, an ideal VCP should not only have the option for either whole screen or current application sharing along with audio conferencing, but also be Health Insurance Portability and Accountability Act (HIPAA) compliant. The HIPAA Security Rule offers no exception in terms of video-conferencing and other forms of online collaboration. Zoom claims to provide HIPAA-compliant protection of patient health data and confidentiality via its security architecture and encryption schemes. Importantly, Zoom provides a straightforward mechanism for institutions to obtain a signed HIPAA Business Associate Agreement (BAA), ensuring the protection of health information transmitted to or via Zoom. Vendor selection should ensure compliance with both the technical and legal requirements of HIPAA to avoid potential data breaches of sensitive patient information discussed during virtual read-out.

We have been using Zoom Video Communications software (15) since 2016 to record and digitally archive our weekly Neuroradiology conferences, which are comprised of didactic lectures, case discussions, journal club, and quality assurance reviews. However, the rapidly evolving landscape of healthcare worker safety amidst the COVID-19 pandemic has necessitated the use of video-conferencing in various other ways.

In diagnostic radiology, education conventionally occurs by three methods. The first is self-learning and involves self-motivated study by each resident. The second pedagogical method requires independent resident interpretation of imaging studies and subsequent review with an experienced faculty mentor. In the video-conferencing platform, the host can customize and specify multiple settings for the virtual read-out session. Meetings can be set up impromptu or held at recurring intervals (orange arrow) to create fixed schedule. Extra security options like specifying a password (red arrow) to enter the read-out, as well as only allowing authenticated users to participate (purple arrow) can be chosen. A waiting room can also be utilized, where users logging in remotely wait to be admitted into the actual video-conference, in order to avoid "zoombombing" from uninvited guests. Other default options, such as turning off participant video and muting microphones can also be specified. (Color version of figure is available online.)

The third method, formal didactic conferences, has long served as an important way for trainees to learn, and the Accreditation Council for Graduate Medical Education (ACGME) has mandated a minimum requirement for Diagnostic Radiology residents to fulfill for graduation (16) . The ACGME indicates that each sponsoring institution must provide at least 5 hours per week of lectures and conferences. Apart from lectures within the institution, other examples of didactic education include online videos, whether open access or purchased. Attendance at regional, national, or international conferences also provides ample didactic opportunities.

Our program has transitioned our noon teaching conference and educational lectures to an entirely virtual format (all Figure 2 . wInviting trainees and colleagues. Once a virtual read-out meeting has been scheduled and setup, the link to log in, meeting ID (anonymized), and password can be emailed or distributed to residents and other colleagues for participation (orange arrows). The invitation information may also be automatically imbedded into group calendars once a readout session is scheduled. (Color version of figure is available online.) Figure 3 . HIPAA compliance and security. At our institution, we utilize a HIPAA-compliant enterprise Zoom account to initiate a virtual readout. This requires that the host creating the virtual readout use a two-factor identity authentication (TFA) to initiate and enter the video-conferencing platform. This safeguards patient protected health information and also serves as a measure to prevent unauthorized access to the video-conference. (Color version of figure is available online.) participants are remotely connected) with a VCP that is especially helpful in maintaining the interactive nature of "hotseat" style case conferences. The use of such a platform allows us to continue this type of interactive conference, which forms the backbone of radiology pedagogy, while allowing participants to be physically separate in keeping with public health recommendations.

In addition to didactic lectures, self-learning is a critical component in the education of radiology trainees. Self-learning can be achieved in several ways, most traditionally by reading textbooks. However, alternative methods are available, including test question banks, online cases of the day, and educational exhibits at regional, national, and international radiology conferences, among others (13).

While didactic education and self-learning are valuable, a critical component of residency training is through actual case experience and active "on-the-job" learning in which the trainee typically focuses on reviewing a study independently in order to generate a primary diagnosis and differential diagnostic considerations. Exposure to a sufficient volume of cases is needed to gain the proper foundation of experience. For this reason, the ACGME requires residents to review a minimum number of studies (radiographs, computed tomography (CT), ultrasound (US), magnetic resonance imaging (MRI), nuclear medicine (NM)) to acquire this essential skill set. An important but sometimes neglected part of this learning process is the feedback that ideally should subsequently be provided to the trainee during a read-out session. Multiple studies have shown that feedback is crucial for improving not only knowledge acquisition, but learner motivation and satisfaction, and is one of the most effective methods for improving learner achievement (17À20).

The key features of a conventional read-out include the following: direct two-way communication between the trainee and supervising radiologist, ability to view images simultaneously, potential for all parties to scroll through images, and a capability to annotate the images. To accommodate such a read-out virtually and remotely, an ideal VCP should incorporate all of these key features.

At our institution, we have applied remote video-conferencing to an "active" read-out setting, which has allowed our trainees to maintain their "on-the-job" education similar to that in a conventional in-person read out, while limiting trainee and faculty exposure by practicing social distancing during the COVID-19 pandemic. This application allows residents both physically present at the hospital and others viewing remotely either from home or from other sites to also partake as a group in education that traditionally occurs at the workstation every day.

The following guide details our experience with setting-up and implementing Zoom Video Communications for virtual read-out. However, other VCPs will have similar functions and controls.

Zoom is available on multiple platforms, including desktop and mobile devices. The participants to the Zoom meeting would need to download and install the client for his/her device in order to participate in the session. Once the application has been installed, the participant can join the meeting by using the uniform resource locator (URL) from the meeting invitation or enter the meeting ID/password manually through the Zoom (a) After the host radiologist launches the read-out session, she or he will have the option to share their PACS screen (red arrow) or any secondary screens to the remote viewers. (b) This screen-sharing function can be switched throughout the readout as needed (orange arrows). For example, the read-out session may involve a CT of the abdomen and pelvis for a patient with right lower quadrant abdominal pain with an incidental adrenal nodule. The host radiologist may want to switch the shared screen from the PACS to a webpage with an adrenal washout calculator, so all of the residents can see how the calculation is done. This screen-sharing function can also be given by the host to others who are involved in the readout remotely. For example, for the case discussing the adrenal nodule, a remote resident may bring up a good review article to show others in order to supplement knowledge on this topic. (Color version of figure is available online.) app. At our institution, we have used the microphone and builtin speaker functions on our dictaphones for the purposes of verbal communication within the software. We use both Philips SpeechMike Premium and Nuance PowerMic II dictaphones.

The Zoom application platform allows the host to customize multiple setting when preparing a meeting. The host can create an instantaneous meeting or schedule a meeting for a future date and time. Furthermore, a recurring meeting can also be scheduled, which will automatically run at a given time based on the specified interval. When scheduling the meeting, there is an option to require a password in order to participate in the meeting. There is also an option to create a "waiting room" so the host of the meeting can screen the participants before allowing him/her to enter the meeting. Other default settings, such as turning off participant video and muting microphones can also be specified (Fig 1) . Finally, an invitation to the meeting session can be disseminated. The invitation can be emailed or simply shared to recipients via the proprietary URL or meeting ID number, which are created for each meeting.

The meeting can be accessed by logging into the Zoom application and entering the meeting ID, proprietary to each session, which may or may not require a password, based on the selected host settings. Alternatively, if joining the meeting through the invitation URL, the participant simply needs to follow the link and enter the meeting password, if necessary (Fig 2) . Depending on the organization's implementation of Zoom, the participant will log into the Zoom account. At our institution, we use the single sign-on method, which uses our institution's two-factor authentication website for access. This provides additional protection of patient health information by serving as a safeguard against unauthorized user access to the meeting (Fig 3) .

Once the video-conference session has been established, the many features of the application can be utilized. After the host radiologist launches the session, she or he will have the option to share her/his PACS screen or any secondary screens to the remote viewers. At our institution, we have three-or fourscreen setups for our workstations. While multiple screens can be shared from a single workstation at a time, the images may show up as too small to be appropriately viewed. For this reason, we share one screen at a time. Nonetheless, the screen-sharing function can be switched to a different screen on the host workstation throughout the readout, as needed. We often use this feature to pull up specific case-related journal articles or radiology reference websites during the read-out, as the supervising radiologist often may do during a conventional in-person session (Fig 4) . This screen-sharing function can also be given by the host to others who are involved in the readout remotely.

Apart from selecting and monitoring the shared screen, there are other features that the host can control. The host is able to manage the participants within the meeting, having the ability to regulate entry to the meeting, as well as mute and unmute participants already in the group meeting. Participants also have the ability to "raise their hand" within a meeting, which the host can see. This adds an additional level of moderation and control for the host to maintain order within the read-out session, particularly if there are a large number of participants (Fig 5) .

The participant features are also critical, as the trainee needs to have functions during a virtual read-out that would be available in the conventional in-person training. As viewed by the participant, the host shared screen is minimalistic, so as to maximize the (a) The hosting radiologist can choose an option to manually admit awaiting guests into the video-conference, which may serve as a protective measure against uninvited guests (Fig 2). (b) While a read-out is going on, the host can mute or unmute participants (orange arrows) in order to ask questions and add to a discussion. Trainees participating remotely can "raise their hand" to indicate if they have a question or something to add to the discussion (red arrow). (Color version of figure is available online.) area of the screen. There is a small tab at the top of the screen and a toolbar with a few basic functions at the bottom of the screen (Fig 6) . There is a function to add participants (if permitted by the host), share one's own screen, a group chat tool, recording option, and ability to toggle self-mute and video (Fig 7) .

The options drop-down menu at the top of the participant screen also provides valuable tools. The "annotate" tab allows a user to draw, write text, and add shapes to the host screen. This is particularly valuable if the trainee has a question about an anatomical structure and can also be used as a method for the host to "quiz" the trainee, just as may be performed during a conventional in-person read-out session (Fig 8) . Another helpful feature for the participant is the ability to request "remote control" of the host screen. With this function, the participant gains access to the host screen and can remotely scroll or annotate the host screen, just as would be feasible during a conventional in-person read-out (Fig 9) .

The Zoom application is also accessible on a mobile device. Of course, a limiting factor is the mobile phone screen size, but it is still a convenient way to participate while on the go. Many of the features available on the Zoom desktop version, including annotation tools, may also be used on the mobile device application (Fig 10) .

One of the strengths of Zoom over other VCPs is its ability to function in less than ideal network conditions. Zoom suggests the following minimal network bandwidth requirements: The arrow feature allows a participant to place an arrow on a desired region of interest, which is pointing to the palatine tonsillar tissue on this sagittal CT. This can be helpful during a virtual read-out "quiz" session. (d) Oval annotation allows the participant to circle an area of interest. Coronal CT of the craniocervical junction shows an avulsion fracture of the left occipital condyle (type III occipital condyle fracture). (e) The "free form" drawing feature is also a very helpful tool during the video-conference read-out. A Coronal CT of the craniocervical junction with a soft-tissue window shows an annotation outlining the left alar ligament in association with the type III occipital condyle fracture, as created using the "draw" tool. (Color version of figure is available online.) 

A clear advantage of this video-conferencing technology is that it allows collaborative learning experiences by permitting multiple trainees to view a single study at the same time as the supervising radiologist, with each having the ability to control the screen and scroll through images. Supervising radiologist or host computer regulation of screen access serves as an important security measure. Each participant may also annotate or point to findings while asking questions over audio or in text chat format.

While specifically applicable to the 2020 coronavirus pandemic in which social distancing is paramount, the ability to review cases remotely has multiple other benefits. Significant time savings can be attained by virtual read-out rather than by travelling on-site, especially at institutions with multiple sites on campus or with distant satellite locations. Flexibility of use across various devices, including mobile phones, allows for convenient portability in the presence of reliable Internet. Another potential benefit may arise in the form of trainees on-call seeking help from co-residents, fellows, or faculty who may not have immediate remote PACS access. Having another person, even a colleague, view a difficult study may provide a component of self-assurance for a trainee and with the remote imaging viewing tools, it is not an onerous task.

As useful as video-conferencing can be, there are some disadvantages. Any time new technology is introduced, there will be a learning curve to familiarize the user to the software, which might cause delays in its acceptance into the normal workflow. Other technical limitations including unreliable Internet service leading to dropped connections, choppy video streams, or camera malfunctions, which may also quickly derail the video-conference. Another downside to the potential virtual workflow is that only a single computer screen can be shared per person at a given time, whereas many radiologists typically use three to four screens at once to display necessary information and images. Effort must be made by the host computer or attending to keep the relevant images and series on the shared screen. Overall image quality is another factor that must be considered, as the remote participants are viewing a compressed stream of the host's display, rather than a high-resolution PACS image directly. Additionally, if the host computer is sharing the screen of a diagnostic monitor, the participants' monitors may not be of equal resolution, requiring the inconvenience of viewing smaller images or having to pan around the full-resolution screen image. The combination of these factors might also make virtual read-outs slower than a standard in-person readout, and thus video-conferencing should be used selectively.

A security breach is an important pitfall that should always be considered. Because the link is generated and hosted through the Internet, there have been many reported cases of uninvited guests attending private conferences (21) , which could result in a breach of protected health information. Conscientious deployment of a "waiting room" to screen participants and password protection can maintain confidentiality and aid in the prevention of critical data leaks (Fig 5) .

Given the speed at which Zoom video-conferencing software has been adopted, Zoom Video Communications has received criticism for its lax security standards (22) . For instance, there have been recent reports that data-mining features within the Zoom software allowed users to covertly access another user's LinkedIn data (23) . Although Zoom's security has been marketed as "end-to-end encrypted," this is only true when participants are using Zoom native and web apps (24) . When users join using a device that is not connected through Zoom's communication protocol, such as a mobile phone, the encryption cannot be directly applied. These devices are connected through "Zoom connectors" which do not necessarily offer the end to end encryption that the Zoom native applications offer, though a BAA established with any VCS vendor should mandate the protection of even decrypted PHI flowing through that vendor's infrastructure. Awareness of the encryption systems employed and the history of vulnerabilities exposed and corresponding vendor responses is essential when selecting a vendor and establishing a BAA. We have specifically focused on Zoom in this paper as a current widely deployed solution, not as an endorsement of a specific vendor. We encourage radiologists to take on Before a participant can obtain remote control access, the host will be prompted to approve the request. This serves as another layer of defense against unwanted interruption of the video-conference. (c) Once requested and approved from both the participant and host, the participant screen will show "control" over the screen and is able to scroll through images as if she/he is physically on the host workstation. (Color version of figure is available online.) active roles in the vendor selection process and to carefully consider which VCS provides the level of security required by their local regulations. Finally, regardless of how effective the current technology is, it will never be as good as an inperson interaction. Lack of potential visual clues provided by other people's body language can lead to a loss of engagement and can dehumanize the supervising radiologist to trainee rapport. It can also be harder to engage with someone through a video screen, which leads to decreased focus from vulnerability to outside distractions. Moreover, a small time delay between responses that often occurs with video-conferencing can result in stilted conversations.

While the implementation of video-conferencing for virtual read-out was quickly adopted in light of the COVID-19 pandemic, there is significant future potential for its continued use in the education of trainees. Education can be bolstered by having set timed read-out sessions accessible by trainees at remote sites or on a different service. Video-conferencing a portion of the read-out allows all to participate and have access to the same cases and faculty facilitation, by providing a more homogeneous experience that might be needed prior to taking call or to supplement trainee education.

Even in the current COVID-19 pandemic, there is a continued responsibility for the education of our radiology trainees, which will serve as the foundation for the diagnosis and treatment of patients in the future. Implementation of the available technology at present day will facilitate optimal trainee education, not only in times of crisis, but also in the future with the return to a normal workflow. 

The coronavirus has pushed courses online. Professors are trying hard to keep up

Campus computing 2019: the 30th National Survey of computing and informational technology in American Higher Education

Virtual radiology rounds: adding value in the digital era

Radiology rounds in the intensive care units through a telepresence model

R)) for teaching ultrasound-guided anesthetic procedures in remote place

Remote training and oversight of sonography for human immunodeficiency virus-associated tuberculosis in Malawi

IT infrastructure in the era of imaging 3.0

The Impact of COVID-19 on radiology trainees

A feasibility study of real-time remote CT reading for suspected acute appendicitis using an iPhone

Radiology education 2.0-on the cusp of change: part 1. Tablet computers, online curriculums, remote meeting tools and audience response systems

Web-based conferencing: what radiology educators need to know

Establishment of a simple and inexpensive remote radiology conference

A review of innovative teaching methods

Running an online radiology teaching conference

Zoom Video Communications Inc.. Security guide

Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in diagnostic radiology

How people learn: brain, mind, experience, and school

Feedback in schools. Feedback: the communication of praise, criticism, and advice

Analysing feedback processes in an online teaching and learning environment: an exploratory study

How to design informative tutroing feedback for multimedia learning

Zoombombing': when video conferences go wrong

Zoom rushes to improve privacy for consumers flooding its service

A feature on Zoom secretly displayed data from people's LinkedIn profiles

The facts around Zoom and encryption for meetings/webinars [Internet] Zoom Blog 2020