key: cord-1026322-zdzs0e0x authors: Czmielewski, Christian; Gallina, Victoria; Tripoli, Dylan; Lenards, Nishele; Hunzeker, Ashley; Zeiler, Sabrina title: Analyzing changes in radiotherapy treatment planning error reporting during the COVID-19 pandemic date: 2022-04-11 journal: Med Dosim DOI: 10.1016/j.meddos.2022.04.001 sha: 156d3d90eade597306aeec9d84871d98993b87da doc_id: 1026322 cord_uid: zdzs0e0x The 2019 coronavirus (COVID-19) pandemic has affected medical physics and radiation oncology departments and the delivery of radiation therapy. Among the changes implemented in response to the onset of the pandemic was a shift to remote treatment planning by healthcare institutions. The purpose of this study was to determine whether the overall frequency of errors changed after the implementation of remote radiation therapy treatment planning during the COVID-19 pandemic. Reported incidents were obtained from an incident reporting database operated by a multi-site cancer care facility in the Northeast. Researchers compared the frequency of reported events in a period prior to the start of the pandemic (March 2019 - February 2020) with a period after the onset of the pandemic (March 2020 - February 2021). No significant increase in reported incidents was detected suggesting the efficiency and safety of remote radiotherapy treatment planning. Errors within the field of radiation oncology are inevitable as it is a complex field with many different jobs, technologies, and processes working together to deliver safe and accurate treatment. Advances in technology have helped to reduce errors but not changed the fact that it is a complex environment with high error probability. 1 While most errors within the field are minor, clinically significant errors potentially detrimental to patient health are possible. An article published by the New York Times in 2010 brought radiation therapy accidents into the national spotlight, putting safety in radiation oncology under a microscope. 2 The most effective way to mitigate errors is to learn more about their nature, frequency, and severity though incident learning. 3 Since 2010, there have been over 40 publications centered around incident learning systems (ILS). 4 Through this research, it was discovered that all incident reports are equally important no matter the level of severity. 3, 4 Additionally, clinics who report more errors through ILS are more successful in creating a safe treatment environment than those who report less often as they foster a culture which is mindful of safety. 1, 4 One Canadian institution quantified the potential to create a safer clinical environment by reporting a significant reduction (28 -47%) in clinical incidents following ILS implementation. 5 Incident Learning Systems can also provide a valuable source of information during times of upheaval and change. It is possible that unforeseen changes in departmental operations could influence the safety culture and incident rates of institutions. The coronavirus disease 2019 (COVID-19) is categorized by the World Health Organization (WHO) as a global pandemic. 6 The COVID-19 pandemic led to a shift in clinical services and the need for hospitals to socially distance workers from one another when feasible. Specifically, radiation oncology and medical physics departments in cancer centers across the globe were faced with decisions about how to effectively treat patients with the challenge of either postponing radiotherapy treatment or finding ways to ensure clinicians can safely plan and deliver treatments. Like other healthcare departments, radiation oncology centers implemented the use of telemedicine services to provide virtual consultation and care to minimize physical contact between patients and their healthcare teams. 7 During this time, radiation oncologists conducted approximately 92% of visits through telemedicine services. 7 Remote telemedicine services allowed for fewer in-person visits where physical contact was eliminated to complete a radiation oncology consultation. Additionally, clinics sought out ways to adjust staffing to maintain social distancing. Social distancing through the management of 'skeleton crew' shifts minimized further interaction between the radiation oncology team. Pods were created where on-site staffing of medical dosimetrists and physicists were limited to a select small group that rotated on a daily or weekly basis. 8 When on-site presence was not required, all work was performed remotely on virtual desktop access provided by institutions. Remote treatment planning involves accessing the treatment planning system (TPS) and other clinical databases off-site and allows for medical dosimetrists to create radiotherapy plans without physically being present at the treatment facility. The policies put in place to reduce patient and staff interactions modified many of the traditional clinical operations completed daily. Such policies, known as pandemic preparedness plans (PPP) were developed by some radiation oncology facilities. 8 Pandemic preparedness plans were based on many factors with the common goal of achieving safe and effective care to patients. One center in the greater New York area achieved this by focusing on managing staff, implementing telehealth visits, reducing patient volume through categorization of treatment priority, encouraging multidisciplinary discussion, and maintaining a culture of safety. 9 Despite the shift to off-site operations for medical dosimetry and physics departments, plan checks, and physics coverage were fulfilled remotely in accordance with Nuclear Regulatory Commission (NRC) requirements. 8 Setup images that required review by physics prior to administering the first fraction of treatment were accessed through offline review with a virtual desktop. Establishing socially distanced staffing adjustments was not the only way clinics maintained a safe environment for patients and staff. In addition to general healthcare practices adopted by radiation oncology staff as a result of the pandemic, treatment delivery strategies also evolved. Both the American Society for Radiation Oncology (ASTRO) and the European Society for Radiotherapy and Oncology (ESTRO) issued clinical practice recommendations to use as a guide for makeshift and novel circumstances that came with the onset of the COVID-19 pandemic. 10, 11 Recommendations included shorter treatment fractionations and even omission of radiation therapy treatments entirely when necessary and deemed clinically safe. 10, 12 Hypofractionated treatments were used more frequently than prior to COVID-19 in an effort to reduce the protraction of treatment and risk of exposure. With pandemic-based staffing and treatment delivery objectives established, many clinics resumed clinical operations while satisfying necessary treatment demands. However, the abrupt shift to remote treatment planning invited potential risks. The research problem is that as treatment planning in radiation oncology shifted to a remote setting due to the COVID-19 pandemic, safety protocols established for in-clinic workflow may have potentially been affected. Therefore, the purpose of this study was to determine whether the frequency of errors changed with remote radiation therapy treatment planning during the COVID-19 pandemic. Researchers tested the hypothesis that (H1 A ) there was a significant increase in reported treatment planning incidents during the COVID-19 pandemic. A query of events was collected on an incident reporting database operated by a multisite cancer care facility in the Northeast. Within this database, any employee may report events and near misses that have the potential to cause harm. Employees may report events anonymously. Events are classified by type and include a radiation treatment category. Category groups included, but are not limited to, disruption in care, documentation discrepancy, plan adjustments, and contouring/segmenting issue. Incident reports were created by nursing, radiation therapy, medical dosimetry, and medical physics staff and include a brief description of the event, the location where the event took place, the event date, and the severity of the event. Event severity was labeled on a scale of numerical levels from 0-4 where level 0 is a near miss, level 1 is no harm, level 2 is temporary/minor harm, level 3 is permanent/significant harm, and level 4 is death. Severity of events were categorized and compared using a Chi-square test to determine if categorization frequency changed in the 2 time periods. Results from this study yielded the total recorded events within the PC-19 and C-19 time frames. To test the null hypothesis, the mean for PC-19 and C-19 error report values were evaluated against a one-sided alternative. All queried events from the collected time frame were level 0 -near miss or level 1 -no harm. The raw data for this study was recorded by month, frequency, and error level ( Table 1 ). The independent sample t-test was implemented for the 3 variables of frequency, level 0, and level 1 events. Based on the QQ-plots of the residuals (Figures 1-3 ) and the Shapiro-Wilk test (P-values: 0.318, 0.638, 0.865), all data sets are reasonably normal ( Table 2 ). The scatter of sampled data is evenly and closely distributed on the 45-degree reference line, demonstrating a normal distribution of error report frequencies (Figures 1-3) . The mean number of reported monthly events in the study period prior to activation of remote treatment planning (40.25, standard deviation 17.48) compared to mean after activation (42.33, standard deviation 19.50) were not significantly increased (P-value 0.393) ( Table 2 ). The mean number of level 0 and level 1 events were also not significantly increased (Level 0 P-value 0.117, Level 1 P-value 0.753) ( Table 2) . Therefore, the null hypothesis (H1 0 ) that there was no increase in reported treatment planning incidents during the COVID-19 pandemic failed to be rejected. Throughout the COVID-19 pandemic, radiation oncology departments modified traditional patient interaction procedures and implemented remote work to adhere to social distancing protocols. [7] [8] [9] [10] [11] The option for remote work has been available for several years. 13 However, the COVID-19 pandemic may have accelerated widespread implementation that would otherwise have taken place in the future. Despite the sudden onset of remote work, researchers in this study failed to show a statistically significant increase in frequency of reported error incidents within a Northeastern multi-clinic cancer care facility in response to the COVID-19 pandemic. From the data collection, it is reasonable to conclude the newly adopted workflow modifications and implemented PPPs were adequate in maintaining safety in the treatment planning process. Remote work policies coupled with the increase in clinical radiation oncology practices to reduce in-person exposures such as hypofractionated treatments made the PPPs safe. Additionally, the researchers discovered there was no significant increase in reported event severity within the queried time frames PC-19 and C-19. The frequency of level-0 and level-1 events did not significantly increase in each time frame. Through analyzing the insignificant change in incident report severity level, it can be deduced the process of quality assurance of treatment planning was also not affected with the onset of increased remote staffing. Though there was no statistically significant increase in the severity level-0 and level-1 events between PC-19 and C19, the frequency of level-0 events was higher during C-19 in each month except for November 2020. On the contrary, PC-19 reported more months where level-1 events occurred more frequently than level-0 events. There was some variability from month to month, such as January and February 2020 which had a much lower frequency of reported events than in January and February 2021, however no definitive factors for the difference could be found. The data in this study agrees with the results from a 2020 study by Darafshea et al 14 which investigated the effect of remote physics staffing in small, medium, and large clinics in a multi-site facility. 14 The incidents queried within this study hail from medium or large clinics, as defined by Darafshea et al. 14 Medium and large clinics had an easier time adjusting to remote work due to the abundance and diversity of resources and staffing. 14 Remote work was supported through staffing adjustments seen through PPPs and other social distancing strategies employed by radiation oncology clinics. 8, 9 Access to increased staffing and resources also allows clinics to adopt a hybrid on-site and remote staffing model which adheres to social distancing protocols while maintaining departmental presence within the clinic. Nonetheless, the results from this study suggested that the pandemic-based implementation of remote treatment planning had no significant effect in error reporting relating to treatment planning, medical physics, and dosimetry. As a result, it is plausible that remote work for treatment planning staff in radiation oncology is a safe and effective way to create optimal treatment plans within a clinic. Furthermore, remote planning offers opportunities for medical dosimetrists to expand flexible work arrangements allowing for improved social distancing of on-site staff. Finally, it is worth noting the only data queried within this research was incidents relating to dosimetry, medical physics, and treatment planning. Treatment planning in radiation oncology shifted largely to a remote setting due to the COVID-19 pandemic, potentially affecting safety protocols established for in-clinic workflow. The purpose of this study was to determine whether the frequency of errors changed with remote radiation therapy treatment planning during the COVID-19 pandemic. Researchers in this study demonstrated that there was no significant increase in frequency of reported treatment planning related incidences after the implementation of remote pandemic protocols. Additionally, there was no significant increase in the severity of reported events when comparing PC-19 to C-19 time frames. There were limitations of this study. One limitation included the data collection limited to a single institution with all treatment sites located in the Northeast. Furthermore, throughout the study, there were changes in variables between PC-19 and C-19 time frames such as staffing and deployment of PPPs. Pandemic preparedness plans required new training and modifications to PC-19 responsibilities, workflow, and workloads. These variable changes between the 2 time periods could skew correlation through disproportionate incident reporting. Additionally, as the incident reporting is voluntary in nature, it is also possible that staff may have exerted less effort into reporting incidents during the initial deployment of PPPs leading to an under reporting of incidents compared to earlier periods. Chera et al 15 found there was a 16% absolute reduction in event reporting during the COVID-19 pandemic. In addition, COVID-19 hotspot locations were found to have a 33% absolute reduction in reporting. 15 The region of interest in the current research data query largely contained COVID-19 hotspot locations in the northeast cancer facility network signifying that error reporting may have been heavily impacted. Finally, this research limited its data to specific incident reports related to medical physics, dosimetry, and treatment planning. Additional conclusions could be drawn within other areas of the cancer care team by completing an unrestricted analysis of incident reports. While the absolute number of reported incidents did not significantly differ during the 2 time periods, another possible limitation is that the number of new patient visits and treatments decreased soon after the onset of the COVID-19 pandemic. 16 Thus, a more accurate determination of error reporting frequency may be established by using a rate metric such as number of reported incidents per number of treatments delivered per month. Further research is needed to scale the frequency reports to the patient load data. Unfortunately, the researchers of this study did not have access to patient load data and were unable to draw parallels between these metrics. Future studies may query error reports over longer time periods of which include more institutions to acquire more robust data and address the limitations of this study. Improving incident reporting in a hospital-based radiation oncology department: The impact of a customized crew resource training and event reporting intervention Radiation offers new cures, and ways to do harm. The New York Times Incident learning systems Incident learning in radiation oncology: A review The management of radiation treatment error through incident learning World Health Organization Radiation oncologist perceptions of telemedicine from consultation to treatment planning: A mixed-methods study Development and execution of a pandemic preparedness plan: Therapeutic medical physics and radiation dosimetry during the COVID-19 crisis Radiation oncology physics coverage during the COVID-19 pandemic: Successes and lessons learned Practice recommendations for risk-adapted head and neck cancer radiation therapy during the COVID-19 pandemic: An ASTRO-ESTRO consensus statement The impact of COVID-19 on radiation oncology clinics and patients with cancer in the United States Breast radiation therapy under COVID-19 pandemic resource constraints-Approaches to defer or shorten treatment from a comprehensive Cancer center in the United States Telemedicine in radiotherapy: a study exploring remote treatment planning Mitigating disruptions, and scalability of radiation oncology physics work during the COVID-19 pandemic Impact of COVID-19 on Cancer Care: How the Pandemic Is Delaying Cancer Diagnosis and Treatment for American Seniors This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors would like to thank Dr. Sherwin Toribio and the University of Wisconsin-La Crosse Statistical Consulting Center for their guidance with data analysis and display of statistical results for the study. Any errors in statistics or interpretation of data are the sole responsibility of the authors. The authors declare no conflicts of interest.