key: cord-0946055-jlosoyf9 authors: Picard, Christopher; Edlund, Meghan; Keddie, Candice; Asadi, Leyla; O'Dochartaigh, Domhnall; Drew, Richard; Douma, Matthew J; O'Neil, Conar R; Smith, Stephanie W.; Kanji, Jamil N title: The effects of trained observers (dofficers) and audits during a facility-wide COVID-19 outbreak: a mixed-methods quality improvement analysis date: 2021-03-25 journal: Am J Infect Control DOI: 10.1016/j.ajic.2021.03.011 sha: 34ad19e8b49f6df6dcbc237aa9c24b98b47e4d76 doc_id: 946055 cord_uid: jlosoyf9 BACKGROUND: In response to a facility-wide COVID-19 outbreak, our tertiary acute care hospital implemented an evidence-based bundle of infection control practices including the use of audits and trained observers “dofficers” to provide real-time constructive feedback. METHODS: We trained furloughed staff to perform the role of dofficer. They offered support and corrective feedback on proper PPE use and completed 21-point audits during a 4-week intervention period. Audits tracked appropriate signage, placement and availability of supplies (equipment), correct PPE use, enhanced environmental cleaning, along with cohorting and social distancing rates. Audit data was used to provide weekly quality improvement reports to units. RESULTS: 962 separate audits recorded 36,948 observations, over 7,696 observer-hours. The most common errors were with environmental cleaning and PPE use; the least common were with regards to equipment availability and cohorting and social distancing. Mean error rates decreased from 9.81% to 2.88% (p<0.001). The largest reduction, 22.57%, occurred in the category of PPE doffing errors. CONCLUSIONS: Dofficer led audits effectively identified areas for improvement. Feedback through weekly reports and real-time correction of PPE errors by dofficers led to statistically significant improvements; however, error rates remained high. Further research is needed establish if these relationships are causal. In response to a facility-wide COVID-19 outbreak, our tertiary acute care hospital implemented an evidence-based bundle of infection control practices including the use of audits and trained observers "dofficers" to provide real-time constructive feedback. We trained furloughed staff to perform the role of dofficer. They offered support and corrective feedback on proper PPE use and completed 21-point audits during a 4-week intervention period. Audits tracked appropriate signage, placement and availability of supplies (equipment), correct PPE use, enhanced environmental cleaning, along with cohorting and social distancing rates. Audit data was used to provide weekly quality improvement reports to units. 962 separate audits recorded 36,948 observations, over 7,696 observer-hours. The most common errors were with environmental cleaning and PPE use; the least common were with regards to equipment availability and cohorting and social distancing. Mean error rates decreased from 9.81% to 2.88% (p<0.001). The largest reduction, 22 .57%, occurred in the category of PPE doffing errors. Dofficer led audits effectively identified areas for improvement. Feedback through weekly reports and real-time correction of PPE errors by dofficers led to statistically significant improvements; however, error rates remained high. Further research is needed establish if these relationships are causal. Since January 3, 2020, the COVID-19 pandemic has affected over 115 million individuals leading to over 2.6 million deaths globally. 1 The SARS-CoV-2 virus is highly infectious; causing outbreaks in numerous settings including acute and long-term healthcare facilities. SARS-CoV-2 is transmitted by respiratory droplets, aerosols, and contact (fomites). Public health and infection control measures targeting these mechanisms of transmission may help to stemming the spread of the virus. 2 Such steps include appropriate physical distancing, the appropriate use of personal protective equipment (PPE) (including the use of face masks), enhanced environmental cleaning, and hand hygiene. 3, 4 However, these relatively simple interventions can be challenging to implement and operationalize. One study demonstrated that training, understanding, and consistency of cleaning in hospitals is often sub-optimal. 5 Hospital crowding and aging infrastructure often makes physical distancing in healthcare facilities difficult. 6 Staff errors have also commonly been noted during the donning and doffing of PPE. 7, 8 Bundled audit tools have been shown to improve practice with regards to many infection prevention measures, as well as improve guideline-compliant practice. To date, however, many bundles have not included trained observers (dofficers) -who can assist in guiding health care workers in the donning and doffing of PPE with the goal of preventing errors leading to self-contamination. 9 The trained observer role was first developed in response to viral haemorrhagic fever outbreaks, however, observers may have an important role in the management of COVID-19 as well. 10 In response to a facility-wide COVID-19 outbreak at our acute care hospital, we developed an evidence based bundled audit which was performed by dofficers who also offered realtime corrective feedback on proper PPE use. In this evaluation, we present our process for the implementation of dofficer auditors and examine the audit data to determine which items were most amenable to influence through direct observation and feedback. This intervention was conducted at a 310-bed tertiary academic suburban hospital located in Edmonton, Alberta, Canada. On June 21, 2020, several hospital-acquired outbreaks of COVID-19 were declared in the facility, resulting in multiple unit closures. The rapid rise in case numbers (58 facility wide) and furloughed staff led to a complete closure of the hospital to admissions and transfers, and cancelation of all elective procedures and outpatient encounters. Patients remained admitted and could be discharged as appropriate, and urgent surgical procedures for admitted patients continued as deemed necessary by their attending physicians. As part of the response to the outbreak, starting July 10, 2020, the hospital administration team implemented a program to train multiple dofficers and also implemented a combined evidence-based audit tool to evaluate infection control practices of healthcare staff on the multiple outbreak-affected patient-care units. Audits were conducted from July 13-August 11, 2020. Training materials used for dofficers were adapted from the Centers for Disease Control Ebola Virus Disease trained observer programme. 9 Training of dofficers was conducted by eight clinical nurse educators. Training consisted of a 60-minute didactic session that reviewed multiple key topics including PPE donning and doffing, providing corrective feedback, and performing infection prevention and control audits (see Supplement 1A for dofficer training checklist). The final component of training was participation in donning and doffing simulations. Dofficer training competency was confirmed using a quiz evaluation, on which dofficer trainees had to achieve a score of ≥90% (Supplement 2). Active feedback was provided to the dofficer trainee during marking of their quiz. Those receiving a score <90%, were asked to re-review training material and re-take the quiz the next day. The healthcare workers training as dofficers consisted of facility staff re-deployed from their other roles as clinical nurse educators, dieticians, medical device re-processing staff, nurses, occupational therapists, physiotherapists, respiratory therapists, social workers, transition coordinators, unit clerks, and unit managers. The primary role of the dofficers was not to train other healthcare workers but to be direct observers to monitor and actively provide feedback for errors noted in the PPE donning and doffing process when seeing patients with suspect or confirmed COVID-19. Audit training consisted of an orientation to the audit, definitions for what constituted a pass or fail (any breach in a 4-hour period for most items), and how to submit completed forms. Following the didactic sessions, the learning objectives were re-reviewed and staff were offered the opportunity to seek clarification and practice what they learned. Audits were meant to be performed by dofficers of the unit they were deployed to on their shifts. The audit tool (Supplement 3) was developed in collaboration with the hospital's Outbreak Response Team (ORT). The ORT consisted of individuals from the hospital infection prevention and control (IPC) team as well as managers representing all the different departments in the hospital (e.g. Emergency, Medicine, Women's Health, Surgery, Critical Care, etc.). The tool was developed after a comprehensive review of literature. [5] [6] [7] [8] [11] [12] [13] [14] [15] [16] Assessment categories that were supported by this review included ensuring appropriate signage, PPE equipment supplies, PPE use, cleaning, as well as cohorting and social distancing. "Signage" is meant to ensure that posters indicating a patient is on precautions are clearly posted, as well as graphics that take staff through the individual PPE donning/doffing steps are available at each PPE station. "Appropriate PPE equipment" refers to ensuring that PPE stations are set up correctly and supplies readily available and regularly re-stocked. "PPE use" refers to ensuring the appropriate steps for donning and doffing of PPE are used as well as consistent use of continuous masking and eye protection in non-patient areas as well. The "cleaning" category audited whether enhanced cleaning schedules and proper terminal cleans of patient rooms was conducted in accordance with institutional IPC guidelines. Lastly, "Cohorting and social distancing" valuates if patients with suspected COVID-19 were preferentially placed in single rooms, and that appropriate social distancing principles were followed for patients in multi-bed rooms and by staff in non-patient care areas whenever possible. Supplement 4 includes information regarding which specific audit points are categorized within each of these five categories. The audit tool adopted a dichotomous (pass/fail) assessment of 21 items (18 of which were assessed every four hours; three of which were assessed every eight hours). 17 A single breach during the assessment interval resulted in the recording of a fail. All 21 fields had a section provided for additional comment. Staff members working in the capacity of a dofficer were to complete an audit form that recorded the date, auditor name, time of the shift, and the unit on which they were working with all forms submitted to the ORT office at the end of each shift. Audit data were extracted and tabulated using Microsoft® Excel (Microsoft Corporation, Redmond, USA). Dichotomous variables were binary coded and any free-text was copied verbatim. Data analysis was performed using IBM SPSS Statistics Version 25 (IBM Corporation, Armonk, USA). Descriptive statistics are reported as means and standard deviations. Welch's ANOVA with Games Howell post hoc testing were used to compare means for repeat measures. Post hoc testing was limited to items that were found to be statistically significant on ANOVA testing. A chi-square goodness of fit test was performed to examine remaining non-repeat measure categorical variables (namely, whether the difference between observed night-time and day-time audit sheet completions was statistically significant). All testing was two-tailed with predefined significance set at p<0.05. Narrative comments collected during the first week of the intervention period were thematically analysed by two authors. Continuous open coding of the earliest returned audit forms was performed until 20 consecutive audit forms were extracted without generating any additional codes. Primary codes were then grouped into 9 error categories and 14 provider codes. These codes were used for all subsequent coding (Supplements 5 and Supplement 6). Audit forms were assessed by one of the authors who also worked in ORT. Quantitative audit metrics were analysed in real time and dashboards with data visualisations of deficiency types was generated on a weekly basis over the implementation period. Data were disseminated to frontline care providers through a designated ORT member who liaised with each unit. The ORT liaison assisted each unit to develop specific improvement targets with concrete outcomes and specified timelines for improvement. Lack of improvement led to additional follow-up and increasingly prescriptive guidance. Feedback about providers' behaviours were handled by their respective leadership teams. For example, physicians received feedback from the site IPC medical lead, unit staff from their managers, and ancillary staff from their respective departments. Approval for use of the data from this intervention implementation and evaluation was obtained from the Human Research Ethics Board at the University of Alberta (study identifier Pro00103046). All staff who successfully completed training and passed the quiz without remediation became dofficers. In total, 186 dofficers were recruited, trained, and deployed on hospital units to perform audit and feedback. All dofficers completed at least one audit form during the intervention period; some audit forms were completed by more than one dofficer. A total of 962 forms were returned. Over the evaluation period, 36,948 observations were recorded on 962 audit forms ( The most commonly noted error categories were noted with proper PPE use and environmental cleaning ( Table 2 and Figure 1 ). The least common sources of error were in equipment availability and cohorting and social distancing ( Table 2) Nrefers to number of observations in that category; SDstandard deviation. Table 2 . Mean error rates by category. A total of 981 narrative fields were examined. Most staff who offered narrative feedback did so in one of two ways: the first was to briefly describe the nature of the error using one or two words. Typically, these entries addressed the nature of the error and read as follows "nurse did not tie waist tie". The second way narrative feedback was offered was through a thorough description of overall performance which occurred in the additional comments section. These had greater amounts of text and thematic or provider codes (Supplement 8). The implementation of a dofficer combined with an infection control audit were used to identify, evaluate, and communicate potential deficiencies in day-to-day clinical procedures with the goal of positive reinforcement and continuous quality improvement. This was done in the setting of a facility-wide COVID-19 outbreak that led to hospital closure and cessation of all ambulatory, emergency, and elective operations. Through this quality improvement intervention, we were able to identify a number of key areas in which significant improvements were measured over the 4-week time period. Although previous studies have examined PPE donning and doffing error rates and the effect of dofficers on doffing breaches, most studies take place in simulated environments, use small samples, focus on specific tasks, and assess materials not used with COVID-19. 18-20 The majority of studies examining the utility of dofficers are associated with caring for patients with Ebola virus disease (EVD). These studies examined fewer than 50 interactions in simulated EVD settings. 20 Previous research has found that a bundled approach to addressing COVID-19 transmission on a respiratory medicine unit at a Singaporean hospital can be successful. 3 The study implemented a bundle including infrastructure enhancements, PPE audits, social distancing requirements, masking of patients when not eating, and enhanced environmental cleaning protocols. Our COVID-19 bundle included many of these interventions but also recommendations regarding appropriate signage and scheduled cleaning of staff workspaces. Critically, we implemented active, real-time feedback to improve deficiencies which has not previously been described in the COVID-19 outbreak response literature. The major strength of this intervention lies in the large number of audit points that were assessed across multiple departments at all times of the work day, and during all aspects of patient care on multiple unit types (e.g. medicine, surgery, rehabilitation etc.). The bundled nature of the intervention allowed us to examine individual components that were deficient and provide feedback that could be tailored and constructivehighlighting the deficiency while providing positive reinforcement based on the successes. Our intervention has several limitations. Given the rapid need to respond to the facility-wide COVID-19 outbreak, instructional materials for dofficer training were not pre-tested on staff, and instructor standardisation was not formally assessed. Eight instructors delivered training to a range of providers, thus the data was collected by a broad range of observers, which may have introduced heterogeneity in how the audit tool was applied and reported. Furthermore, given the nature of our intervention, we are unable to determine if observed error rates were causally linked to intervention or due to changes in infection rates or fomite transmission. A number of audit forms had data missing or were illegible, which may decrease the reliability of the audit results. Because multiple episodes of PPE donning and doffing could occur in each 4-hour interval, we cannot establish a per episode error rate and our results only reflect whether the intervention yielded an improvement. Likewise, the grouped nature of the donning and doffing process precluded us from determining where in the process the errors occurred (for example, with hand hygiene versus equipment). In conclusion, we implemented a COVID-19 dofficer training program in conjunction with a dofficer-driven audit and feedback tool. Overall, the implementation noted a significant decrease in healthcare worker error rates associated with posting of signage to indicate need for precautions and provision of graphics to provide instruction, PPE provision and station setup, PPE donning and doffing processes, cleaning in patient and staff areas, as well as social distancing of staff and patients when possible. The use of dofficers, aside from helping to decrease donning and doffing errors, may offer additional protection to both staff and patients. Additional research is required to determine if this additional protection is clinically significant and if the model is financially sustainable. Approval for use of the data from this intervention implementation and evaluation was obtained from the Human Research Ethics Board at the University of Alberta (study identifier Pro00103046) Funding This work did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Resources and funding to implement and evaluate the intervention All authors have filled out an ICMJE form to make any declarations. None of the authors have conflicts of interest relevant to this manuscript to declare Christopher Picard -Conceptualization, methodology, formal analysis, investigation, data curation, writing -original draft, writing -review and editing, project administration, resources management Meghan Edlund -Data curation, formal analysis, writing -original draft, methodology, formal analysis, and project administration, Writing -review and editing Candice Keddie -Writing -review and editing, project administration, resources, supervision, and funding acquisition Leyla Asadi -Validation, formal analysis, resources, data curation, writing -review and editing Dochartaigh -Conceptualization, Writing -original draft, writing -review and editing Richard Drew -Writing -original draft, writing -review and editing Writing -review and editing Neil -writing -review and editing Stephanie Smith -writing -review and editing Jamil N Kanji -validation, writing -original draft, writing -review and editing References 1. World Health Organization (WHO). Coronavirus disease (COVID-19) pandemic Scientific brief: Transmission of SARS-CoV-2: implications for infection prevention precautions Containing COVID-19 outside the isolation ward: The impact of an infection control bundle on environmental contamination and transmission in a cohorted general ward Infection Control Guidance for Healthcare Professionals about Coronavirus (COVID-19) Changes in knowledge and attitudes of hospital environmental services staff: The Researching Effective Approaches to Cleaning in Hospitals (REACH) study Bed occupancy rates and hospital-acquired infections--should beds be kept empty? Assessment of Healthcare Worker Protocol Deviations and Self-Contamination During Personal Protective Equipment Donning and Doffing Personal protective equipment doffing practices of healthcare workers Ebola: Personal Protective Equipment (PPE) Donning and Doffing Procedures -your roles as the trained observer Development of a Sterile Personal Protective Equipment Donning and Doffing Procedure to Protect Surgical Teams from SARS-CoV-2 Exposure during the COVID-19 Pandemic. Surg Infect (Larchmt) Impact of doffing errors on healthcare worker self-contamination when caring for patients on contact precautions Frequent and unexpected deviations from personal protective equipment guidelines increase contamination risks Contamination of Health Care Personnel During Removal of Personal Protective Equipment Does size matter? The impact of a small but targeted cleaning training intervention within a paediatric ward. Infection Prevention in Practice Bundling hand hygiene interventions and measurement to decrease health care-associated infections The infection control audit: the standardized audit as a tool for change A care bundle approach for prevention of ventilator-associated pneumonia Self-contamination during doffing of personal protective equipment by healthcare workers to prevent Ebola transmission Design strategies to improve healthcare worker safety in biocontainment units: learning from ebola preparedness Human Factors Risk Analyses of a Doffing Protocol for Ebola-Level Personal Protective Equipment: Mapping Errors to Contamination Effect of an Intervention Package and Teamwork Training to Prevent Healthcare Personnel Self-contamination During Personal Protective Equipment Doffing SARS-CoV-2 RNA detection in the air and on surfaces in the COVID-19 Environmental contamination by SARS-CoV-2 in a designated hospital for coronavirus disease 2019 The association between self-perceived proficiency of personal protective equipment and objective performance: An observational study during a bioterrorism simulation drill How to rapidly design and operationalise PPE donning and doffing areas for a COVID-19 care facility: quality improvement initiative The authors would like to acknowledge other individuals who all contributed to this work in their roles: Kara Schick Makaroff, Alison Morin, Corrine Barret-Rose, Kristy Barry, Michelle Stadler, Tracy Miskiw, Jesse Montgomery, and Ryan Long. We also acknowledge all the staff and patient affected by our hospital closure as a result of the COVID-19 outbreak.