key: cord-0897789-61k4a9lw authors: Kim, Bo; Petrakis, Beth Ann; McInnes, D. Keith; Gifford, Allen L.; Sliwinski, Samantha K.; Smelson, David A. title: Applying after action review to examine residential treatment programs' responses to COVID‐19 date: 2022-04-13 journal: Int J Health Plann Manage DOI: 10.1002/hpm.3475 sha: b0291b885b3f6a0c18d8731efa523f12364151d2 doc_id: 897789 cord_uid: 61k4a9lw Resurgences of COVID‐19 cases are a grave public health concern. Hence, there is an urgent need for health care systems to rapidly and systematically learn from their responses to earlier waves of COVID‐19. To meet this need, this article delineates how we adapted the World Health Organization's After Action Review (AAR) framework to use within our health care system of the United States Department of Veterans Affairs. An AAR is a structured, methodical evaluation of actions taken in response to an event (e.g., recent waves of COVID‐19). It delivers an actionable report regarding (i) what was expected, (ii) what actually happened, (iii) what went well, and (iv) what could have been done differently, and thus what changes are needed for future situations. We share as an example our examination of Mental Health Residential Rehabilitation and Treatment Programs in Massachusetts (a COVID‐19 hotspot). Our work can be further adapted, beyond residential treatment, as a consistent framework for reviewing COVID‐19 responses across multiple health care programs. This will identify both standardized and tailored preparations that the programs can make for future waves of the pandemic. Given the expected resurgences of COVID‐19 cases, the time to apply AAR is now. Resurgences of COVID-19 cases are a grave public health concern. Spikes in infections are happening worldwide, and additional waves of the pandemic are widely predicted. 1 For health care programs, their staff, and their patients to best respond to future waves, there is an urgent need to rapidly and systematically learn from their responses to earlier waves of the COVID-19 pandemic. This need is especially urgent for programs that care for vulnerable populations who are at increased risk during the pandemic, such as those experiencing homelessness and requiring behavioural health care (e.g., for substance use). To meet this need, we applied an After Action Review (AAR) framework to examine the COVID-19 response of national residential treatment programs run by the United States Department of Veterans Affairs (VA). In particular, we examined Mental Health Residential Rehabilitation and Treatment Programs in Massachusetts (a COVID-19 hotspot), characterised by 24/7 staffing for residents, many of whom are experiencing homelessness and are in need of behavioural health care. 2 Hence, a structured review of the programs' response to recent waves of the pandemic was indispensable for planning their effective responses to subsequent waves. AARs are structured, methodical evaluations of actions taken in response to an event (e.g., recent waves of COVID-19); they provide explicit plans to strengthen future responses. AARs have been successfully used on numerous occasions by the World Health Organization and others, to systematically learn from major events (e.g., H1N1 influenza outbreak, natural disasters). 3 We adapted WHO's AAR framework to help VA residential treatment programs prepare for future waves of the COVID-19 pandemic. 3 Appendix 1 outlines the steps to conducting an AAR of a programme's response to COVID-19. Notably, the approach is applicable to different types of residential treatment programs, from standalone programs to programs that belong to a large health care system. Objectives of the AAR: Our objectives were to (i) conduct the AAR steps at a couple residential treatment programs prior to wider use and (ii) generate actionable recommendations that the programs can incorporate into their ongoing handling of the COVID-19 crisis. This work was approved by the VA Boston/Bedford Healthcare Systems' Institutional Review Boards. Context of the AAR: We conducted the AAR steps at two VA residential treatment programs in Massachusetts. (Table 1) , and participating residents had received care at the programs during the early months of the pandemic. Other relevant stakeholders were primarily leadership at the programme, medical centre, and regional VA network levels. AAR process: We conducted semi-structured hour-long small-group discussions for the AAR, following the WHO's 'Debrief AAR' format. 3 The format is appropriate when seeking to conduct sessions that take less than half a day and do not require large amounts of resources. We deemed these criteria to be important to minimise the burden on participants and be useable even at resource-limited sites. The discussions were conducted virtually using Microsoft Teams, and lasted approximately 1 h. BAP and BK co-facilitated the discussions, and SKS led the note taking during the discussions. The discussions were also audiotaped for subsequent verbatim transcription by a professional transcription service. Roles of AAR study team members: BK served as the overall AAR lead, overseeing initiation, planning, and conducting of the AAR. The AAR sessions were co-facilitated by BAP and BK. SKS led the note taking during the sessions. DAS and BAP led the liaising with local and regional VA operations leaders. ALG reviewed the AAR steps for accurate reflection of medical knowledge, and DKM and DAS guided the project's efforts to tightly align to VA's overarching focus on continuous quality improvement for veterans experiencing homelessness and other vulnerable veterans. Assessment of the AAR approach's feasibility: Feasibility outcomes were achievement of (i) one session with staff conducted at each programme and one session with residents, (ii) four to seven participants recruited for each session, and (iii) recommendations resulted from the sessions and deemed actionable by leadership. Key milestones for assessing the AAR approach's feasibility: We assessed whether we can meet milestones within planned timeframes -to design, prepare for, conduct, debrief after, and follow up after the AAR within 2, 4, 5, 6, and 7 months of study initiation, respectively. Specifically, we used our regularly scheduled study team meetings at these five timepoints to assess whether the milestones were met: • Two months into the study, we confirmed that we had set the AAR's objective to identify both (i) challenges encountered during the response and (ii) modifications made to practices to adapt to the pandemic. We also confirmed that, to structure the scope of the AAR, we had applied WHO's most commonly used technical categories of focus 3 for reviewing responses to public health events: for example, coordination and emergency response, communication and community engagement, and case management and countermeasures. • Four months into the study, we confirmed that we had prepared for the AAR by collecting, reviewing, and sharing a common understanding of relevant background information among those to be conducting the AAR. This had involved becoming familiar with each of the residential treatment programme's baseline standard operating procedures prior to the current COVID-19 crisis, as well as reviewing available official documentation of the programme's and its health care system's plans in response to the crisis. • Five months into the study, we confirmed that we had conducted the discussions by introducing the agenda, objectives, scope, methodology, and expected outputs of the AAR to the participants. We also confirmed that we had focussed the discussions on the strengths and challenges of the residential treatment programme's crisis response, as well as new practices that were developed as part of the response. • Six months into the study, we confirmed that we had used debriefings with stakeholders to discuss the need for additional debriefings, especially with stakeholder groups whose members' engagement is key to the AAR's identified follow-up actions. We also confirmed that we had (i) put recommendations into a table that the stakeholders edited after the AAR debrief session, (ii) considered potential relevance of the recommendations for other residential treatment programs, (iii) specified the plan for sharing the recommendations with those programs, and (iv) reflected on the AAR sessions overall for opportunities to improve the methodology for future AARs. • Seven months into the study, we confirmed that we are prepared to discuss with various health care systems the extent to which the AAR approach may be applicable to programs other than residential treatment. We also confirmed that we are prepared to provide consultation to a variety of programs, including residential treatment as well as other programs, that are looking to adapt the AAR approach. • COVID safety: Participants stressed the importance of protective equipment, testing, professional cleaning services, and communal space that allows social distancing. Having an area for quarantining and treating non-hospitalisation-requiring residents with COVID-19 (e.g., negative pressure unit on site) was seen as valuable. Despite infection control measures, staff expressed concerns about their safety and that of their families. Residents who had jobs in the hospital particularly felt that they were exposed to more individuals and risky situations. • Communication: One-on-one conversations were most effective for staff to provide COVID education to residents. Constantly changing COVID-19 information and guidelines made communication challenging. Both staff and resident participants felt that programme and hospital leadership did what they could to communicate amidst these challenging circumstances. • Programing: Continuing the programing and keeping the residential treatment programs open helped maintain the sense of community, which is especially valuable for residents who cannot easily make alternative housing arrangements. The residents are there because they need mental health and/or recovery treatment, so participants felt that treatment groups should continue, even if at lower-than-usual frequencies due to COVID-19. Residents emphasised the importance of keeping programing focussed on recovery and separate from COVID-19-related information sharing. Decreased programing and resulting decrease in structured daily activities were especially difficult for residents who were in recovery from substance use. • Mental well-being: Psycho-emotional support made available to residents by staff was valuable in the evenings, when residents more often opened up about their anxieties and frustrations. Nurses sought additional guidance from after hours social workers to appropriately respond when residents shared their feelings regarding restrictions, masks, and isolation. One-on-one support provided to residents by staff, including coping strategies and anxiety reduction techniques, were appreciated by the residents. • Technology: Participants identified training, devices/Internet availability, and eliminating confusion by using a single telehealth platform as being important. It became necessary to implement virtual modes of communication. Both staff and resident participants described challenges during the initial transition to virtual programing, yet felt that the transition was important to ensure the safety of residents and staff by minimising community spread of COVID-19. A sample of recommendations resulting from the AAR: Recommendations included (i) conveying to residents the reasons for COVID-related precautions/changes, (ii) keeping recovery-oriented programming separate from COVID-related information sharing, (iii) including 'how to use technology' training into program orientation, and (iv) developing safe procedures for family interactions, community activities, and off-site health care appointments. A sample of changes made by programs: Changes made by the programs that were identified in the AAR discussions included (i) the program manager further emphasizing the reasons and details for COVID-related precautions/ changes during all-resident community meetings, (ii) treatment groups being used less for COVID-related information sharing, (iii) installing additional Wi-Fi access points for more reliable virtual connections and consolidating remote programming under one technology platform, and (iv) short passes for family visits being made available when prevalence of COVID in the community is safely low. How the AAR informs subsequent research: We will refine our AAR steps based on this experience and apply them to examining how a variety of residential treatment programs are responding to the continuously changing COVID-19 risk/restriction levels. Based on data from multiple programs, we will identify factors and program procedures associated with successful responses -both at all programs and at subsets of programs exhibiting particular characteristics. Applying the AAR framework will identify challenges that programs and residents faced during recent COVID-19 waves, such as personal distance of 6 feet or more, communal areas used for food preparation and meals, personal protective equipment, and designated screening/quarantine rooms. It will also identify changes made to address those challenges, such as changing therapeutic programing and operating procedures to avoid transmission to other residents and hospital staff. Debriefing staff and residents using this methodology will better prepare programs for working with residents through ongoing and similar future crises (including the expected resurgences of COVID-19). The results will also inform any associated health care systems of the types of organization-wide support necessary for residential treatment programs and other communal housing programs during such crises. We therefore urge other non-VA residential treatment programs to examine their own COVID-19 response using the AAR. Lessons from residential treatment programs' responses to COVID-19 will be far-reaching, if responses of many residential treatment programs across multiple health care contexts, beyond our immediate work, can be examined simultaneously using a common methodological approach like the AAR. Such examination will allow the field to gain insights into the unique challenges that residential treatment programs faced due to COVID-19. Importantly, the insights will be highly applicable to other congregate (or 'project-based') housing programs, grant and per diem and transitional housing programs, and homeless shelters where individuals live in close quarters and often share communal living space. 9, 10 There are two main limitations that need acknowledging. First, our work using the presented AAR approach is actively under way through this continued era of COVID-19, and thus has not yet been applied to a large number of programs. However, given (i) the AAR's successful application to reviewing other major public health events, (ii) the potential large benefits of multiple programs becoming aware of and simultaneously applying the AAR as a common approach to assessing and learning from their COVID-19 responses, and (iii) the expected resurgences of COVID-19 that programs must prepare for, we see the importance of sharing our adaptation of the AAR framework with the field as early as possible. Second, our work is limited in its geographic reach, as it is being conducted in Massachusetts, USA. However, the steps of our adapted AAR (Appendix 1) (i) do not require Massachusetts as the AAR setting and (ii) are based on the international WHO's AAR framework. Even beyond housing-related programs, our AAR steps can be further adapted as a consistent approach for reviewing COVID-19 responses across multiple health care programs, bringing to light key commonalities and heterogeneities among programs that operate in widely varying contexts. Doing so can lead to delineating both standardized and tailored preparations that the programs, their organisations, and their communities can make for future waves of the pandemic. Especially given the expected resurgences of COVID-19 cases, the time to apply AAR is now. As Some Countries Ease up, Others Are Reimposing Lockdowns amid a Resurgence of Coronavirus Infections. The Washington Post Guidance for after Action Review (AAR) H1N1 after action review: learning from the unexpected, the success and the fear After-action review of the 2009-10 H1N1 influenza outbreak response: Ohio's public health system's performance Analyzing after-action reports from hurricanes Andrew and Katrina: repeated, modified, and newly created recommendations Getting the most from after action reviews to improve global health security World Health Organization. Actions Taken to Reduce Plague Burden in Madagascar and Recent Cases Show Decline. press release Department of Housing and Urban Development-VA Supportive Housing (HUD-VASH) Program The authors would like to thank our team members of the Bridging the Care Continuum Quality Enhancement The authors declare that they have no conflicts of interest. The study was approved by the Institutional Review Boards (IRBs) at the VA Boston Healthcare System (FWA00001270; IRB00000629) and the VA Bedford Healthcare System (FWA00000895; IRB00001387). The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.