key: cord-0806826-f80vzavq
authors: Puspitasari, Ajeng J.; Heredia, Dagoberto; Gentry, Melanie; Sawchuk, Craig; Theobald, Bernie; Moore, Wendy; Tiede, Michael; Galardy, Christine; Schak, Kathryn; Clinic, Mayo
title: Rapid Adoption and Implementation of Telehealth Group Psychotherapy During COVID 19: Practical Strategies and Recommendations
date: 2021-06-24
journal: Cogn Behav Pract
DOI: 10.1016/j.cbpra.2021.05.002
sha: 8c186d69752bd63c852652e206c0b91249e00c86
doc_id: 806826
cord_uid: f80vzavq

Behavioral health services have been tasked with rapidly adopting and implementing teletherapy during the SARS-CoV-2/COVID-19 pandemic to assure patient and staff safety. Existing teletherapy guidelines were developed prior to the pandemic and do not capture the nuances of rapidly transitioning in-person individual and group-based treatments to a teletherapy format. In this paper, we describe our approach to quickly adapting to a teletherapy technology platform for an intensive outpatient program (IOP) guided by cognitive and behavioral modular principles for adults with serious mental illness. A review of existing guidelines was conducted and the staged approach for teletherapy implementation (Muir et al., 2020) was selected as the most appropriate model for our organizational context. We describe the most pertinent implementation strategies and report our preliminary findings detailing the feasibility of IOPs delivered via telehealth. This model of rapid teletherapy implementation offers practical clinical guidelines for administrators and clinicians seeking to transition traditional in-person behavioral health services to a teletherapy format.

]Implementation of Telehealth Group Psychotherapy SARS-COV-2/ has dramatically changed how behavioral health care is delivered. The adoption of physical distancing measures led to an unplanned shutdown of in-person behavioral health programs as health care systems worked to modify treatment areas, waiting rooms, and visitor policies. Behavioral health providers were tasked with changing group activities (e.g., group therapy) by implementing telehealth technology or scheduling smaller in-person group sessions. Although physical distancing measures reduced the risk of COVID-19 transmission in health care settings, subsequent practice closures and census limitations magnified the preexisting behavioral health disparities related to access and service utilization (Kazdin, 2015) . Barriers to health care access can have far-reaching consequences for people living with serious mental illness (SMI). Research suggests that social isolation and lack of services can increase risk of symptom exacerbation, mental health crises, and suicide (Kessler et al., 2001) . To best support patients with SMI during the COVID-19 pandemic, guidelines for transitioning intensive outpatient services to teletherapy must be developed, disseminated, and implemented throughout behavioral health services.

The economic and societal costs of the COVID-19 pandemic have been profound and emerging evidence suggests that a global mental health crisis is imminent (Cullen et al., 2020) .

In particular, the psychological impact of quarantine and related physical distancing measures have been shown to have negative psychological effects, including posttraumatic stress symptoms, confusion, and anger (Brooks et al., 2020) . These effects may be compounded for people with preexisting anxiety and mood disorders who report greater pandemic-related distress (Asmundson et al., 2020) . Additionally, experts suggest that suicide, suicidal behaviors, and selfharm are likely to become a more pressing concern for the general population as the pandemic progresses (Gunnell et al., 2020) . The proliferation of high-quality research on the mental health [running head ]Implementation of Telehealth Group Psychotherapy effects of the pandemic has generated a sense of urgency to rapidly generate solutions that bridge research-practice gaps and psychological consequences for vulnerable groups under pandemic conditions (Holmes et al., 2020) .

As the pandemic unfolds, behavioral health services have been tasked with quickly implementing alternatives to in-person psychotherapy to assure patient and staff safety.

Teletherapy has emerged as an effective delivery mode (Luiggi-Hernández & Rivera-Amador, 2020 ) that leverages the power of remote-access technology to deliver services via mobile device applications, remote monitoring, online content, and live video teleconferencing (Grady et al., 2011; Kramer et al., 2012; Turgoose et al., 2018) .

Accumulating research prior to this pandemic indicates that teletherapy is comparable to in-person therapy in terms of its effectiveness for patients with diverse backgrounds and presenting problems (Backhaus et al., 2012; Barak et al., 2008; Bolton & Dorstyn, 2015; Morland et al., 2015; Rosen et al., 2014; Turgoose et al., 2018) . Teletherapy is also comparable to in-person service in terms of patient satisfaction, strength of therapeutic alliance, and attrition rate (Fortney et al., 2016; Frueh et al., 2007; Gentry et al., 2019; Turgoose et al., 2018) . One recent study also found that patients with SMI attended more teletherapy visits compared to non-SMI patients, indicating the potential added benefit of teletherapy for this population (Miu et al., 2020) .

Individual-, organizational-, and system-level barriers to teletherapy implementation existed before the COVID-19 outbreak. On the individual level, cautious clinicians were hesitant to adopt teletherapy (Jencius & Sager, 2001) and questioned their ability to provide competent [running head ]Implementation of Telehealth Group Psychotherapy care using this delivery platform (Ronen-Setter & Cohen, 2020) . Clinicians also reported feeling anxious about using new technology, reported difficulties with creating private spaces, and expressed concerns about maintaining patient confidentiality and safety (Bischoff, 2004; Gordon et al., 2015) . At the organizational level, technological difficulties, such as poor Internet connection, low image resolution, and poor audio, are potential barriers to care (Turgoose et al., 2018) . Additional barriers include inadequate staffing, staff turnover, lack of staff training, poor communication regarding priorities and workload, and poor planning regarding the change of workflow, as well as concerns around the availability of space, devices, and other resources to support a successful teletherapy practice (Muir et al., 2020) .

At the system level, given that the practice of teletherapy is relatively new, common systemic issues include the lack of insurance coverage (Luiggi-Hernández & Rivera-Amador, 2020) and professional licensure and credentialing limitations (Brooks et al., 2013) . Despite these barriers, teletherapy has been shown to be an effective solution for overcoming access disparities for people residing in isolated areas (Cason, 2017; Hoge et al., 2004; Ouimette et al., 2011) , patient time constraints, scheduling difficulties, and concerns about mental health stigma (Morgan et al., 2008; Shore & Manson, 2005; Uscher-Pines et al., 2018) . Although the COVID-19 pandemic has led to additional challenges in accessing behavioral health treatment, health care systems are motivated to rapidly adopt and implement teletherapy services to ensure the continuation of adequate patient care .

A review of the behavioral health literature identified four health service psychology programs that rapidly adopted and implemented teletherapy services after COVID-19. The four programs included one outpatient psychiatry clinic (Miu et al., 2020) , one obsessive-compulsive disorder (OCD) specialty intensive outpatient program (IOP; Sequeira et al., 2020) , one outpatient relationship-focused clinic (Burgoyne & Cohn, 2020) , and one outpatient training clinic (Scharff et al., 2020) . Before the COVID-19 pandemic, the implementation preparation of teletherapy services was described as a time-consuming process spanning several months or years (Shore & Manson, 2005) . Surprisingly, the programs described above demonstrated that the transition from in-person therapy to teletherapy could occur within 5-8 days (Miu et al., 2020; Scharff et al., 2020; Sequeira et al., 2020) , suggesting that practice changes can be accelerated to provide faster and greater access to care.

In order to mitigate the effect of the COVID-19 pandemic on behavioral health functioning, knowledge describing accelerated teletherapy implementation processes must be synthesized and made readily available to clinicians, administrators, and public policy makers.

At the organizational level, all programs mentioned above abruptly reduced or canceled inperson appointments as soon as shelter-in-place orders went into effect. Meetings with organizational leaders and clinicians were immediately conducted to plan for teletherapy adoption. In terms of technology adoption, similar strategies were conducted across organizations, such as selecting a Health Insurance Portability and Accountability Act (HIPAA)compliant video teleconferencing platform (e.g., Zoom, Microsoft Team, Skype), ensuring remote access to clinical health records, providing information technology (IT) support for clinicians, and setting up remote work capabilities. Some organizations also developed new [running head]Implementation of Telehealth Group Psychotherapy billing systems and modified their processes for patient screening, program admission, informed consent, and delivery of therapy materials to patients' homes.

At the individual level, varying strategies were implemented to assure clinicians and patients were ready to utilize teletherapy. Training to increase clinician knowledge and comfort with teletherapy was provided as soon as a plan to transition to this modality was confirmed by organizational leadership. Training seminars included protocols for utilizing teletherapy technology effectively (e.g., demonstration of Zoom chat and screen sharing features). Ongoing supervision and consultation were made available in some practices. To assure clinician privacy, some organizations established professionalism guidelines recommending that clinicians block their personal phone numbers when making calls to patients, use virtual background to cover personal spaces, and ensure clinical work is conducted in private areas.

During the initial transition phase, clinical workload was temporarily reduced while clinicians were adapting to new workflows. Organizations typically provided patients with an initial orientation session before starting their course of teletherapy. Doing so helped ensure that patients had the ability to use the technology and provided a platform for establishing teletherapy norms (e.g., muting the microphone when not talking). To assure patient safety, clinicians were encouraged to assess the patient's location in case of emergency. Last, at the system level, organizations verified that teletherapy services were covered by third-party payers. Fortunately, many insurance companies expanded their coverage as the pandemic spread and several companies readily provided reimbursements for teletherapy services. Organizations also made sure to adhere to state licensure regulations pertaining to the provision of teletherapy treatment across state lines; at the start of the COVID-19 pandemic, [running head]Implementation of Telehealth Group Psychotherapy many states temporarily relaxed these regulations to facilitate the rapid expansion and utilization of behavioral health care.

To facilitate practice change, behavioral health stakeholders can turn to recently published articles describing the rapid adoption and implementation of teletherapy. Knowledge presented by programs that have already transitioned to teletherapy can serve as valuable benchmarks for organizations seeking to follow suit. The health care system's sudden investment in teletherapy services may require organizations to prioritize "on-the-go" strategies that expedite the planning phase of change-making. Nevertheless, a review of guidelines and best practices published prior to the COVID-19 pandemic can be helpful for instilling effective and sustainable changes that address present-day needs. Before the COVID-19 pandemic, commonly used teletherapy guidelines included those developed by the American Psychological Association (2013), the American Psychiatric Association (Shore et al., 2018) , and the American Telemedicine Association (Turvey et al., 2013) .

While reviewing clinical guidelines is an important start point to teletherapy implementation, a review of evidence-based implementation strategies (Powell et al., 2012 (Powell et al., , 2015 and frameworks can help ensure the effective translation of best practices into clinical practice (Grimshaw et al., 2004) . Among these, the Promoting Action of Research Implementation in Health Services (PARIHS) is the most used implementation framework in studies conducted before COVID-19 (Bauer et al., 2018; Lindsay et al., 2015; Naik et al., 2015) .

More recently, a model for the implementation of teletherapy in the public sector was proposed by Muir and colleagues (2020) wherein they consolidated recommendations proposed by the [running head]Implementation of Telehealth Group Psychotherapy American Telemedicine Association, the American Psychological Association (Shore et al., 2018) , the PARIHS model (Kitson et al., 2008) , and the model proposed by Shore and Manson (2005) .

The primary aim of this study is to describe the process for the rapid adoption and implementation of teletherapy in the Adult Transitions Program (ATP), an IOP for adults with SMI within the Department of Psychiatry and Psychology at Mayo Clinic. We selected the implementation of teletherapy in the public sector model (Muir et al., 2020) as a guiding framework for this transition and describe the different implementation strategies selected for each of the phases proposed in the model (further elaborated in the "A Staged Approach for ATP Teletherapy Adoption and Implementation" section below). This model was developed to guide health care systems to plan, introduce, and assess the outcome of teletherapy implementation.

When planning for the teletherapy adoption, the team decided that this model offered the most relevant and comprehensive guidelines for our clinical practice. As a secondary aim, we report on the initial feasibility outcomes after the pilot implementation phase between March and August 2020, which included program completion rate, program attendance rate, and referrals provided to patients to ensure successful care transition. Last, we provide future research directions and clinical implications.

Mayo Clinic's Department of Psychiatry and Psychology developed the ATP in 2013 in response to the high cost of psychiatric hospital admission and readmission (Heslin & Weiss, [running head ]Implementation of Telehealth Group Psychotherapy 2015; Stensland et al., 2012; Weiss et al., 2017) and the need to provide an immediate step-down program between acute psychiatric hospitalization and longitudinal aftercare services for adults with SMI. We adopted a transitional care model (Viggiano et al., 2012) to facilitate a successful psychiatric hospitalization discharge, which has been found to reduce readmission rates by 13-37% (Vigod et al., 2013) . While existing psychiatric transitional programs focus on managing discharge needs, assessment, care coordination, and medication management, the ATP also incorporated group psychotherapy guided by cognitive and behavioral principles for patients struggling with suicidal and self-injurious behaviors, high depression and anxiety, and addiction.

The inclusion criteria were adults with SMI, such as mood disorders, anxiety disorders, psychotic disorders, and substance use (Kessler et al., 2001) , who were recently discharged from or determined to be at risk for psychiatric hospitalization. We excluded patients with poor cognitive functioning (as determined through clinical interview during the preprogram evaluation) and high acuity of symptoms requiring inpatient hospitalization. We received referrals primarily from Mayo Clinic inpatient psychiatric units, the Emergency Department, Mayo Clinic primary care clinics, and outpatient practices in the area. The standard in-person ATP consists of two tracks. Track 1 included patients with comorbid transdiagnostic conditions (e.g., mood and personality disorders) and substance use. Track 2 included patients without comorbid substance use. Since the program was developed as a short-term bridge between psychiatric hospitalization and lower levels of care, all patients received 3-week programming, delivered 5 days a week for 3 hours per day.

The standard in-person ATP had the capacity to take 16 patients split between two tracks.

Each track of the ATP consisted of three group psychotherapy sessions delivered by a licensed professional clinical counselor (LPCC) trained in cognitive-behavioral therapy (CBT). Track 1 [running head]Implementation of Telehealth Group Psychotherapy consisted of a behavioral activation (BA) group (Busch et al., 2009; Martell et al., 2013) , a recovery group for substance use guided by dialectical behavior therapy (DBT) principles (Dimeff & Linehan, 2008) , and a DBT skills group (Linehan, 2014) . The BA group focused mainly on activity scheduling and homework review, while offering behavioral strategies when patients struggled to complete their goals (e.g., problem solving, strategies targeting avoidance).

Of note, due to the shorter length of the programs, modular strategies for these evidence-based psychotherapies were selected and patients received all modules during their 3-week stay. Track 2 also consisted of the BA and DBT skills groups and included and occupational therapy group instead of the recovery group for substance use.

All patients received additional individual sessions during their time in the program: a preprogram evaluation with an LPCC or a registered nurse (RN), a psychiatric admission session with a nurse practitioner/physician assistant (NP/PA), a psychology admission session, a midpoint session, and a discharge session with an LPCC. Due to the potential acuity, our program also offered adjunctive individual sessions that included medication management with an NP/PA and suicide assessment and intervention guided by the Collaborative Assessment to Manage Suicidality principles (Jobes et al., 2017) by an LPCC. Measurement-based care (MBC), where patients' symptoms were monitored at admission and discharge to assess treatment progress and outcomes (Lewis et al., 2019) , has been an integral part of the program. We leveraged the Electronic Health Record (EHR) system (i.e., EPIC) to administer patient-reported outcome measures that were automatically recorded in their EHR to ease the process for clinicians to review and discuss scores with patients.

Our multidisciplinary team for the standard in-person program consisted of a clinical director (clinical psychologist), a medical director (psychiatrist), a clinical psychologist, four [running head]Implementation of Telehealth Group Psychotherapy LPCCs, one NP/PA, one occupational therapist, and two RNs. The program was also supported by a group of leadership and administrative team members within the Department of Psychiatry and Psychology.

On March 27, 2020, the State of Minnesota instituted a State of Emergency, including a shelter-in-place order due to the exponential increase of COVID-19 cases. In compliance with the shelter-in-place order, several outpatient mental health services within Mayo Clinic were suspended or limited to urgent and emergent care. Due to the acuity of the ATP patient population and the potential increased risk of psychiatric hospitalization during this distressing time, department leadership decided that the ATP should remain open and rapidly transition to teletherapy.

All telehealth developments and coordinated operational efforts were overseen by the departmental multidisciplinary Telehealth Oversight Committee (TOC), a multidisciplinary group. The TOC was formed in early April 2020 to help facilitate the adoption of telehealth services. This committee built on an existing telehealth planning committee that had been in place since 2015. The TOC included psychiatrists, psychologists, allied health, administrators, finance experts, and support from an institutional telehealth resource center, named the Center for Connected Care. This committee met twice weekly for the first month during the most rapid phase of implementation. This committee reviewed, updated, and expanded upon existing training resources, guidelines, and policies, including development of the process for providing virtual groups in this study. [running head ]Implementation of Telehealth Group Psychotherapy Due to the closing of many behavioral health services in the surrounding area and increased need for intensive outpatient care, the ATP expanded capacity and added an additional track for patients suffering with mood and anxiety disorders. This new track was guided by the newly published Process-Based Cognitive and Behavioral Therapy (Hayes & Hofmann, 2018) resource, which included a BA group and two process-based CBT groups each day for 3 weeks.

The program implemented rolling admission and provided services to eight patients at any given time. This expansion within the ATP teletherapy practice increased patient capacity from 16 to 24 patients.

In this section, we delineate our process for rapidly adopting and implementing the teletherapy-guided model proposed by Muir and colleagues (2020) . We describe the challenges we faced and the solutions we perused, all of which were guided by the model and existing implementation science literature. Table 1 defines the different phases of the model and summarizes the examples of implementation strategies selected to support ATP teletherapy adoption and implementation.

The first phase of the teletherapy implementation started soon after the State of Minnesota initiated the shelter-in-place order toward the end of March 2020. The preparation phase was conducted in 2 weeks, starting on March 23, and our first implementation rollout started on April 6. [running head]Implementation of Telehealth Group Psychotherapy Needs assessment. Although the recommendation is to conduct needs assessment using a quantitative, qualitative, or mixed-methods approach (Goldstein & Ford, 2002) , it was not feasible to conduct a formal analysis given that the decision to transition to teletherapy was made within days. Thus, one immediate challenge faced was assuring that relevant stakeholders were engaged to plan for the rapid teletherapy adoption as efficiently and feasibly as possible. As a solution, ATP clinical and medical directors, with the support of the operations manager, communicated the rationale for teletherapy to the department leadership, the Clinical Practice Committee, the TOC, and the clinical team. We determined through consensus that the need to continue providing care was significant and that the risks of closing down the program were high (i.e., potential increased rates of Emergency Department visits and psychiatric hospitalizations).

Assess organizational capacity. We did not conduct a formal organizational capacity assessment due to limited time. Instead, we conducted a multidisciplinary team meeting with ATP clinicians, administrators, IT supports, and financial consultants to determine what resources were needed to ensure successful teletherapy adoption.

Technological capacity. In terms of technological capacity, the institution provided clinicians with hospital-issued laptops with secure remote work capabilities (i.e., virtual private network [VPN] system). Prior to COVID-19, Mayo Clinic had an established process for using Zoom as a HIPAA-compliant platform for video-based care, though this had not been previously used for group therapy visits. The main challenge we faced in this area was organizing the distribution of laptops, headsets, and other tools to assure that clinicians were equipped to see patients remotely. We relied significantly on TOC and managers from each discipline to assure [running head]Implementation of Telehealth Group Psychotherapy that clinicians had access to this necessary technology in a timely manner. Technological glitches occurred more frequently early on but gradually improved. All clinicians had 24/7 access to the IT help desk for additional technological support.

Environmental capacity. Given Mayo Clinic's interest in implementing telehealth services prior to COVID-19, most clinical offices were equipped with web cameras and headsets.

Those not equipped were rapidly updated. Larger rooms were designated as safer working spaces for physical distancing of clinicians who preferred to provide teletherapy from the hospital.

Individual office spaces were assigned for clinicians to provide group teletherapy without compromising privacy or confidentiality. One barrier that was not easily solved was how to best support patients who did not have the hardware required for teletherapy (e.g., tablet). As an alternative, we provided referrals or offered in-person bridge visits to patients with acute needs, consisting of meetings with either an NP/PA for medication management or an LPCC for solution-focused therapy. Between March and August 2020, only five patients were not able to attend teletherapy or receive bridge visits.

Financial capacity. We did not make significant changes in the process of billing since many third-party payers readily covered teletherapy when the shelter-in-place order started.

Human factors. We were aware that preparing clinicians to adopt teletherapy was one of the most important strategies to ensure successful implementation. Since the decision to adopt teletherapy was made rapidly, it naturally increased anxiety among our team members, as many steps of delivering group teletherapy were new and the outcome remained uncertain at that time. [running head ]Implementation of Telehealth Group Psychotherapy Given that the pandemic affected all of us, not just patients, team members had to balance quality care expectations in tandem with personal stressors (e.g., caring for family members with . As such, we prioritized supporting team members at every stage of the teletherapy implementation process.

Starting March 23, we engaged clinicians in daily discussions to provide the rationale for providing teletherapy, answer questions, and discuss workflow changes. These discussions occurred in our previously established daily huddles, as well as weekly DBT consultation team meetings. The virtual daily huddles were 20-to 30-minute meetings attended by all team members to discuss patients' progress, safety risk, schedule of the day, and other topics pertinent to clinic operation. Specific therapeutic topics that we routinely discussed as a team included how to build rapport over teletherapy and addressing patients' therapy-interfering behaviors (e.g., turning off camera, multitasking, falling asleep, and addressing safety concerns).

The change to teletherapy also affected the workflow of other administrative team members (e.g., desk staff)-thus we engaged closely with supervisors and maintained close communication with administrative staff to ensure that they were prepared for the teletherapy implementation and associated coordination nuances. Written and easily accessible resources were also determined to be helpful in supporting clinicians during implementation. Thus, the clinical director and IT team created several quick reference guides (QRGs) that were uploaded to a shared folder. The QRGs contained technical information about conducting successful teletherapy visits via Zoom. One QRG was written to guide clinicians to assist a suicidal caller over video teleconferencing or phone. During the week of teletherapy implementation, a formal training was provided for all ATP staff that included review of the teletherapy technology and QRGs, in addition to addressing team member concerns. [running head]Implementation of Telehealth Group Psychotherapy Task analysis. Although a formal task analysis was conducted 2 years prior to teletherapy implementation, it was not feasible to conduct an additional analysis during this transition time.

Instead, we aimed to maintain the role delineations present prior to the pandemic with minor modifications. Several examples of task shifting included having the LPCCs temporarily involved in administering patient-reported outcome measures (a task previously assigned to RNs and desk staff), having RNs conduct the preprogram evaluation (a task previously conducted by LPCCs), temporarily adding a second LPCC in each group teletherapy as cofacilitator to assist with technological issues and therapy-interfering behaviors, and having desk staff mail program binders and therapy materials to patients prior to their program start.

Task shifting and changing workflows engendered stress among team members. Helpful strategies for maintaining a culture of close collaboration within the team included engaging regularly with interdisciplinary supervisors, clearly describing the task or workflow changes, increasing communication between team members, and responding to questions or coverage needs in a steadfast manner.

Feature development for teletherapy platforms. Two additional technological platforms supported our teletherapy implementation including EPIC (e.g., EHR), and a shared electronic ATP team folder that housed relevant documents, such as QRGs, program census with deidentified patient information, therapy materials, and program policies and procedures.

Recruit and repurpose a teletherapy team. Due to the economic impact of COVID-19, an organizational hiring freeze was put in place. Thus, we were unable to recruit new staff for ATP teletherapy implementation. However, since several programs were temporarily closed due to the pandemic, nine additional LPCCs were available to assist with the expansion of the new track in the month of May. They served as cofacilitators and provided suicide assessment and intervention as needed. To successfully support our teletherapy initiative, we ensured that our implementation team was representative of our multidisciplinary staff and administrative team members (e.g., clinical psychologists, LPCCs, nurses, operations managers, psychiatrist). When needed, we consulted with Mayo Clinic legal representatives to ensure that our teletherapy implementation aligned with state licensure, accreditation, and other legal requirements.

New roles and responsibilities (and personnel). As mentioned in Phase 1, we aimed to maintain previous roles and responsibilities as much as possible. When task shifting or new needs emerged, we learned that close communication with interdisciplinary supervisors and staff members was essential to help promote understanding of new duties, to attain input from staff, to keep them engaged, and to problem-solving "pain points" that would inevitably occur.

Teletherapy champions. We identified teletherapy champions early in the preparation phase. These champions included directors of the ATP, an operations manager, members of the TOC IT specialists, and several clinicians who either had experiences with teletherapy or were enthusiastic and recognized the importance of the rapid adoption. In the first 5 months of implementation, these champions were fully integrated into the team to provide adequate support [running head]Implementation of Telehealth Group Psychotherapy for the rest of the team members (e.g., clinical director modeled how to deliver teletherapy, other champions attended daily huddle and helped team members navigate Zoom).

Teletherapy coordinator. The ATP operations manager became the primary point person and coordinated the preparation, adoption, and implementation phase. This individual was responsible for managing the workflow and engaging other stakeholders within and outside of the department to ensure a smooth transition to teletherapy.

We employed several communication strategies to ensure that changes were rapidly shared with team members and challenges were identified and quickly solved. Formal communication occurred during daily huddles, weekly consultation team meetings, monthly ATP service meetings, monthly ATP leadership meetings, and other adjunctive meetings when indicated. We also found that reporting back to the larger departmental leadership was helpful. In such meetings, the clinical director described the results of this rapid implementation to the TOC and Department Clinical Practice Committee. Formal written communication was typically delivered via e-mail and distribution of QRGs. Informal daily communications also occurred across several avenues, such as curbside consultations, pager communications, phone calls, and EPIC secure chats, when clinicians needed to discuss patient-related concerns to facilitate in-person communication.

While we aimed to be as organized as possible, efficient cross-specialty communication was another area in which glitches often happened. In a busy clinical practice where team members balance patient care, administrative tasks, and self-care, important information often "fell through the cracks" or was not communicated in a timely manner. Our most effective [running head]Implementation of Telehealth Group Psychotherapy solution was to consistently learn from each communication issue and to emphasize the importance of timely and efficient communication, and team flexibility.

The TOC decided that the ATP would be the first pilot site to implement group teletherapy. While the typical evaluation plan for an implementation project might require a year to complete, these unique circumstances dictated taking a more rapid approach to collect implementation outcome data, consolidate lessons learned, and share our experiences with other programs within the department. A week after the ATP teletherapy rollout, the clinical director met with the TOC and representatives from other programs to share challenges, solutions, and lessons learned. It was also pertinent to quickly share our knowledge with external behavioral health programs. Thus, we developed our process and outcome evaluation plan together with our teletherapy implementation process. Implementation plan. The TOC decided to use a staged implementation strategy where the ATP was identified as the first pilot site to adopt group teletherapy. Several factors influenced this decision, which included the acuity of our patient population, readiness for teletherapy implementation among ATP team members, openness from clinicians to adopt telehealth, availability of resources and space, and increased demand and referrals to provide this level of behavioral health care for patients who were recently discharge from, or at risk for, psychiatric hospitalization, particularly in light of the pandemic crisis. [running head]Implementation of Telehealth Group Psychotherapy Process evaluation plan. Challenges faced during the teletherapy rollout were informally assessed and communicated to team members for efficient problem solving. After a week of telehealth implementation of the initial two ATP tracks, we learned that overall, the process was successful and realized that there was a need to accommodate the increased number of referrals.

These data were brought to the TOC and upper departmental leadership. Approval was received to expand the program and add an additional track to meet the growing demand. Several barriers were identified within the first month of implementation. These included the need to add more office space for clinicians, improve the process of administering patient-reported outcome measures, and the need to provide more support for clinicians to engage in self-care for burnout mitigation.

Outcome evaluation plan and data consideration. The primary goal and objective of the ATP teletherapy format was to continue providing services for our patient population that were comparable to in-person services. We achieved this goal by maintaining the full census of our two original tracks. Additionally, we were able to expand the program a month after the start of the ATP teletherapy format and added one more track with eight-patient capacity.

One of the formal implementation outcome evaluation plans was to create a quantitative and qualitative survey for the first cohort of patients who received ATP teletherapy. This survey was completed at the end of discharge to assess a patient's perception on ATP teletherapy acceptability, feasibility, and appropriateness (Weiner et al., 2017) . We collected written qualitative feedback from patients to assess the feasibility of ATP telehealth, as well as areas of improvements for future implementation. We also monitored patients' attrition, absences, and successful establishment of the next level of care after the ATP. [running head]Implementation of Telehealth Group Psychotherapy Since MBC was already an integral part of the ATP even prior to teletherapy implementation, we continued to collect data on patient-level outcomes since the start of teletherapy. Our primary measures were the Patient Reported Outcomes Measurement Information System (PROMIS) Global-10 ( Barile et al., 2013) to assess quality of physical and mental health, Suicide Status Form (Conrad et al., 2009) to assess suicidality, Patient Health Questionnaire 9-item (Kroenke & Spitzer, 2002) to assess depression, Generalized Anxiety Disorder 7-item (Spitzer et al., 2006) to assess anxiety, and the Alcohol Use Disorders Identification Test (Bush et al., 1998) to assess alcohol problems.

Outcome evaluation design considerations. Given that we are still early in the implementation phase, we have focused our efforts on collecting pre-post data during the first 6 months of implementation. We are currently delineating future directions to assess the effectiveness of ATP teletherapy implementation and consider other study designs, including quasi-experimental designs or clustered randomized control trial designs, to assess the outcome of ATP teletherapy implementation in comparison to other (including in-person) delivery models.

Define exit and reevaluation points. Our defined reevaluation point was set for 8 months following the initial ATP teletherapy rollout (i.e., December 2020). There were several lessons learned that were communicated to team members and department leadership, and informed iterative changes for future implementation. First, team members had become more comfortable with facilitating group teletherapy. As a result, we decided to have only one facilitator in each group and created a process of communication in case a facilitator needed additional support for [running head]Implementation of Telehealth Group Psychotherapy technology issues or patient safety concerns (e.g., using a group chat feature on our EHR to request assistance from other team members). Second, additional staff support was gradually reduced, which allowed clinicians to return to their previous clinical practices since many other programs within our department started to reopen in 2021. Last, additional department resources and support (e.g., TOC and IT resources) were gradually reduced since the ATP had adjusted to the group teletherapy process and did not require additional administrative and technological support moving forward.

Select teletherapy software and ensure data security. We followed Mayo Clinic's institutional decision to use Zoom as the software to deliver teletherapy, which is a HIPAAcompliant platform. Usability: Anecdotally, many of our clinicians were familiar with Zoom and demonstrated confidence while using the software as observed by the clinical director when clinicians provided teletherapy. Integrate new technology: As an institution widely, and the ATP specifically, we started integrating Zoom even prior to ATP telehealth implementation to ensure that the new software was fully integrated to clinician daily workflows. This strategy was also selected to gradually increase clinicians' confidence with the technology.

Develop teletherapy guidelines and other resources. We decided to keep the therapy topics and materials the same as the original in-person format. Several changes, however, were made regarding the therapy process, such as updating group norms and crisis management procedures. For the consent process, each patient who started the program went through the main consent for treatment document that also included a consent to receive teletherapy. We revised our group therapy norms to include items specific to teletherapy (e.g., patients were requested to [running head]Implementation of Telehealth Group Psychotherapy be in a private area and turn their video on to ensure confidentiality of other group attendees).

We also developed a teletherapy crisis management procedure and made it easily accessible to clinicians. A formal training to assist a suicidal caller was conducted by the clinical director, which included modeling and role play as experiential exercises. The training was audio recorded and stored in the shared drive for other clinicians to access if needed. The daily huddle started by discussing patient safety issues. Regarding the clinical intervention, we continued implementing CAMS and all patients completed the SSF at admission and discharge. A stabilization plan was developed by the LPCCs and patients when needed. The clinical director was available to be paged or join suicide assessment and intervention sessions if clinicians required additional support. Ongoing communication between team members regarding patient safety also occurred in person or via EPIC secure chat.

Resources and support for patients was also provided to ensure that they could utilize teletherapy technology smoothly. Our desk staff were available to provide additional guidance for patients learning to use Zoom. We developed a temporary "command center" staffed by LPCCs and RNs to support group facilitators as needed. Team members in the command center contacted patients if they did not log in to group on time, needed to be assessed for suicide risk, or to provide timely support to patients navigating technological issues. This setup was implemented for the first 6 months and was transitioned to clinicians who were not engaged in direct patient care (e.g., during documentation periods).

Provide education and training. Education, training, and ongoing supervision was an integral implementation strategy for the ATP prior to teletherapy adoption. This strategy was of particular importance at the start of teletherapy implementation to support clinicians' successful 25 [running head ]Implementation of Telehealth Group Psychotherapy engagement with the technology, as well as to establish an effective practice for virtual therapy.

In-house formal training on teletherapy and navigating calls from suicidal patients was provided.

More importantly, we implemented on-the-job, ongoing training and supervision throughout the pilot implementation. Strategies included discussing teletherapy implementation in our weekly DBT consultation team, peer observations (e.g., by being a cofacilitator), and sharing newly developed resources to learn best practices to provide teletherapy during COVID-19 (e.g., free webinar, readings).

As previously mentioned, compassion fatigue and burnout were naturally occurring challenges faced by team members. To address this, team leaders and members alike practiced the psychotherapeutic skills taught in the ATP. Mindfulness practices were incorporated into daily huddles and consultation team meetings. All team members were encouraged to practice DBT skills for emotion regulation, distress tolerance, and interpersonal effectiveness.

We are currently in the pilot implementation phase and project to maintain this phase for approximately 6-12 months. Guided by Muir and colleagues (2020) , the major steps in this phase include implementation of ATP teletherapy, process and outcome evaluation, providing ongoing support and training for clinicians and staff, increasing staff capacity through recruitment, and demonstrating meaningful use of teletherapy both to the ATP team and the department at large.

Phase 5: Full Implementation [running head]Implementation of Telehealth Group Psychotherapy ATP has established a plan to initiate full implementation after 6 months of pilot implementation.

Between April 6 and August 28 of 2020, 90 patients started the program and were assigned to one of three tracks (N = 33 in Track 1, N = 32 in Track 2, and N = 25 in Track 3).

Among those who started the program, N = 81 (90.0%) completed the program and N = 1 (1.1%) attended less than 10 sessions (partial completers) and N = 8 (8.9%) attended seven or less session (noncompleters). The average sessions attended was M = 14.4 days (SD = 1.4). Since the ATP is intended as a bridge program to assure that patients have continuous care, we analyzed the teletherapy format and in the context of COVID-19 whether our program could continue to support patients to connect with the next level of care successfully prior to ATP discharge. Our data showed that N = 16 (17.8%) had at least one behavioral health service appointment, 55 (61.1%) patients had two behavioral health services appointments, and 13 (14.4%) had more than two appointments scheduled within 2 weeks of ATP discharge. Six patients (6.7%) did not have any behavioral health appointment scheduled, four of them were noncompleters, and two patients refused to receive referrals prior to discharge.

The aim of this study was to describe the rapid adoption and implementation of a group teletherapy program for adults with SMI in the context of COVID-19. We discuss the rationale for teletherapy as an alternative delivery mode to ensure access to behavioral health care for patients struggling with acute symptoms at higher risk for psychiatric hospitalization. We [running head ]Implementation of Telehealth Group Psychotherapy summarized published guidelines for the rapid implementation of teletherapy during the COVID-19 pandemic and highlighted the most common implementation strategies used in those studies.

We advance the existing literature in this area by utilizing established guidelines for teletherapy in tandem with an evidence-based implementation framework (Muir et al., 2020) to guide our adoption and implementation of teletherapy services. We hope this model serves as a resource for other behavioral health services charged with rapidly transitioning in-person behavioral health treatments to teletherapy.

Our preliminary data demonstrated that even with 2 weeks of preparation phase, adapting the ATP for teletherapy delivery was feasible. The implementation strategies we prioritized (e.g., engaging relevant stakeholders, providing ongoing staff training, and ensuring environmental and technological capacity) were essential to facilitating a successful pilot teletherapy program.

We were able to accept most referrals and expanded our program after a month of the initial start of the teletherapy format. Data on patient attrition indicated that the teletherapy is feasible to assure patient retention, since many of our patients completed the program and the average number of sessions attended was high.

This study is consistent with the previous studies (Burgoyne & Cohn, 2020; Miu et al., 2020; Scharff et al., 2020; Sequeira et al., 2020) where we found that rapidly transitioning an inperson IOP to teletherapy in the context of COVID-19 was feasible. While these four programs switched to teletherapy within 5-8 weeks, it took us 2 weeks to complete the preparation phase before starting the teletherapy format. One potential factor for a longer period was the need for our practice to engage more stakeholders and committees since we are located within a larger academic medical center. Furthermore, we implemented many strategies described in these previous studies, such as engaging with relevant stakeholders and leadership, ensuring [running head]Implementation of Telehealth Group Psychotherapy technological capacity, and staff training and supervision. One aspect that differentiates our study was the use of an implementation model (Muir et al., 2020) to guide this rapid transition, which allowed us to consider a comprehensive list of strategies in a more systematic way.

Several challenges unique to hospital-based IOPs were faced-however, our findings are generalizable to nonhospital behavioral health practices. First, many clinicians experienced heightened anxiety while rapidly switching from in-person to group teletherapy practice. This emotional reaction was natural and understandable. Several strategies were implemented to address this emotional shift among the team, including intentional validation provided by leadership and peers, attaining additional departmental support and staffing in the first 8 months of teletherapy implementation, and providing more training, QRGs, and consultation for clinicians. Second, role changes and communication difficulties were common. Department leadership and ATP team members worked together to develop communication processes that worked best for our team (e.g., chat features, paging systems). Last, logistical challenges, such as distribution of devices (e.g., laptops, headsets), administration of patient-reported outcome measures, and limited office space to accommodate for physical distancing, were a daily challenge. We found that administrative support for these barriers was key.

Since this preliminary study of an implementation model was conducted in a single location, building on a previously established program, the strategies selected and findings might not be generalizable to all behavioral health service settings. It is important to highlight the unique aspects of implementing group teletherapy in an IOP setting within a large academic health center. In our context, the network of relevant stakeholders was relatively larger, including different committees and leadership structures that had to be engaged and approved the teletherapy adoption. This meant that more time and streamlined communication was required to [running head]Implementation of Telehealth Group Psychotherapy ensure successful adoption. This element might not be applicable for smaller behavioral health practices. Additionally, we were fortunate in that resources were readily available (e.g., IT support, additional administrative support), especially during the planning and early implementation phase. Other settings might not have the luxury of these resources and thus might come across different barriers that were not discussed in this paper. As such, future studies should aim to describe strategies for tailoring this work to address additional organizational barriers and needs. Although the Muir et al.'s (2020) model was selected a priori, in the context of a pandemic and a rapid transition to teletherapy, many of the stages recommended by the model was not feasible to be conducted (e.g., comprehensive quantitative and qualitative needs assessment). Thus, future studies should consider a longer period for each stage tested in diverse clinical settings to incorporate more evidence-based implementation strategies.

Furthermore, since this finding only includes preliminary data on initial feasibility, further studies need to assess other implementation outcome metrics. In line with this assertion, we are currently in the process of collecting patient-level outcome data at admission and discharge to assess the initial effectiveness of ATP teletherapy on patient symptoms and quality of life. We will also conduct a more rigorous feasibility study (Bowen et al., 2009 ) by employing mixed methodology (e.g., survey and interview) to assess ATP teletherapy acceptability, feasibility, and appropriateness perspectives from patients, clinicians, leaders, and other administrative staff members. A quasi-experimental study comparing patient outcomes pre-COVID (i.e., in-person ATP) and post-COVID (i.e., ATP teletherapy) is warranted. These patient and implementation outcome data will provide further guidance on the appropriateness of future larger-scale implementation efforts and will inform future implementation research trials (e.g., cluster randomized clinical trial). [running head]Implementation of Telehealth Group Psychotherapy  Conducted a stakeholder meeting to assess organizational capacity, including ATP clinicians, adminsitrators, IT team, and financial team.  The institution provided laptops for clinicians and assured remote work capability.  Office computers were equipped with web cameras and headsets.  Access to 24/7 IT support.  Engagement with clinical and administrative staff and supervisor to assess human resource capacity to switch to teletherapy.

 Generate an understanding of the specific processes in staff roles  Did not make significant changes on role delineation.  Minor task shifting (e.g., process of administering patient-reported outcome measures) that was discussed with staff and supervisors.  Produce a logic model with goals and objectives for teletherapy  Establish a set of key variables (success indicators) to be used to evaluate the implementation and effectiveness of the teletherapy service  Success indicators were maintaining full census of two initial ATP tracks, patient's perspective on ATP teletherapy acceptability, feasibility, and appropriateness, and patient-level clinical outcomes.

 Identify critical decision points in the implementation model  Define actions to be taken in response to evaluation results  Pilot implementation stage was determined to be 6 months (i.e., October 2020).  Data on key variables above will be evaluated and used to guide future

Guidelines for the practice of telepsychology

Do pre-existing anxiety-related and mood disorders differentially impact COVID-19 stress responses and coping

Videoconferencing psychotherapy: A systematic review

A comprehensive review and a meta-analysis of the effectiveness of Internet-based psychotherapeutic interventions

Monitoring population health for Healthy People 2020: Evaluation of the NIH PROMIS Global Health, CDC Healthy Days, and satisfaction with life instruments

Implementing and sustaining team-based telecare for bipolar disorder: Lessons learned from a model-guided, mixed methods analysis

Considerations in the use of telecommunications as a primary treatment medium: The application of behavioral telehealth to marriage and family therapy

Telepsychology for posttraumatic stress disorder: A systematic review

How we design feasibility studies

Provider barriers to telemental health: Obstacles overcome, obstacles remaining

The psychological impact of quarantine and how to reduce it: Rapid review of the evidence

Lessons from the transition to relational teletherapy during COVID-19

Behavioral activation: Distinctive features

The AUDIT Alcohol Consumption Questions (AUDIT-C): An effective brief screening test for problem drinking

Telehealth is face-to-face service delivery

A psychometric investigation of the Suicide Status Form II with a psychiatric inpatient sample

Dialectical behavior therapy for substance abusers

A tipping point for measurement-based care

Therapist adherence and competence with manualized cognitive-behavioral therapy for PTSD delivered via videoconferencing technology

Evidence for telehealth group-based treatment: A systematic review

Training in organizations: Needs assessment, development, and evaluation

Comparing psychodynamic teaching, supervision, and psychotherapy over videoconferencing technology with Chinese students

Evidence-based practice for telemental health

Effectiveness and efficiency of guideline dissemination and implementation strategies

Suicide risk and prevention during the COVID-19 pandemic

Process-based CBT: The science and core clinical competencies of cognitive behavioral therapy

Hospital readmissions involving psychiatric disorders, 2012. Statistical Brief

Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care

Multidisciplinary research priorities for the COVID-19 pandemic: A call for action for mental health science

The practice of marriage and family counseling in cyberspace

A randomized controlled trial of the collaborative assessment and management of suicidality versus enhanced care as usual with suicidal soldiers

Technology-based interventions and reducing the burdens of mental illness: Perspectives and comments on the special series

The prevalence and correlates of untreated serious mental illness

Evaluating the successful implementation of evidence into practice using the PARiHS framework: Theoretical and practical challenges

A standard telemental health evaluation model: The time is now

The PHQ-9: A new depression diagnostic and severity measure

Implementing measurement-based care in behavioral health: A review

Implementation of video telehealth to improve access to evidence-based psychotherapy for posttraumatic stress disorder

DBT skills training manual

Reconceptualizing social distancing: Teletherapy and social inequality during the COVID-19 and loneliness pandemics

Behavioral activation for depression: A clinician's guide

Teletherapy with serious mental illness populations during COVID-19: Telehealth conversion and engagement

Does the use of telemental health alter the treatment experience? Inmates' perceptions of telemental health versus face-to-face treatment modalities

Telemedicine versus in-person delivery of cognitive processing therapy for women with posttraumatic stress disorder: A randomized noninferiority trial

Videoconferencing psychotherapy in the public sector: Synthesis and model for implementation

Building a primary care/research partnership: Lessons learned from a telehealth intervention for diabetes and depression

Perceived barriers to care among Veterans Health Administration patients with posttraumatic stress disorder

A compilation of strategies for implementing clinical innovations in health and mental health

A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project

Becoming "teletherapeutic": Harnessing accelerated experiential dynamic psychotherapy (AEDP) for challenges of the Covid-19 era

Cognitive processing therapy for posttraumatic stress disorder delivered to rural veterans via telemental health: A randomized noninferiority clinical trial

Shifting a training clinic to teletherapy during the COVID-19 pandemic: A trainee perspective

Transitioning to virtual programming amidst COVID-19 outbreak

A developmental model for rural telepsychiatry

Best practices in videoconferencing-based telemental health

A brief measure for assessing generalized anxiety disorder: The GAD-7

An examination of costs, charges, and payments for inpatient psychiatric treatment in community hospitals

Digital technology can revolutionize mental health services delivery: The COVID-19 crisis as a catalyst for change

Systematic review of lessons learned from delivering tele-therapy to veterans with post-traumatic stress disorder

ATA practice guidelines for video-based online mental health services

Virtual first responders: The role of direct-to-consumer telemedicine in caring for people impacted by natural disasters

Care transition interventions in mental health

Transitional interventions to reduce early psychiatric readmissions in adults: Systematic review

Psychometric assessment of three newly developed implementation outcome measures

Trends and projections in inpatient hospital costs and utilization

 [running head