key: cord-0825843-g2qww6rp authors: Sklar, Marisa; Reeder, Kendal; Carandang, Kristine; Ehrhart, Mark G.; Aarons, Gregory A. title: An observational study of the impact of COVID-19 and the rapid implementation of telehealth on community mental health center providers date: 2021-03-11 journal: Implement Sci Commun DOI: 10.1186/s43058-021-00123-y sha: 1f21fa21dd213aff58de4f6e0ea1ff1d21566c1b doc_id: 825843 cord_uid: g2qww6rp BACKGROUND: The COVID-19 pandemic has remarkably altered community mental health service delivery through the rapid implementation of telehealth. This study reports provider perspectives on the impact that COVID-19 and the transition to telehealth had on their work and their ability to deliver evidence-based practices (EBPs). METHODS: Providers (n = 93) completed online surveys with quantitative measures and open-ended items exploring their reactions to COVID-19 and to the transition to providing services via telehealth. RESULTS: Perceptions of personal risk and rumination around COVID-19 were low, while telehealth was viewed positively by providers. Three major themes emerged regarding the major impacts of COVID-19 on work: (1) the altered nature of interactions between patient/client and provider due to telehealth implementation, (2) changes in provider expectations regarding productivity, and (3) challenges maintaining work-life balance. In regard to the major impacts of COVID-19 on EBP delivery, three themes emerged: (1) increased difficulty delivering certain therapies via telehealth, (2) potential limitations to session confidentiality, and (3) challenge of engaging children in telehealth. CONCLUSIONS: In the context of the COVID-19 pandemic, community mental health providers continued to engage with clients and deliver EBPs while navigating a number of changes related to the rapid transition to and implementation of telehealth. This study highlights the need for further work on what supports providers need to effectively engage with clients and deliver EBPs via telehealth, and has implications for how telehealth is sustained or de-implemented post-COVID-19. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s43058-021-00123-y. In response to 2019-nCoV/COVID-19, community mental health centers (CMHCs) rapidly implemented telehealth services to minimize the risk of COVID-19 transmission. This report introduces new and adapted measures to explore providers' experiences in making this transition to telehealth and providing services in the context of a pandemic. As services continue to be offered via telehealth, CMHCs should consider strategies for supporting providers' ongoing delivery of EBPs via telehealth to optimize treatment for clients. The novel coronavirus disease (2019-nCoV/COVID- 19) drastically impacted the context in which mental health services are provided. Many community mental health centers (CMHCs) rapidly implemented telehealth services in order to comply with 'stay at home' restrictions set across the country [1] . Mental health providers were subsequently tasked with transitioning their services, including the delivery of evidence-based practices (EBPs), from in-person treatment to telehealth. Although evidence for the effectiveness of telehealth services (i.e., services provided via phone and/or video platforms) has existed for over 20 years, numerous barriers have delayed its widespread use until now [2] [3] [4] . In response to COVID-19, a nationwide telehealth implementation effort occurred in only a matter of weeks. This rapid telehealth implementation [5] directly contrasts with the 17year research-to-practice pipeline [6] and is even more rapid than the 3-year length of time EBPs are integrated into real-world settings with the aid of an implementation team [7] . In accordance with the STROBE checklist of items included in reports of observational studies (Additional file 1), this paper reports on the impact of the COVID-19 pandemic and rapid implementation of telehealth services from the perspectives of CMHC providers in one Midwest state and discusses implications regarding the quality and sustainment of rapid implementation efforts. Data were collected as part of an ongoing service contract between the UC San Diego and the statewide behavioral health services system in one Midwest state to engage the state and their CMHCs in the Leadership and Organizational Change for Implementation strategy [8, 9] . Participants were behavioral health services providers (n = 93, response rate = 77%) from 6 CMHCs that were also contracted to improve upon provider delivery of combined motivational enhancement therapy and cognitive behavioral therapy (MET/CBT) and other EBPs. In response to COVID-19, CMHCs implemented telehealth services, typically including both video conference and telephone modes of delivery. Providers identified mostly as female (n = 77; 84.6%), non-Hispanic (n = 81; 89%), and White (n = 79; 86.8%) and were 41 years old on average (sd = 14.8 years). Most providers had completed master's level education (n = 65; 71.4%) and identified social work as their primary discipline (n = 44; 48.4%). Providers reported spending the greatest percentage of their work time in psychotherapy and/or counseling ( x=44.1%) and reported an average caseload of 52.1 (sd = 39.5) clients per month. See Table 1 for provider demographics. CMHC providers completed surveys via the Qualtrics web-based platform that included measures targeting their reactions to the COVID-19 outbreak and transition to providing services via telehealth. The measures that were utilized are described below. Unless otherwise noted, participants responded to each measure using a 5-point response scale ranging from 0 = "Strongly Disagree" to 4 = "Strongly Agree." When appropriate, the internal consistency of each measure was assessed for this sample using Cronbach's α; results are included in Tables 2 and 4 . This 9-item measure was adapted from Wu et al.'s measure assessing perceptions of personal risk around SARS [10] . Items were adapted to assess participants' perceived risk of being exposed to, and getting infected with, COVID-19. This 3-item measure was developed by LeNoble and colleagues to assess participants' rumination about COVID-19's interference with their work [11] . Based on feedback from providers and their leaders, three distinct types of work changes were identified. A single item was developed to capture each type of work change through an iterative process of item generation, discussion, and refinement until consensus on item wording was achieved. The resulting three items assessed changes in tasks, settings, and teams that mental health providers experienced following the COVID- capture three distinct types of work changes and were not intended to represent a single underlying construct, internal consistency was not assessed [12] . The Copenhagen burnout inventory assesses participants' emotional exhaustion and work-related frustration [13] . Participants responded to three items targeting work-related burnout within the past 2 weeks using a 5point scale ranging from 0 = "Never" to 4 = "Always." We used three items from Eisenberger et al. to assess perceptions of the helpfulness, care, and concern of the agency for providers [14] . This 4-item measure adapted from a measure developed by Lau and Brookman-Frazee assessed participant's confidence, knowledge, understanding, and preparation to deliver therapy via telehealth [15, 16] . This 3-item measure adapted from Jex and Bliese assessed efficacy beliefs targeting the agency's transition to telehealth [17] . This 5-item measure adapted from the University of Michigan's Behavioral health Workforce Research Center assessed whether providers had a positive view of telehealth [18] . Seven items evaluating the transition to telehealth were developed by the study authors through an iterative process of item generation, discussion, and refinement until consensus on item wording was achieved. These items measured the extent to which different aspects of treatment were better or worse when serving clients via telehealth as opposed to in-person treatment; see Table 4 for the individual items. Participants responded using a 5-point response scale ranging from 0 = "Significantly worse with telehealth relative to in-person" to 4 = "Significantly better with telehealth relative to in-person." Participants responded to two open-ended survey items. The first asked participants about the major impacts of COVID-19 on their work, and the second asked about the major impacts of COVID-19 on the use of a specific EBP (MET/CBT) and other EBPs in general. Participant responses were aggregated across all items to obtain an overall scale mean with the exception of the work changes and the transition to telehealth items, which were analyzed individually. Descriptive statistics of quantitative measures were assessed to explore providers' responses and/or reactions to the COVID-19 outbreak, and subsequent transition to providing services via telehealth. Potential between-agency differences in provider responses were explored using univariate analysis of variance (UNIANOVA). Provider responses to open-ended survey questions were analyzed using a template organizing style of interpretation. Specifically, responses were first reviewed by authors (MS, KR, and KC) to gain familiarity with the content and to isolate broad themes. Text was then sorted and organized in accordance with broad themes, and new themes were generated when appropriate. All authors held meetings to review "chunks" [19] of text and develop summaries of findings. See Tables With regard to the questions evaluating the effects of telehealth (Table 4) , providers reported the largest benefits for scheduling ( x = 3.12), and the biggest challenge with patient/client focus (x = 2.38). Results from UNIANOVA indicated no significant differences between agencies in any of the survey items and/or scale scores (see Table 2 ). Analysis of open-ended item responses provided by 89 unique participants identified three major themes (see Table 5 ). One centered on the impact that the transition to telehealth had on interactions with patients/clients. Providers described technological barriers to high quality interactions identifying challenges such as "blocked cell number," "caseload lives in rural areas…not all kids have access to internet or stable internet," and "some clients do not have technological capacity for video conferencing." Some providers commented on the challenge of developing/maintaining rapport through the transition to telehealth. For example, providers reported "not being able to build rapport with new clients" and that "not being able to provide therapy in person and be able to read client's body language has been the major impact." Some providers reported beliefs that telehealth facilitated improvements in communication with clients. For example, providers stated that "it has been more enjoyable regarding relationships with clients…due to the less formal atmosphere as clients are more comfortable in their homes," and that they experienced "stronger communication" with clients. Another theme centered on the changes in provider expectations regarding productivity. Some providers commented on reduced productivity due to this transition, stating "my productivity has dropped…because I have been having issues coordinating with some families" and "I used to have very high guardian engage-ment…this has decreased; guardians have limited to no access to technology at times." Some providers commented on increased demands like "increase in documentation required," "increased pressure regarding productivity and revenue," and "more new changes in documentation and more paperwork without receiving productivity for increased time spent." A third theme centered on challenges maintaining a work-life balance. For example, providers stated that "work-life balance has been disrupted as I have difficulty separating myself from work" and "lack of home life/ work balance…time management is difficult." Providers also reported challenges with working from home while caring for their children, stating "working from home while taking care of my child makes it difficult to do my best work with clients" and "it can be difficult to work from home with children." When asked about the major impacts of COVID-19 on delivery of EBPs, three themes emerged based on responses from 86 unique participants (see Table 6 ). One centered on the challenge delivering certain therapies via telehealth wherein providers stated "barriers to implementing services like play therapy are somewhat dependent on the setting," "prize draws and drug screens are difficult to do," "lack of client ability to access the worksheets or use a video format," and that they "haven't tried [Eye Movement Desensitization and Reprocessing]bilateral stimulation" via telehealth. A second theme centered on the potential limitations to confidentiality and/or lack of privacy when providing treatment with telehealth. Providers reported challenges engaging clients in EBP for the treatment of trauma Responses ranged from 0 = "Significantly worse with telehealth relative to in-person" to 5 = "Significantly better with telehealth relative to in-person." Internal consistency of this measure was high at Cronbach's α = .82 Developing and/or maintaining rapport Not being able to build rapport with new clients-it's difficult to introduce yourself via telephone or even video. It is difficult to have longer appointments. It has been difficult to fully engage families as I have been able to before. I used to have very high guardian engagement, and for me, this has decreased. It has been very difficult to see my clients consistently. A lot of the people I work with are students, and unfortunately, some of the parents don't hold their children accountable for doing video sessions. My clients' participation has gone down tremendously. It is difficult for students to pay attention and they are easily distracted when I am trying to have sessions with them. It is sometimes difficult to connect with the client or to judge their state of mind without the visual clues. Sessions are shorter. It has also been more enjoyable regarding relationships with clients. I think it may be due to the less formal atmosphere as clients are comfortable in their homes. Reduced productivity My productivity has dropped because I have been having issues coordinating with some families. Less productivity hours. Loss of productivity, increased phone calls and chasing clients, helping clients download Zoom, Skype, etc. Increased demands Increase in documentation required. Increase pressure regarding productivity and revenue. More paperwork without receiving productivity for increased time spent. because some clients are less willing to discuss traumas due to limited confidentiality/privacy. Some providers stated "many clients now have less privacy at homemay have partners/kids around;" other providers reported discomfort processing trauma without being able to see how their clients are responding stating that "I have not been able to work with some of my patients on healing their trauma…I am uncomfortable due to not being able to see if they are upset, being triggered, etc." The final theme that emerged centered on the general challenge of engaging children in treatment via telehealth. Providers stated "it is difficult to teach my students over the video sessions at times depending on the subject we are discussing and distractions," "some parents don't hold their children accountable for doing video sessions," "parents prefer I work with their children face-to-face," and that "younger kids often engage better face to face." Table 6 Provider responses and themes regarding major impacts of COVID-19 on implementation of evidence-based practices (EBPs) Provider responses Play therapy Barriers to implementing services like play therapy that are somewhat dependent on the setting. Eye movement desensitization and reprocessing (EMDR) Haven't tried EMDR bilateral stimulation. Clients discomfort rating Outcome and Session Rating Scale (ORS/SRS) verbally. I cannot do EMDR treatment-clients not preferring to try the phone way of doing it. Prize draws and drug screens are difficult to do. Can't supply "rewards." Can't administer urine drug screens to assess for substance use. Group therapy My Intensive Outpatient treatment and Relapse Prevention groups don't feel as productive using Zoom. Interaction with each other is more difficult. People that were in an Intensive Outpatient Program group are now contacted 1-2 times weekly by an LCSW that may not have addictions experience of the level of experience needed. So clients are missing out a lot on the value of group therapy. No longer facilitating a group and having to contact clients individually, which has been time consuming. This study investigated major impacts of COVID-19 and the rapid implementation of telehealth services on CMHC providers' work and EBP delivery. Consistent with a recently proposed definition of rapid implementation, telehealth services were provided to those in need, with speed and efficiency, through a pragmatic reconceptualization of rigor [5] . CMHCs and representative providers utilized various modes of telehealth delivery including both video conference (e.g., via Zoom, Skype, Cisco Webex) and telephone. For clients unable to access videoconferencing sessions, providers typically reported using telephone. Providers viewed CMHC transitions to telehealth positively and reported confidence in their abilities to deliver services via telehealth. Notably, providers held generally favorable views of telehealth. Provider acceptance of telehealth services has been found to play a primary role in the implementation and sustainment of telehealth [20] , and lack of provider acceptance is noted as the greatest barrier to widespread implementation of telehealth to date [21] . This suggests it may be fruitful to explore how significant circumstances that necessitate implementation impact attitudes toward implementation and innovation adoption. In order to support rapid implementation, collaboration between researchers, funders, health systems workers, practitioners, and community partners toward a common cause is necessary [5] . Implementation efforts are most successful when propelled by the alignment of support across system and organizational contexts [22] . Historically, telehealth has been difficult to implement, scale-up, and sustain [3, 23] often due to lack of funding and/or policy support at the system-level [24, 25] . In the context of COVID-19, restrictions around the privacy of patient health information (i.e., HIPAA) and Medicaid billing requirements were adjusted, which ultimately aided the rapid transition to telehealth [26] [27] [28] [29] . The sustained use of telehealth will likely be a function of continued billing capabilities and policies to facilitate sustainment. To maximize the success of rapid implementation, the needs of a range of stakeholders, focusing on timepressured, clinically relevant questions, must be considered [5] . Collecting and providing timely information of value to stakeholders (i.e., practitioners, decision-makers, and policy makers) can guide specific actions to support rapid implementation. In this study, results suggested that although providers perceived both therapeutic relationships and clients' willingness to schedule telehealth sessions as somewhat better than in-person services, some providers reported difficulty maintaining engagement and using EBPs via telehealth. Providers also reported experiencing stress related to billing, documentation, and productivity demands while adapting to work changes and maintaining work-life balance. To sustain rapid implementation of telehealth services, it will be essential for CMHC leaders to consider this information and identify specific actions and supports that are needed such that providers can effectively engage with clients and deliver EBPs via telehealth [30, 31] . This study lends important insight into provider experiences with telehealth implementation and EBP delivery in the context of the COVID-19 outbreak. There are, however, several limitations. Due to the novel nature of this outbreak and the related work changes, some measures were created or adapted for this study and do not yet have published psychometrics, making cross-study comparison challenging. Also, this study did not include client-or treatment-level information, such as client perspectives on telehealth, nor did it include the perspectives of other relevant stakeholders, such as CMHC or system-level leadership. Additionally, though not directly assessed in this study, it is very likely that pandemic-related stressors and the associated impacts on cognitive functioning could have impacted the quality of the rapid implementation to telehealth. Finally, it is unclear how expeditious implementation will impact the sustainability of telehealth; moving too quickly through implementation without adequate planning may result in the omission of key implementation activities, and ultimately non-sustainment [32] . Future research should focus on the extent to which the rapid implementation of telehealth in the context of COVID-19 results in sustainment. In the context of the COVID-19 outbreak, CMHC providers continued to provide services and deliver EBPs through telehealth. Persistence was needed to connect and engage with clients, and creativity was crucial for continued EBP delivery. Though rapid telehealth implementation was supported by the relaxing of national policies, it remains unclear whether telehealth will be sustained moving forward, and, if so, what support providers need to do so effectively. The COVID-19 pandemic: the 'black swan' for mental health care and a turning point for e-health Improving implementation of eMental health for mood disorders in routine practice: systematic review of barriers and facilitating factors Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies Treatment for anxiety and depression via clinical videoconferencing: evidence base and barriers to expanded access in practice The rise of rapid implementation: a worked example of solving an existing problem with a new method by combining concept analysis with a systematic integrative review Yearbook of medical informatics 2000: patient-centered systems In search of program implementation: 792 replications of the teaching-family model Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol Statewide system and organizational strategy for evidencebased practice implementation and sustainment in adolescent substance use disorder treatment (R01DA049891): National Institute on Drug Abuse The psychological impact of the SARS epidemic on hospital employees in China: exposure, risk perception, and altruistic acceptance of risk Effects of institutional responses to the COVID-19 pandemic on undergraduate faculty and students across STEM disciplines Formative versus reflective measurement models: two applications of formative measurement The Copenhagen burnout inventory: a new tool for the assessment of burnout Perceived supervisor support: contributions to perceived organizational support and employee retention The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children's mental health services Predictors of burnout among community therapists in the sustainment phase of a system-driven implementation of multiple evidence-based practices in children's mental health Efficacy beliefs as a moderator of the impact of workrelated stressors: a multilevel study The use of telehealth within behavioral health settings. Utilization, opportunities, and challenges Using codes and code manuals: a template organizing style of interpretation Clinician acceptance is the key factor for sustainable telehealth services Attitudes towards digital treatment for depression: a European stakeholder survey Aligning leadership across systems and organizations to develop a strategic climate for evidencebased practice implementation Why business modeling is crucial in the development of eHealth technologies Using technology in the delivery of mental health and substance abuse treatment in rural communities: a review Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) Bulletin: Civil rights, HIPAA, and the coronavirus disease 2019 (COVID-19). U.S. Department of Health and Human Services Office for Civil Rights. FAQs on Telehealth and HIPAA during the COVID-19 nationwide public health emergency. U. S. Department of Health and Human Services Notification of enforcement discretion for telehealth remote communications during the COVID-19 nationwide public health emergency. U.S. Department of Health and Human Services Center for Medicare & Medicaid Services. State Medicaid & CHIP telehealth toolkit: policy considerations for states expanding use of telehealth: COVID-Sklar et al. Implementation Science Communications Department of Health and Human Services Delivery of evidence-based psychotherapy via video telehealth Implementing telehealth to support medical practice in rural/remote regions: what are the conditions for success? Predicting program start-up using the stages of implementation measure Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations The online version contains supplementary material available at https://doi. org/10.1186/s43058-021-00123-y.Additional file 1. STROBE Statement-checklist of items that should be included in reports of observational studies. Funding This project was supported in part by a professional service contract with a state behavioral health services system and the National Institute of Drug Abuse R01DA049891. The opinions expressed herein are the views of the authors and do not necessarily reflect the official policy or position of the state behavioral health services system or NIDA. Ethics approval and consent to participate Ethics approval, and a waiver to consent, was obtained from UCSD HRPP (protocol 200855) Competing interests GAA is an Associate Editor of Implementation Science; all decisions on this paper were made by another editor. The authors declare that they have no other competing interests.