key: cord-0706305-414nlri6
authors: Barnett, Miya L.; Sigal, Marika; Green Rosas, Yessica; Corcoran, Frederique; Rastogi, Medini; Jent, Jason F.
title: Therapist Experiences and Attitudes About Implementing Internet-Delivered Parent-Child Interaction Therapy During COVID-19
date: 2021-05-12
journal: Cogn Behav Pract
DOI: 10.1016/j.cbpra.2021.03.005
sha: 2878bc845bd4027e4f5db944350080eb94d78415
doc_id: 706305
cord_uid: 414nlri6

It has been widely recognized that access to mental health treatment is imperative to address current and long-term stressors for children and parents during COVID-19. Internet-delivered Parent-Child Interaction Therapy (iPCIT, previously referred to as I-PCIT) is a strong model for remote service delivery during social distancing restrictions due to its empirical base. However, this treatment modality was not widely implemented before COVID-19, likely due to barriers to providing telehealth services. This mixed methods study conducted a follow-up survey to gather therapist experiences (N = 223) in delivering iPCIT during COVID-19, including qualitative data on the benefits and challenges to delivering iPCIT. The vast majority of therapists (82%) indicated that they transitioned to deliver PCIT via telehealth in response to COVID-19. PCIT caseloads decreased slightly from the first survey to the COVID-19 follow-up survey, but the racial and ethnic composition of caseloads were not significantly different between the two surveys. Of the 183 therapists who transitioned to deliver PCIT via telehealth, 82% expressed interest in continuing to provide iPCIT following the COVID-19 pandemic. Reported benefits of iPCIT included decreased barriers to access and the ability to practice skills within the naturalistic home environment. Challenges to iPCIT were primarily issues with technology as well as other logistical barriers, which could limit engagement for some families. Findings from this study may be beneficial in improving future implementation of iPCIT during and post-COVID-19.

In the wake of physical distancing measures during the global COVID-19 pandemic, children and families are experiencing high levels of stress, which is increasing the risk of both child maltreatment and poor mental health outcomes (Cameron et al., 2020; Imran et al., 2020; Masten & Motti-Stefanidi, 2020; Nunn, 2020; Teo & Griffiths, 2020) . Importantly, heightened levels of stress in a household can disrupt family functioning and the parent-child bond, hindering a child's ability to adapt and build resilience (Gewirtz et al., 2008) . Further, parental stress due to crisis and disaster is positively linked to increases in child mental health concerns, including child behavior problems (Kerns et al., 2014) . Thus, COVID-19 not only presents a concern for the physical health of individuals throughout the United States, but also has significant implications for the emotional and mental health of children and caregivers as they navigate a myriad of stressors, including: financial stress related to high rates of unemployment, school closures, and isolation (Imran et al., 2020; Masten & Motti-Stefanidi, 2020; Nunn, 2020; Riegler et al., 2020) . A major concern for these families is the increased reports of violence and child abuse that have been shown to escalate during school closures linked to health emergencies (Cluver et al., 2020; Piquero et al., 2020) . It has been widely recognized that access to high-quality mental health treatment is imperative to address current and long-term stressors for children and parents during COVID-19.

The shift to telemental health services has been critical in offering and continuing support for clients and families during the pandemic (Douglas et al., 2020; Sharma et al., 2020) . This was made largely possible by increased insurance coverage for telehealth visits, including mental health, as well as looser HIPAA restrictions on telehealth services and platforms (Gurwitch et al., 2020) . At the beginning of the pandemic, for instance, the federal government initiated an emergency order whereby providers were protected against penalty for any unintended HIPAA violation under the good faith provision of telehealth (U.S. Department of Health and Human Services, 2020). The notice has allowed providers to continue providing services over video-communication platforms without risk of penalty during COVID-19, even if they do not fully comply with HIPAA requirements. Despite strong efforts to implement telehealth services, there are several barriers that both clients and clinicians still face that need to be addressed to ensure effective care (Wright & Caudill, 2020) . Many challenges that providers face include concerns over maintaining therapeutic alliances via technology-based strategies, as well as effectively communicating emotion and compassion to clients with limited nonverbal cues (Anton & Jones, 2017; Békés et al., 2020) . For clients, disparities in access to telehealth intersect and exist along many digital divides including SES, race/ethnicity, immigration status, language, and age (Zhai, 2020) . These factors strongly influence an individual or family's access to the necessary components of telehealth (technological devices, stable broadband access) as well as their digital literacy (Beaunoyer et al., 2020; Khilnani et al., 2020) . COVID-19 has exposed the digital inequalities that exist among underserved populations, potentiating their lack of access to telehealth resources during this time.

In order to address increasing challenges with parental stress (Cameron et al., 2020; Riegler et al., 2020) , child behavior problems (Imran et al., 2020; Nunn, 2020) , and caregiver use of harsh or abusive discipline (Piquero et al., 2020; Teo & Griffiths, 2020) , it is increasingly important that caregivers have access to evidence-based parenting programs via telehealth. Behavioral parent training programs, such as Parent-Child Interaction Therapy (PCIT), are best-practice, evidence-based interventions to prevent and treat child disruptive behaviors (e.g., tantrums, aggression, defiance) and child physical maltreatment (Kaminski & Claussen, 2017) . PCIT emphasizes the parent-child relationship, uses in vivo feedback (i.e., coaching), and monitors client progress in treatment with weekly assessments of parent skill use and child behavior problems. PCIT does this through two major phases: child-directed interaction (CDI) and parent-directed interaction (PDI). CDI targets the parent-child relationship and establishes a foundation for treatment, whereas PDI focuses on structured and consistent discipline strategies (i.e., time-out) to address behavioral problems. Many states and counties have invested in the dissemination and implementation of PCIT given its potential to prevent the enormous personal and societal costs of early-onset conduct problems and child maltreatment (Beveridge et al., 2015; Scudder et al., 2017; Timmer et al., 2016) .

PCIT had already been adapted to be provided via telehealth prior to the COVID-19 pandemic, which is referred to as internet-based PCIT (iPCIT), to increase access to the treatment (Comer et al., 2017) . In iPCIT, therapists conduct client sessions using a video platform in which therapists and caregivers utilize either a laptop computer, tablet, or cell phone device positioned in a way that enables therapists to see the child and caregiver interacting. The caregiver receives in vivo coaching via either a bluetooth device directly connected to the visual interface (i.e., computer, tablet, or cell phone screen) or headphones connected separately to a cell phone. Studies comparing traditional in-person PCIT to iPCIT demonstrated promising results for iPCIT (Comer et al., 2017; Kohlhoff et al., 2019) . Comer et al. found that children who received iPCIT were more likely than those who received standard PCIT to be rated by evaluators masked to treatment condition to have an "excellent response" at posttreatment and the 6-month follow-up.

Furthermore, results indicated significantly fewer parent-perceived barriers to treatment for iPCIT in comparison to clinic-based PCIT, as well as high treatment satisfaction ratings for both iPCIT and clinicbased delivery. Though iPCIT had promising outcomes, it had not been widely implemented prior to COVID-19, in part because of the challenges that arise with insurance (e.g., billing and reimbursement) as well as variability in technological capacity for clients and agencies (Gurwitch et al., 2020) . Nonetheless, at the onset of COVID-19, iPCIT emerged as a strong model in the transition to telehealth, as it had an empirical base, and traditional PCIT was already widely implemented in community settings (Gurwitch et al., 2020) . However, questions remained regarding how to best support therapists across different settings in delivering the model via telehealth. To inform further implementation of iPCIT and best serve families during the COVID-19 pandemic, it is necessary to better understand therapist experiences, including perceived challenges as well as benefits in delivering iPCIT.

The current study was conducted as a follow-up to a survey about PCIT implementation, which was completed by 324 PCIT therapists in fall 2019. Building off of the previous study, we had a unique opportunity to follow up with therapists who completed the original survey to investigate how the transition to telehealth impacted their delivery of PCIT. Specifically, the current study sought to investigate three research questions regarding the transition to telehealth using a mixed methods design, in order to inform implementation of iPCIT, both during the COVID-19 pandemic and following stay-at-home orders. Our research questions were as follows: (1) How did the transition to online delivery impact client caseloads, especially in regard to clients at risk for disparities in access to care? (2) What were the perceived challenges of delivering iPCIT during COVID-19? (3) What were the perceived benefits of delivering iPCIT during COVID-19?

In October and November 2019, 324 therapists were recruited to complete a study on PCIT implementation on two list serves for PCIT, one managed by PCIT International for therapists certified in the model by this organization and another by UC Davis, which does not require certification. A follow-up survey about experiences with clinical care during COVID-19 was sent to the participants from the previous study who had provided their contact information (N = 309). The participants were asked to complete a 15minute self-report survey regarding clinical practice during COVID-19. To gain a representative sample of those who had and had not transitioned to telehealth, the recruitment email specified that they did not need to have delivered PCIT during COVID-19. Each participant received a unique Qualtrics link to allow their responses to be connected to their original survey responses. Therapists were first asked screening questions that assessed whether or not they transitioned to delivering PCIT via telehealth. Display logic tool on Qualtrics was used to display the appropriate question based on participants' previous answers. Upon completion, participants were e-mailed a $20.00 Amazon gift card as compensation. The study was determined exempt by the Institutional Review Board at the University of California, Santa Barbara.

Out of the 309 emailed participants, 72% responded to the follow-up survey (N = 223). The sample was predominantly female (90%), with an average therapist age of 36.42 (SD = 8.24). The majority of therapists self-identified as non-Hispanic, White (74%) and most described their mental health discipline as clinical psychology (35%). A large proportion of therapists (89%) indicated that they were working at the same agency they were working for during the first survey in 2019, and nearly half (43%) of therapists described their primary work setting as a community mental health clinic. The majority of therapists reported that they were certified in PCIT (70%). Participants had an average of 18 clients on their caseload, including 5 PCIT clients on average. Regarding reimbursement for services, 39% of therapists saw predominantly clients with Medicaid or state insurance, 13% saw predominately clients with private insurance, 14% saw predominantly private-pay clients, and 2% predominantly saw uninsured clients. Additional sample characteristics can be found in Table 1 .

Differences in therapist demographics were compared for individuals who participated only in the original survey to those who completed the follow-up survey related to COVID-19. A chi-square test of independence showed that therapists did not significantly differ in gender,  2 (2) = 3.78, p = .151, race  2 (5) = 3.89, p = .566, or ethnicity  2 (2) = .042, p = .979. Further independent samples t-test analysis revealed no significant differences in age t(321) = 1.59, p = .114, number of total cases t(147) = 1.36, p = .175 or PCIT cases t(321) = .814, p = .416. As there were no significant differences between participants who responded to the follow-up survey and those who did not, it appeared to be a representative sample of therapists from the original study.

A modified version of the Therapist Background Questionnaire (Brookman-Frazee et al., 2012) was used to collect demographic information about participants (e.g., age, gender, place of work, type of services provided) in the original survey. In the follow-up survey, participants were also asked if they were still working at the same agency as in Fall 2019 when they completed the first survey.

Therapists were asked demographic information about their current total and PCIT caseload in both surveys. Questions about their caseload included items about the number of cases on their total caseload, the number of PCIT cases on their caseload, and the racial and ethnic composition of their caseload (i.e., percentage of caseload who were non-Hispanic, White, Latinx/Hispanic, Black/African American, Asian American/Pacific Islander, and Native American). To gain a better understanding of the types of clients served, the follow-up survey also asked therapists to describe the form of reimbursement they used for their clients (i.e., Medicaid/state insurance, private insurance, uninsured, private pay). As these frequencies had a bimodal distribution, the variable was dichotomized to understand funding source for the majority of clients on the caseload (i.e., > 75% of the caseload).

To assess rates of transition to telehealth services, therapists were asked: (1) if they provided PCIT via telehealth (yes/no) and (2) if they provided other telehealth services other than PCIT (yes/no).

Participants who had transitioned to telehealth were asked what proportion of their caseload from before COVID-19 transitioned to telehealth services (i.e., all, 50-99%, less than 50%, none). If it was reported that any portion of clients did not transition to telehealth, those therapists completed a checklist of reasons clients gave for not transitioning to iPCIT (e.g., client did not have access to internet). Finally, participants were asked if they were interested in continuing to provide iPCIT after COVID-19 restrictions lifted (yes/no).

Participants who had provided iPCIT were asked two open-ended questions to understand their perspectives of the benefits and challenges of the delivery model. Benefits of iPCIT were measured by the following open-response question, "Briefly explain some of the benefits you experienced delivering PCIT via telehealth" and challenges of PCIT via telehealth were measured by the following open-response question, "Briefly explain some of the challenges that you experienced delivering PCIT via telehealth."

A mixed method design to data analysis was employed for the current study in order to combine qualitative and quantitative data (Palinkas et al., 2011) . Simultaneous data collection and analysis (QUAL + quan) occurred in order to explore findings descriptively regarding the transition to telehealth. The function of the mixed method approach was complementarity, with quantitative data used to understand the uptake of iPCIT amongst therapists and clients, and qualitative data analyzed to understand therapist perceptions of the challenges and benefits to this uptake.

To better understand uptake of iPCIT, frequencies of therapists who transitioned to iPCIT and desired to continue to deliver PCIT remotely following COVID-19 restrictions were investigated. To see if caseload composition changed over time, repeated measure t-tests were conducted from time 1 (i.e., 2019 study) to time 2 (i.e., current study) for proportion of total and PCIT cases, as well as ethnic composition of caseload.

Of the 183 participants who had transitioned to provide PCIT via telehealth, 151 therapists reported on perceived benefits and challenges of iPCIT. Open-ended responses were coded following recommendations for conducting qualitative analyses within mental health services and implementation research (National Cancer Institute, 2018; Palinkas et al., 2011) . First, a coding team of three coders (two graduate student research assistants and one undergraduate research assistant) read all responses and developed a coding manual with 21 codes for different benefits and challenges of iPCIT. Research assistants used the coding manual to individually code the first 25% of responses. Next, group consensus on occurrence or nonoccurrence of codes was reached to ensure the reliability of the coding manual's definitions before research assistants were allowed to move forward with the next half of the responses. This method continued in a stepwise manner until all responses were coded and consensus had been reached on each open-ended response. Following coding, the entire authorship team discussed how to codes were organized into themes that expanded on the challenges and benefits specific to iPCIT. See Tables 3 and 4 for overarching themes, frequencies of subcodes, and illustrative quotes.

Almost all therapists (92%; n = 204) stated that they were currently providing some form of telehealth services (other than PCIT), with 82% (n = 183) indicating that they had provided iPCIT during COVID-19. Furthermore, 82% of these participants indicated interest in continuing to deliver iPCIT following COVID-19, which suggested acceptability of the delivery model.

Therapists were asked what proportion of their caseload transitioned to telehealth services during COVID-19. The majority of providers (79%) reported that half or more of their PCIT caseload made the transition to telehealth services during COVID-19. Only 6 therapists reported that none of their clients chose to transition to iPCIT. The most common client reasons cited for not transitioning to iPCIT included client preference (i.e., client uncomfortable with remote services) and challenges with other children (e.g., lack of childcare). 183. There were no significant differences in the caseload composition for any of the racial or ethnic groups (see Table 2 ).

Qualitative themes expanded on reasons why the majority of therapists indicated an interest in continuing to deliver iPCIT, by identifying components that made it an appealing delivery model. Overall, therapists recognized the benefit of being able to continue providing PCIT in a safe manner during the pandemic. For instance, one therapist explained the added benefit of "not having to sanitize toys, not worrying about virus exposure." Therapists expressed that iPCIT does not significantly differ from in-person PCIT, making the transition to telehealth a smooth experience: "I feel as though most aspects of PCIT transferred extremely well to telehealth." Two overarching themes related to specific benefits of iPCIT: (1) the benefit of applying the treatment skills within the home setting and (2) increasing access and engagement for some families.

More than half of the benefits discussed had to do with providing treatment within the home setting.

For example, many therapists mentioned the ability for treatment to occur in a naturalistic environment, which they perceived increased the generalizability of skills to the home: "Transfer of the parents' skills learned in treatment is better than if they were in the clinic because they are learning/practicing learned skills in their natural environment." The added benefit of seeing the home setting also contributed to greater treatment gains, as one therapist expressed, "One of my client families did everything perfectly in in-person sessions, yet the ECBI remained high. The transition to telehealth allowed us to see what was breaking down during play and the timeout sequence at home and work on targeted generalization." Further, seeing the home environment gave providers an opportunity to gain realistic insight into the family dynamic and allowed for targeted support for challenging activities: " […] ability to see dynamics in the home and provide coaching through problem areas (dinner, morning routine, etc...)."

The transition to telehealth reduced logistical barriers for both therapists and their clients to increase access and engagement in PCIT. For instance, iPCIT made treatment more accessible to a broader population, "Some families that had wanted to do PCIT in the past but couldn't due to scheduling issues are now able to participate." Additionally, therapists reported that the switch to telehealth significantly improved attendance, "Parents can more easily attend, when [there] may be challenges regarding transportation or other issues that may inhibit them from coming to the office" and allowed for "greater family engagement." In the context of COVID-19 specifically, iPCIT helped ease stress for some clients, "PCIT is relieving significant parental stress related to COVID, working from home, schools being closed, etc." as well as for therapists, "It helps me, as a PCIT therapist, to save time and able to navigate my dual roles: PCIT therapist and 'full-time' mom (we lost childcare due to COVID)." In sum, the majority of therapists reported generally positive perceptions of iPCIT, with some expressing that it had been fulfilling even with new challenges introduced by the model, "Delivering PCIT via telehealth […] has been more challenging and more rewarding."

Though the home setting and opportunities for engagement were seen as benefits, these facets of iPCIT also posed challenges: "Keeping parents engaged when they are distracted by siblings, taking calls, other service providers (e.g., exterminator) arriving." The majority of therapists agreed that the main challenge of iPCIT "has been primarily with technology." Challenges fit into two overarching themes related to (1) limited resources, such as Internet access or equipment (headsets, toys) and (2) challenges with delivering the treatment protocol via telehealth. For instance, some therapists expressed concerns with rural populations being disproportionately disadvantaged: "The biggest challenge is the factor of people having adequate technology and internet access. My community is very rural and over 40% of our county does not have access to high speed internet." The same was true for low-income families because of the "expectation of caregivers to buy a variety of toys even though they may lack resources." Additionally, therapists reported that low-income families might not have adequate space to have private treatment sessions. However, therapists still experienced challenges with some clients who had adequate resources, access and equipment

Furthermore, technology posed challenges regarding providing coaching to parents while they interacted with their child: "Hard to see the child consistently during session as they wander off screen; hard to hear what is happening in the room and child's statements as clearly as in-office; disruptions to technology (clients getting disconnected, headphones running out of battery, etc.)." In addition, therapists noted that preparing for iPCIT sessions was more difficult for parents, "Home environment (setup) is difficult for parents to manage and many parents are resistant to adapting their setups for a more efficient therapy session." This was especially challenging for the discipline phase of treatment (PDI), which typically starts in a highly controlled clinical setting with therapist ability to provide in-person support before parents implement the use of time-out within the home-setting: "Parents refusing to go into PDI because of the fear of tantrums and not being able to manage them appropriately."

It is widely recognized that while the "first wave" of COVID-19 is the impact of the virus on the health of individuals, the "second wave" impact will be on mental health, and the number of individuals personally affected will be larger and far-reaching (Masten & Motti-Stefanidi, 2020) . In fact, this second wave is being exacerbated by many of the measures needed to prevent spread of the virus, including physical distancing. Stay-at-home policies that require children and caregivers to learn and work from home ultimately interfere with social support networks and protective factors that help to alleviate mental health concerns. These conditions create heightened challenges with parenting, including managing children's emotional and behavioral responses to stress (Imran et al., 2020) . Therefore, it is of utmost importance and urgency to understand the implementation of providing high-quality, evidence-based telehealth services to promote resiliency in children and their families. PCIT is an excellent model to address many of the challenges of COVID-19, in that it has an evidence-base in decreasing parental stress, child behavior problems, depression, and anxiety (Gurwitch et al., 2020; Lieneman et al., 2017) . Furthermore, evidence on its effectiveness via telehealth (Comer et al., 2017; Fleming et al., 2020) supports the intervention as a frontline response to address many mental health and parenting challenges arising from COVID-19.

Encouragingly, our study found that the vast majority of PCIT therapists transitioned to providing iPCIT, with 82% of our respondents indicating they had made the transition. Of those therapists, 79%

reported that the majority of their clients transitioned to iPCIT from in-person services. Though therapists reported some client attrition following the transition to telehealth, for reasons including concerns about receiving remote services and logistical barriers (e.g., childcare for siblings), the majority continued. These findings indicate a continuity in evidence-based services for many families receiving PCIT at the onset of COVID-19. Furthermore, our findings suggested that for our sample, in which approximately 40% of therapists provided care for Medicaid or uninsured populations, there were no significant changes in the racial and ethnic composition of client caseloads. This is consistent with other recent research that demonstrated successful transition from PCIT to iPCIT during COVID-19 (98%) for predominantly Latinx families (71%; Garcia et al., in press). However, it is important to note that qualitative themes identified that some therapists identified challenges in internet and technology access for low-income, underserved, and rural communities, consistent with concerns about disparities driven by the digital divide (Beaunoyer et al., 2020; Khilnani et al., 2020; Zhai, 2020) . Therefore, it is important to continually identify how telehealth services impact disparities in access to care.

Though there were not significant changes in the racial or ethnic composition of caseloads following the transition to iPCIT, it needs to be noted that across both surveys, therapists reported seeing a majority non-Latinx, White client population on average. Therefore, further efforts may need to be made to increase the access and acceptability of standard PCIT and iPCIT, along with other evidence-based practices to address mental health disparities in care (Alegria et al., 2010; Alegría et al., 2016) . Ameliorating service disparities for communities of color requires concerted efforts regarding how to implement evidence-based practices, such as PCIT, within communities of color to increase access to and acceptability of the intervention (Baumann & Cabassa, 2020; Woodward et al., 2019) . Strategies to increase initial engagement in treatment could include culturally tailored, direct-to-consumer advertisements or therapists could partner with trusted community members (i.e., natural helpers, community health workers) to conduct outreach with families who would benefit from PCIT (Barnett et al., 2016; Barnett, Miranda, et al., 2020) . Specific to telehealth, given that many videoconferencing platforms may not be apps that families typically use, trusted community members may be able to share their experiences regarding the ease of use and the privacy of videoconferencing apps with families seeking services. Further, therapists may enhance the perceived fit and acceptability of PCIT for communities of color if they attend to the family's culture and tailor how they frame the intervention and skills to a parent's values and expectations for treatment (McCabe et al., 2020) . At a systems level, implementation of PCIT needs to focus on increasing reach within communities that have been historically marginalized and underserved. This includes attention to cultural humility and reduction to language barriers within training and consultation models (Baumann & Cabassa, 2020) . For example, the PCIT Spanish Coalition, which is comprised of bilingual community and universitybased therapists, has focused on increasing the availability of high-quality translated intervention materials and case consultations for working with Spanish-speaking families. COVID-19 has not only provided an opportunity to increase access to telehealth mental health services, but also remote training and consultation models, which might be more cost-effective for low-resource settings. Further research is needed to identify how low-cost implementation strategies impact client, fidelity, and health equity outcomes, in order to increase access to evidence-based services for communities of color (Weisz et al., 2020; Woodward et al., 2019) .

The majority of challenges to providing iPCIT that therapists identified in open-ended qualitative responses related to difficulties with technology. To address technological challenges therapists should plan to schedule a session to set up technology before proceeding with service delivery as this provides an opportunity to effectively structure the environment and test out what equipment works the best for each family's space. Additional practical solutions common challenges to iPCIT implementation are provided in Supplemental Table 1 . Specific guidance is provided related to access to technology and materials, video and audio connectivity issues, resistance to altering the home environment for session, and hesitation about starting PDI in a telehealth format. Beyond these brief proposed solutions, additional open-source recommendations regarding providing iPCIT during COVID-19 have been detailed in a training guide that was developed and disseminated via listservs for certified PCIT therapists (Peskin et al., 2020) and is freely available on the PCIT International website (http://www.pcit.org/covid-19-professional-resources.html).

Beyond challenges with access to technology and adequate resources, some respondents noted that the added burdens during COVID-19 (e.g., working and attending school from home) made engagement in session challenging. For these reasons, beyond adaptations to iPCIT, it is important to consider brief and scalable interventions for families that are not able to engage in intensive treatments, but still need mental health and parenting support (Gruber et al., 2020; Schleider et al., 2020) . Further, given the differential impact of COVID-19 on different communities throughout the country, it is important for PCIT providers to seek out support from other PCIT providers within their community and region to learn from one another the most effective strategies for delivering iPCIT for families in their community. In fact, recent iPCIT research demonstrated that the use of locally developed iPCIT recommendations for service delivery and communities of practice were related to reported reductions in children's disruptive behaviors during COVID-19, whereas, the use of national resources for iPCIT service delivery by therapists actually led to reported increases in reported disruptive behaviors during iPCIT (Garcia et al., in press).

Beyond challenges with access to technology and adequate resources, respondents noted that the added burdens during COVID-19 (e.g., working and attending school from home) made finding appropriate space and time to do session in treatment especially difficult. Even with the identified challenges, the benefits of delivering iPCIT were recognized by participants with qualitative themes pointing to the opportunities provided by serving families in the home environment, which overcame logistical barriers to access for some families and allowed treatment skills to be applied in naturalistic environments. These benefits are consistent with findings from an efficacy trial of iPCIT, which found higher rates of treatment engagement and response in comparison to clinic-based PCIT (Comer et al., 2017) . Given challenges with dissemination and implementation of treatments tested within efficacy trials (Weisz et al., 2014) , it is encouraging that PCIT therapists identified these benefits of the model after delivering it under the stressful circumstances of COVID-19. Ideally, future research will investigate the effectiveness of iPCIT when delivered within community settings to see the clinical impact of this delivery model in real-world settings.

Further, though therapists attributed providing treatment in the home environment to greater generalizability of skills, it still needs to be investigated if parents are better able to generalize skill use to new settings (e.g., public outings) when they receive iPCIT compared to standard PCIT.

Notably, over 80% of participants who had transitioned to iPCIT expressed that they hoped to continue to deliver this model following the COVID-19 pandemic and stay-at-home orders. This finding, which was similar to a study showing that psychologists predicted an increased use of telehealth following COVID-19 (Pierce et al., 2020) , points to the potential for how mental health treatment delivery in general and PCIT delivery specifically will change in the years to come. Though mental health therapists and clients have had concerns about delivering treatment via telehealth, opportunities to build capacity and identify benefits might support sustained use of this service delivery model following the pandemic (Anton & Jones, 2017; Pierce et al., 2020) . Given pervasive challenges to access of evidence-based practices, such as PCIT, these changes could provide an opportunity to reach more families in need of these services (Gruber et al., 2020; Gurwitch et al., 2020) .

This study was able to leverage a past study of PCIT implementation to provide insight into how treatment delivery changed following the onset of the COVID-19 pandemic and stay-at-home orders for a large sample of therapists. However, as with all research, findings from this study needed to be interpreted within its limitations. First, though there was a high response rate to the follow up survey (72%), the original sample and therefore the current one may not be representative of PCIT therapists in general. Respondents were predominantly certified in the model and may have held more positive attitudes and commitment to delivering PCIT as compared to therapists who did not complete the survey. Therefore, these therapists may have been more likely to make the transition to iPCIT and sustain their delivery of the treatment during the stress of COVID-19 than therapists with less experience. Further, though PCIT has been disseminated internationally (Solomon & Orengo-Aguayo, 2018) , our sample is predominantly from the United States and therefore limited in perspectives from providers from other countries.

The COVID-19 pandemic has required dramatic shifts in mental health service delivery, which has brought many challenges, but also opportunities to reimagine how to increase access of evidence-based practices (Gruber et al., 2020; Moreno et al., 2020) . By identifying how therapists perceive delivering iPCIT, researchers and practitioners will be able to develop strategies to improve the implementation and accessibility of iPCIT to families in need of services during COVID-19 and in the years to follow. Though many benefits to iPCIT were identified, challenges pointed to areas for enhanced training or systems-level change to support equitable access to care. Specific to delivering iPCIT, implementation supports are needed to help therapists identify how to address challenges for delivering treatment within a home setting. These could include trainings and consultation around how to manage difficulties with technology (e.g., child leaves the screen), along with strategies to manage child behaviors during iPCIT, especially during the discipline (PDI) phase. Future research should identify implementation strategies that best support PCIT therapists in online treatment delivery and also examine how increased use of iPCIT might change access to treatment over time.

At a systemic level, COVID-19 has further illuminated structural barriers to equity, including health insurance coverage and the divide in access to internet and appropriate technology (Beaunoyer et al., 2020; Khilnani et al., 2020; Zhai, 2020) . Further, COVID-19 has disproportionately impacted racial and ethnic minority communities, both in terms of disease burden, job loss, and discrimination, reflecting the necessity that mental health providers promote resiliency in the face of the racial trauma that has been heightened during the COVID-19 pandemic (Liu & Modir, 2020) . Changes to health and mental health systems following COVID-19 need to include structural changes to insurance, internet coverage, and workforce capacity building to make sure the most impacted and underserved groups receive equitable access to high quality services, including PCIT (Gruber et al., 2020; Moreno et al., 2020) .  Of therapists who delivered iPCIT, 82% indicated an interest to continue with the telehealth delivery model after COVID-19.  Qualitative data identified benefits of delivering iPCIT during COVID-19, which included increased access and the ability to generalize skills within the home setting  Challenges to iPCIT were primarily issues with technology, including limited access to internet for underserved communities.

Removing obstacles to eliminating racial and ethnic disparities in behavioral health care

Racial and ethnic disparities in pediatric mental health

Adoption of technology-enhanced treatments: Conceptual and practical considerations

Direct-to-consumer marketing for parent-child interaction therapy: Impact of language and messenger

The development and evaluation of a natural helpers' training program to increase the engagement of urban, Latina/o families in parent-child interaction therapy

Developing and evaluating a lay health worker delivered implementation intervention to decrease engagement disparities in behavioural parent training: a mixed methods study protocol

Reframing implementation science to address inequities in healthcare delivery

COVID-19 and digital inequalities: Reciprocal impacts and mitigation strategies

Barriers and facilitators to the integration of web-based interventions into routine care

State-wide dissemination and implementation of parent-child interaction therapy (PCIT): Application of theory

Maternal psychological distress & mental health service use during the COVID-19 pandemic

Parenting in a time of COVID-19. The Lancet

Remotely delivering real-time parent training to the home: An initial randomized trial of Internet-delivered parent-child interaction therapy (I-PCIT)

Strategies to enhance communication with telemental health measurement-based care (tMBC)

An effectiveness open trial of internet-delivered parent training for young children with conduct problems living in regional and rural Australia

Parenting practices as potential mechanisms for child adjustment following mass trauma

Mental health and clinical psychological science in the time of COVID-19: Challenges, opportunities, and a call to action

Leveraging parent-child interaction therapy and telehealth capacities to address the unique needs of young children during the COVID-19 public health crisis

Mental health considerations for children & adolescents in COVID-19 Pandemic

Evidence base update for psychosocial treatments for disruptive behaviors in children

Caregiver distress, shared traumatic exposure, and child adjustment among area youth following the 2013 Boston Marathon bombing

The COVID-19 pandemic: new concerns and connections between eHealth and digital inequalities

Internet-delivered parent-child interaction therapy: Two clinical case reports

Parent-child interaction therapy: current perspectives

The outbreak that was always here: Racial trauma in the context of COVID-19 and implications for mental health providers

Multisystem resilience for children and youth in disaster: Reflections in the context of COVID-19

Personalizing behavioral parent training interventions to improve treatment engagement and outcomes for culturally diverse families

How mental health care should change as a consequence of the COVID-19 pandemic

Qualitative Methods In Implementation Science

Keeping our children safe and calm in troubled times

Mixed method designs in implementation research

The COVID-19 telepsychology revolution: A national study of pandemic-based changes in U.S. mental health care delivery

Staying home, staying safe? A short-term analysis of COVID-19 on Dallas domestic violence

Pilot trial of a telepsychotherapy parenting skills intervention for veteran families: Implications for managing parenting stress during COVID-19

Future directions in single-session youth mental health interventions

A mixed-methods study of system-level sustainability of evidence-based practices in 12 large-scale implementation initiatives

Implementation of home-based telemental health in a large child psychiatry department during the COVID-19 crisis

Handbook of parent-child interaction therapy: Innovations and applications for research and practice

Child protection in the time of COVID-19

Filling potholes on the implementation highway: Evaluating the implementation of Parent-Child Interaction Therapy in Los Angeles County

Notification of Enforcement Discretion for Telehealth Remote Communications During the COVID-19 Nationwide Public Health Emergency

Odd couple? Reenvisioning the relation between science and practice in the dissemination-implementation era

Clinician training, then what? Randomized clinical trial of child STEPs psychotherapy using lowercost implementation supports with versus without expert consultation

The health equity implementation framework: Proposal and preliminary study of hepatitis C virus treatment

Remote treatment delivery in response to the COVID-19 pandemic