key: cord-0818449-3d7aje7h
authors: Jones, Deborah J.; Loiselle, Raelyn; Zachary, Chloe; Georgeson, Alexis; Highlander, April; Turner, Patrick; Youngstrom, Jennifer K.; Khavjou, Olga; Anton, Margaret; Gonzalez, Michelle; Bresland, Nicole; Forehand, Rex
title: Optimizing Engagement in Behavioral Parent Training: Progress Toward a Technology-Enhanced Treatment Model
date: 2020-07-15
journal: Behav Ther
DOI: 10.1016/j.beth.2020.07.001
sha: ada190b3691c4652325e92fb64c6841f845b84dc
doc_id: 818449
cord_uid: 3d7aje7h

Abstract Low-income families are more likely to have a child with an early-onset Behavior Disorder (BD); yet, socioeconomic strain challenges engagement in Behavioral Parent Training (BPT). This study follows a promising pilot to further examine the potential to cost-effectively improve low-income families’ engagement in and the efficiency of BPT. Low-income families were randomized to (a) Helping the Noncompliant Child (HNC; McMahon & Forehand, 2003), a weekly, mastery-based BPT program that includes both the parent and child or (b) Technology-Enhanced HNC (TE-HNC), which includes all of the standard HNC components plus a parent mobile-application and therapist web-portal that provide between-session monitoring, modeling, and coaching of parent skill use with the goal of improved engagement in the context of financial strain. Relative to HNC, TE-HNC families had greater homework compliance and mid-week call participation. TE-HNC completers also required fewer weeks to achieve skill mastery and, in turn, to complete treatment than those in HNC without compromising parent satisfaction with treatment; yet, session attendance and completion were not different between groups. Future directions and clinical implications are discussed.

J o u r n a l P r e -p r o o f TECHNOLOGY-ENHANCED TREATMENT MODEL Results from a preliminary randomized control trial suggested that the Technology-Enhanced HNC (TE-HNC) prototype held promise as a cost-effective approach to increasing engagement in (i.e., session attendance, mid-week call participation, homework compliance) and efficiency of (i.e., fewer sessions to meet mastery criteria) services relative to HNC (Jones et al., 2014) . TE-HNC did not increase the likelihood that low-income families would complete treatment in the pilot; however, the relatively low drop-out rate in both groups (n = 2 TE-HNC; n = 2 HNC) suggested the need for further inquiry. This study aimed to replicate and extend that pilot work by designing and testing an integrated TE-HNC parent mobile application and therapist web-portal in a larger RCT. It was predicted that the TE-HNC group would have higher levels of engagement relative to HNC and that those gains would be achieved with greater efficiency of service delivery without significantly increased implementation costs. Given documented clinicians concerns regarding technology compromising treatment process (Anton & Jones, 2017; , we also compared parent satisfaction between the two groups.

English-speaking, low-income [< 250% of Federal Poverty Guidelines (FPL)] legal guardians and their 3 to 8-year-old children with clinically-significant problem behavior (Eyberg Child Behavior Inventory Problem >15 or Intensity >131; Eyberg & Pinus, 1999) participated (see Figure 1 ).

Although various measures of socioeconomic status have strengths and limitations (see Jones et al., J o u r n a l P r e -p r o o f TECHNOLOGY-ENHANCED TREATMENT MODEL 7 places, retail outlets, and social media with an overrepresentation of low-income parents; (b) healthcare, social service, and other agencies that serve low-income families; (c) local schools; and (d)

word-of-mouth (see Khavjou , Turner, & Jones, 2018; in press for details).

Given clinical concerns inherent in a treatment approach in which a child's symptoms may worsen before they improve (i.e., extinction burst), families were excluded if the parent had a current mood, psychotic, and/or alcohol or drug abuse/dependence (excludes marijuana use) disorder that should be the primary or more imminent focus of services.In addition, parents who had a current pending and/or prior substantiation of child abuse/neglect were excluded in order to optimize the safety of the child without substantively modifying the treatment manual in the context of research. Finally, families were excluded if the child had significant developmental and/or physical impairment that prohibited use of HNC (e.g., unable to hear parent's verbal use of skills or physically unable to do Time-Out).

Interested families contacted study staff who conducted a brief phone screen to determine initial eligibility (e.g., 3 to 8 y.o. child, low-income, clinically significant behavior). Phone-screen eligible families were then scheduled for a more extensive baseline assessment at a community-based clinic to obtain parent consent for self and child, to confirm eligibility criteria, and to gather more detailed demographic and psychosocial information. Baseline-eligible families were then randomized to HNC (McMahon & Forehand, 2003) or TE-HNC (Jones et al., 2010; and, thus, considered enrolled at the first session. Post-assessment procedures were similar to baseline. Families were paid $50 per assessment. All procedures were approved by the university's institutional review board.

J o u r n a l P r e -p r o o f TECHNOLOGY-ENHANCED TREATMENT MODEL 8 check-ins to assess progress and problem solve obstacles to home-based skill practice. In Phase I, Differential Attention, the parent is taught to: (1) Increase the frequency and range of positive attention (i.e., Attends, Rewards); (2) Eliminate instructions, questions, and criticism; and (3) Ignore minor inappropriate behavior (i.e. Ignoring). Parents practice these skills in the context of child-directed play (i.e., Child's Game), which they are also instructed to do at home for at least 15 minutes per day, as well as to use the skills throughout the day. In Phase II, Compliance Training, parents are taught the Clear Instruction sequence in order to maximize child compliance, as well as a non-physical consequence (i.e., Time-Out) for noncompliance, as well as safety-related behaviors (e.g., aggression).

Phase II skills are taught in the context of Parent's Game (i.e., clean-up task); however, parents are instructed to continue to practice Child's Game at home to maintain mastery of Phase I skills.

In addition to the standard HNC format and content, TE-HNC families also had access to a HIPAA-compliant, interactive system that allowed therapists (via web-portal) to monitor and tailor parent activity on the mobile application (Tantrum Tamers©), as well as the focus and pace of subsequent mid-week calls and sessions. Building upon the prototype functionality and content tested in the pilot study (Jones et al., 2010; , TE-HNC components included: (1) daily surveys of skills practice, which guided mid-week calls and sessions; (2) weekly video-recorded home practice, which afforded an opportunity for personalized feedback regarding skill development;

(3) daily text reminders (e.g., skill practice, appointments), as well as reinforcing messages regarding progress; (4) video calls with the family midweek to problem solve obstacles to skill practice and progress; and (5) skills videos series to model new parenting skills and share with other parents and/or coparents. In addition, based on recommendations of pilot families, a homework checklist was added to remind parents of daily and weekly assignments (e.g., assigned handouts, skill practice, days/times sessions).

Master's-level therapists were trained in and treated families in both the HNC and TE-HNC.

Training included establishing knowledge of the relevant manuals, reliability with the HNC mastery coding criteria, role-play and session observation and discussion, as well as weekly observation, Mid-week call participation. Therapists recorded family mid-week call participation weekly.

Mid-week call participation was calculated by the ratio of mid-week calls in which families participated to the total number of mid-week calls as initially scheduled. For example, if a family participated in five of the 10 mid-week calls as initially scheduled, again due to missed calls or rescheduling, they would have 50% mid-week call participation. Mid-week call participation was averaged for families in each group.

Homework compliance. Homework is a common practice element in children's mental health

generally; yet, there are well-documented challenges to relying on therapist-report of client's homework completion (e.g., Busch et al., 2010; Detweiler & Whisman, 1999; Primakoff, Epstein, & Covi, 1986) . Therefore, we developed an observer-based coding system to characterize parentresponses to therapist prompts, explicit in the therapist guide, regarding if and how often parents first Phase I skills-focused session (i.e., Attends); 2) first Rewards session (2 nd skill families learn in HNC program); 3) first Phase II session (i.e., Path A, Clear Instructions, the first phase in the Clear Instruction Sequence); and 4) the last session for which homework is assigned [i.e., Paths A (Clear Instruction), B (Warning), and C (Time-Out) of Clear Instruction Sequence]. The coding team, which was trained and supervised by a Master's-level graduate student, used a 3-point scale: 0 = parent did not do homework, 1 = parent did homework, but less than half of the possible days, and 2 = parent did homework at least half of the possible days. Sessions were initially reviewed as a group to allow coders to practice identifying where in the session the discussion of homework typically occurred (i.e., first 15-minutes), as well as to practice using the coding system in response to therapist check-in regarding home practice (e.g., "Were you able to practice Child's Game this week?" If yes, "how often?") until reliability with the master coder was consistently achieved. Then, selected sessions were coded by a single coder. Twenty-five percent of coded sessions were coded by 2 nd person to assess inter-rater reliability (83% observer agreement). Any code deviations were reviewed and reconciled by a 3 rd coder and/or consensus of the group. Homework completion was averaged for families by group.

Efficiency and program costs. Service efficiency was measured using the number of weeks and sessions required for each family to complete the program. Of note, only weeks in which no session occurred due to client reasons (e.g. cancellation, no show, unavailable), rather than therapist reasons (e.g., therapist sick) were included in analyses. Program costs were quantified using a payer approach.

Data on program costs were collected using Excel-based cost instruments that therapists and program staff used to report non-labor resources and time spent on various program activities ( technology-specific costs (e.g., watching home practice videos, reviewing survey responses). Phonerelated costs, such as phone cases, screen protectors, and monthly phone bills, were also tracked for TE-HNC participants because these were provided to families that participated in the study. Even though research-specific and phone-related costs are not expected to be incurred during the implementation of this intervention in community-based practice settings, these costs are important to consider to better understand the resources required to deliver a program in various settings.

J o u r n a l P r e -p r o o f TECHNOLOGY-ENHANCED TREATMENT MODEL 14 50% worked at least part-time. Children were 4.18 years old on average, approximately one-third (38.7%) were girls, and half (54.1%) were ethnic or racial minorities.

As shown in 

As shown in the for phone-related costs/family (see Table 3 ). Over and above these research costs, program delivery cost was approximately $300 per family that completed the program and most of that cost was for therapist time. Program delivery costs were similar between HNC and TE-HNC groups.

There were no significant differences in program satisfaction or ease of use of skills (see Table 2 ).

Patterns suggest that parents in TE-HNC rated the usefulness of the overall program higher than HNC (ES = .052). Components of TE-HNC were also all rated as easy/very easy to use and useful/very useful except daily reminders which were rated as neutral to somewhat useful (see Table 4 ).

This study aimed to improve BPT engagement and efficiency without increasing intervention costs or compromising consumer satisfaction. Similar to the pilot, families in TE-HNC were more likely to do their daily practice of Child's Game, as well as to use their skills throughout the day, than those in as well as on BPT homework in particular (e.g., Chacko et al., 2009; Chacko et al., 2013; Nock & Kazdin, 2003) , suggests that the primary reasons parents fail to complete homework are less specific to the therapist or the session and more related to remembering to do it and then finding the time and suggest that chronic socioeconomic strain not only increases the severity and frequency of stressors with which low-income families must cope, but also depletes reserves necessary to cope with those stressors. This study was completed prior to the COVID-19 pandemic; however, COVID-19 provides a context for precisely this point, given that lower-income families have been disproportionality affected by both the health and economic impacts of the virus. Thus, future work targeting service completion must continue to explore how technology can play a more central and perhaps tailored role as lowincome parents continue to navigate how best to seek and engage in mental health services for their children while also coping the emergence of often acute, unpredictable, and even devastating stressors.

Although TE-HNC did not improve rates of completion relative to HNC, TE-HNC completers mastered program skills in fewer weeks and sessions, although the latter was not statistically significant. One way to conceptualize this is that TE-HNC did not facilitate skill mastery per se (i.e.,

TECHNOLOGY-ENHANCED TREATMENT MODEL 17 number of sessions per skill determined by parent skill mastery) but instead reduced the overall amount of time (i.e., weeks) required for families to progress through the skills/sessions. This suggests that if we can keep families coming consistently each week and get them to complete then TE-HNC may allow them to do this more efficiently. Moreover, TE-HNC efficiency did not come at the cost of consumer satisfaction, which we know is a worry for therapists as technology increasingly intersects with mental health delivery (Anton & Jones, 2017; . Efficiency comes without significant added implementation costs as well, suggesting the promise of a cost-effective approach.

As with all research this project has limitations. First, we provided all families with phones and service plans, given data to suggest that 44% of low income users let service plans lapse due to finances (Pew Research Center, 2015) . While families with a mobile phone may have found carrying two phones inconvenient, it also did not prevent a sizeable portion of families from having technical issues. This affected our sample size, analyses, and interpretations, but also reflects the reality of consumer uptake of technology in general and telemental health will not be an exception. Second, costs limited us to choosing one platform and we chose the iPhone as at the time iOS allowed us to develop more functionality; however, price points suggest that low-income families are more likely to choose an Android phone. Third, technology-enhanced models have generally been recommended over technology-delivered or standalone technologies for intervention versus prevention work, particularly with more vulnerable populations (e.g., Mohr et al., 2013; Lindheim et al., 2015; Tate & Zabinski, 2004) ; however, technology-enhanced models still require weekly, clinic-based session which our findings further highlight are difficult for low-income families to continue. Fourth, our findings suggest that TE-HNC requires minimal additional time outside of session for therapists relative to

HNC; yet, further work must continue to examine uptake of and satisfaction with technology-enhanced treatment models particularly in publicly-funded clinics where low-income families are more likely to seek children's mental health services (Anton & Jones, 2017; . Fifth, we conceptualized supervision as a research cost and estimated time per family, rather than actual time by family or J o u r n a l P r e -p r o o f TECHNOLOGY-ENHANCED TREATMENT MODEL 18 group. We did this given discussion in the literature regarding the lack of consistency between the more intensive nature and clarified focus of supervision in clinical trials relative to what happens or may be feasible in community mental health settings. That said, that decision may be less relevant for front-line service settings that do provide more intensive supervision and would be interested in actual costs per family and by group. A final limitation is unrelated to technology and that is our decision to exclude parents with current psychopathology, as well as to rely on study-specific rather than community-based therapists which may limit generalizability (see Weisz et al., 2015 for a review).

This study also has strengths. First, there has been significant investment in technology-enhanced service delivery models. For example, the National Institute of Mental Health reports awarded 404 grants totaling 445 million dollars for technology-enhanced mental health interventions between fiscal years 2009 and 2015 (NIMH, 2017). While improving engagement has been cited as a primary rationale such work, few studies in child or adolescent mental health in general or BPT in particular assess or report whether engagement improves as a function of a technology-enhanced delivery model (Georgeson et al., in press) . Second, we focused on low-income families who are more likely to have a child with an early-onset BD and less likely to engage in mental health services; yet, it is important to note that we do not have a theory-driven reason to expect TE-HNC to work differently for higher income families. Third, like most (if not all) evidence-based treatments for children, homework compliance is critical to skill generalization in BPT; yet, rates of homework compliance (and other outof-session markers) are less commonly reported in the literature than other engagement measures (e.g., Chacko et al., 2016b; Chorpita, Daleiden, & Weisz, 2005; McMahon & Forehand, 2003) . We look at both in and out of session indicators of engagement in this study. Finally, given that HNC is one example of a family of evidence-based BPT programs with common history, theory, and practice elements, findings may be generalizable to other programs as well (see Kaehler, Jacobs, & Jones, J o u r n a l P r e -p r o o f TECHNOLOGY-ENHANCED TREATMENT MODEL 19

increase engagement in children's mental health (see Georgeson et al., in press; Jones et al., 2013 for sreview; also see Chacko et al., 2016a for promising pilot work in this area). Additional work is needed to further understand how to continue to optimize session attendance and program completion in particular, which is critical if all children, including low-income children, are to benefit fully. In addition, our work and that of others must continue to explore if, how, and when improved engagement is linked to improved clinical outcomes. In the case of mastery-based programs like HNC we have increased confidence that parents' skill use is improved and children's problem behavior reduced as a function of reliance on the mastery-criterion to determine progress in and completion of the program. Yet, prior pre-to-post pilot analyses has shown technology-enhanced approaches can bolster outcomes even relative to standard BPT (Jones et al., 2014) . It will also will be important to examine variability in the extent to which TE-HNC parents were motivated to and engage with the technology-enhancements in order to determine if greater or more consistent use is linked to improved outcomes Dewar et al., 2017) . Importantly, these latter two directions, symptom reduction and technology-use, can most richly be examined using time-intensive data analyses, which is increasingly a hallmark of the technology-enhanced treatment literature (see Baraldi, Wurpts, MacKinnon, Lockhart, 2014 for a review). Finally, others have highlighted the difficulties inherent in inconsistent definitions of engagement throughout the literature (Becker et al., 2018; Chacko et al., 2016b) . We chose to focus this examination of engagement on families who enroll in treatment, rather than those who are eligible but never enroll (Chacko et al., 2016b) . We believe that both directions are critically important, albeit slightly different in terms of the feasibility of clinician's time and resources in community settings. We developed our technology-enhancements with the goal of not substantively increasing therapist out-of-session activity for current cases. It is more difficult to consider how best to do this for cases who have perhaps had a general intake with an agency, but never initially met with the therapist or started BPT. That said, data on the implications of children failing to receive treatment highlights the importance of increasing families' access to and initiation of BPT as well.

Footnote 1 Variability in the total possible sessions (i.e., mastery-based nature of HNC), as well as number of number of sessions to drop-out, made it difficult to determine the appropriate numerator and denominator for the engagement variables (i.e., ratio of actual/possible sessions, mid-week calls, homework) for drop-outs for parallel ITT analyses and common missing data strategies could not remedy this. The last observation carried forward for example would mean that a family's last session attended would be included in the "number of weeks to completion" calculation, while their overall participation at the last session would be carried forward resulting in group-level statistics that were skewed towards a less conservative estimate of outcomes (i.e., greater participation and quicker treatment completion). Therefore, ITT analyses were conducted with raw counts, rather than suggest the same pattern of findings. The between-group differences for mid-week calls and homework are no longer statistically significant, however, suggesting that the obtained engagement boost for TE-HNC relative to HNC is more pronounced when using percentages that reflect the mastery-based nature of the program and for families who complete treatment. Ease of use 7.0 Very Easy Usefulness 6.0 Useful Convenience of App 7.0 Very Convenient Range of response 1-7 with 7 representing easier and more useful. Given that scores were positively skewed, medians are presented rather than means.

 Treatment engagement is a perennial challenge in children's mental health  BPT engagement is particularly difficult for low-income families  Results suggest technology-enhanced BPT can improve some engagement outcomes  Additional personalization of technology-enhanced BPT may further improve engagement

Evidence-based practices addressed in communitybased children's mental health clinical supervision

Adoption of Technology-Enhanced Treatments: Conceptual and Practical Considerations

Parent-therapist alliance and technology use in behavioral parent training: A brief report

Caregiver use of the core components of technology-enhanced HNC: A case series analysis of low-income families

Evaluating mechanisms of behavior change to inform and evaluate technology-based interventions

More practice, less preach? the role of supervision processes and therapist characteristics in EBP implementation. Administration and study

Helping the Noncompliant Child: An Assessment of Program Costs and Cost-Effectiveness

Moderators of outcome in a brief family-centered intervention for preventing early problem behavior

Engagement in Technology-Enhanced Interventions for Children and Adolescents: Theory, Measurement, and Outcomes

Provider responses to emergent life events during evidence-based treatment implementation. Administration and Policy in Mental Health and

Emergent Life Events During Youth Evidence-Based Treatment: Impact on Future Provider Adherence and Clinical Progress

The effectiveness of telemental health: a 2013 review

Review of the Key Considerations in Mental Health Services Research: A Focus on Low Income Children and Families

Harnessing Innovative Technologies to Advance Children's Mental Health: Behavioral Parent Training as an Example

Technology-Enhanced Program for Child Disruptive Behavior Disorders: Development and Pilot Randomized Control Trial

Distilling common history and practice elements to inform dissemination: Hanf-Model BPT programs as an example. Clinical Child and Family systematic review

Does socioeconomic status matter? A meta-analysis on parent training effectiveness for disruptive child behavior

Mobile technology boosts the effectiveness of psychotherapy and behavioral interventions: a meta-analysis

A meta-analysis of parent training: Moderators and follow-up effects

Psychological perspectives on pathways linking socioeconomic status and physical health

Helping the Noncompliant Child: Family-Based Treatment for Oppositional Behavior

Epidemiology of mental disorders in children and adolescents

Engagement of families in treatment for child conduct problems

Randomized control trial of a brief intervention for increasing participation in parent management training

Prediction of differential adult health burden by conduct problem subtypes in males

Phone coaching in Dialectical Behavior Therapy: frequency and relationship to client variables

The smartphone difference

Mobile Fact Sheet

Effects of early family/parent training programs on antisocial behavior and delinquency

A meta-analysis update on the effects of early family/parent training programs on antisocial behavior and delinquency

Annual Research Review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents

Homework compliance: An uncontrollable variable in cognitive therapy outcome research

Constance "Connie" Hanf (1917-2002): The mentor and the model

Predictors of parent training efficacy for child externalizing behavior problems -a meta-analytic review

The intention to treat analysis is not always the conservative approach

New directions and challenges in preventing conduct problems in early childhood

Evidence base updates: The evolution of the evaluation of psychological treatments for children and adolescents

Computer and internet applications for psychological treatment: Update for clinicians

How to calculate effect sizes from published research articles: A simplified methodology

Smartphone ownership and perspectives on health apps among a vulnerable population in East Harlem

Testing the adaptation to poverty-related stress model: predicting psychopathology symptoms in families facing economic hardship

Shrinking the gap between research and practice: Tailoring and testing youth psychotherapies in clinical care contexts

Computer-assisted intervention improves patient-centered diabetes care by increasing autonomy support