key: cord-0795335-jn3xwyww
authors: Larson, Madeline; Cook, Clayton R.; Sullivan, Margaret M.; Lyon, Aaron R.; Lewis, Cara C.
title: Validation and Use of the Measure of Effective Attributes of Trainers in School-Based Implementation of Proactive Classroom Management Strategies
date: 2022-01-12
journal: School Ment Health
DOI: 10.1007/s12310-022-09499-1
sha: a144ca146736a37c55cccb9d5f432970ca7bf8e1
doc_id: 795335
cord_uid: jn3xwyww

In-service training is a critical and frequently utilized implementation strategy to support the adoption and delivery of evidence-based practice (EBP) across service settings, but is characteristically ineffective in producing provider behavior changes, particularly when delivered in single exposure didactic events. EBP trainers are in a strategic position to leverage their trainee-perceived characteristics to influence trainees’ attitudes, motivation, and intentions to implement, and ultimately increase the likelihood of successful uptake of skills. The purpose of this study was to extend research on the measure of effective attributes of trainers (MEAT) by examining its underlying factor structure and reliability in the context of in-service EBP training for teachers (i.e., structural validity). This study also examined the predictive validity of the MEAT by examining relationships with a measure of teacher intentions to implement EBPs following a standardized training experience (i.e., predictive validity). An exploratory factor analysis (EFA) was employed to determine the latent factors (i.e., subscales of characteristics) that underlie the data. Additionally, a forward selection, stepwise regression was conducted to determine the extent to which trainer attributes could explain variance in intentions to implement. Results indicated that the MEAT was a valid and reliable measure to examine trainer attributes in school settings. Moreover, findings suggested that trainer attributes, particularly those related to trainee perceptions of the trainers’ welcoming disposition (i.e., related to trainers’ warm, positive temperament and internal character traits), were significantly associated with trainees’ intentions to implement the trained upon EBP.

In-service training has been identified as a key implementation strategy to support the adoption and delivery of evidence-based practice (EBP) across service settings (Beidas & Kendall, 2010; Han & Weiss, 2005) . However, in-service training (hereinafter referred to as training) is characteristically ineffective in producing changes in workplace behavior. For example, estimates indicate that only 10% of what is taught in training is transferred to actual on-thejob use (Fixsen et al., 2009; Rogers et al., 2002) . While a large body of evidence indicates that training are generally ineffective in producing changes in trainee behavior specifically, there is an overall lack of research examining facets of effective training that influence key mechanisms of change that preceded implementation outcomes (i.e., intentions to implement), which is a widely cited barrier to successful EBP implementation (Kazdin, 2008; McHugh & Barlow, 2010; Wandersman et al., 2008) . Consequently, efforts to identify features and characteristics of effective training have increased (e.g., Dimeff et al., 2009; Long, 2008; Lyon et al., 2011) . The purpose of this study was to extend the research on training as a core implementation strategy by cross-validating a measure of trainer attributes in the education sector and examining the impact of trainer attributes on providerlevel mechanisms (i.e., behavioral intentions to implement) of implementation outcomes (e.g., adoption, fidelity).

The dearth of empirical research dedicated to investigating features of effective in-service training is not unique to a particular service setting. Multiple disciplines grapple with how to adequately train their workforces to adopt and deliver EBPs as an approach to enhance client outcomes (e.g., Davis & D'Lima, 2020; Grimshaw et al., 2002) . As such, researchers have articulated empirically supported models of training transfer (i.e., the use of trained skills on-the-job) to characterize the key facets of training that can inform precise approaches that produce lasting provider behavior change. Across disciplines, the common elements of successful training include: (1) attention to organizational factors (e.g., climate, culture, support); (2) intentional strategies used during and after training; and (3) personnel characteristics or attributes (e.g., trainees, supervisors/leaders, and trainers).

Indeed, seminal research conducted in the field of human resource management by Baldwin and Ford (1988) suggested that training yielded employee behavior change only when: (a) there was an intentional focus on training design/ delivery and enhancing the work context (i.e., climate, support); (b) activities occurred during and after training to support application of knowledge and skills beyond the training; and (3) an emphasis was placed on cultivating the optimal trainer and supervisor characteristics and actions that facilitate training transfer. With regard to EBP implementation specifically, Beidas and Kendall's (2010) work on EBP training in the mental health sector indicated that the transfer of EBPs from training to on-the-job use was enhanced when a systems contextual approach was taken, which emphasized attention to organizational, provider, and client variables as part of the professional development process. In addition, Lyon and colleagues (2011) put forth a conceptual framework of training, consisting of two complementary processes that have been previously identified as essential to effective implementation (Fixsen et al., 2005; Joyce & Showers, 2002) : (1) methods through which professionals learn new skills and behaviors; and (2) systems of ongoing feedback or support to refine and further develop the application of those skills (i.e., consultation and coaching).

The common elements of effective training have been unpacked by researchers to better understand and test specific approaches that impart provider behavior change and ultimately improve the transfer of EBP training to on-thejob use. With regard to organizational factors, research in the mental health sector has established that cultivating implementation climate (i.e., the shared perception of policies, practices, and procedures employees experience and the behaviors they observed being reward, supported, and expected) and culture (i.e., shared beliefs, attitudes, and assumptions that characterize a setting, on implementation outcomes following training through supportive leadership; Aarons & Sawitzky, 2006; Ehrhart et al., 2014; Schneider et al., 2013) has a significant impact on the uptake and use of EBPs following training.

In addition, specific approaches and techniques occurring during and after training have been associated with a variety of training outcomes (i.e., changes in professionals' knowledge, understanding, or skill implementation), and are at the core of most training research. In their review of strategies that improved transfer, Lyon and colleagues (2011) provided detailed descriptions of specific training approaches and evidence of their use across multiple disciplines. Training approaches used during and after training identified in their review included academic detailing, interprofessional education, problem-based learning, coaching, point-of-care reminders, and self-regulated learning. Of the approaches identified, intensive coaching and mentoring as well as the use of reminders or checklists was moderately effective and could be implemented at a low cost following training. While these are effective and complementary strategies that frequently accompany training, initial meetings or training sessions focused on interactive didactics (i.e., active learning strategies) yield significant gains in trainee acquisition of knowledge and skills (Markey & Schattner, 2001; Valenstein-Mah et al., 2020) which are important precursors for actual behavior initiation and building skill fluency.

Another factor within training sessions that may influence training effectiveness focuses on manipulating the characteristics or attributes of personnel central to the training process has been linked to improved training outcomes. At the provider level, changes in trainee attitudes and beliefs, self-efficacy, and motivation prior to and following training have been associated with skill transfer (Cook et al., 2015) . Moreover, prior experience with training has been shown to influence a trainee's transfer of skills to on-the-job use (James et al., 2001) . Supervisor characteristics are also essential to provider behavior change. A strong literature base has been established across disciplines on the impact of transformational leadership qualities on employee work behavior (Green et al., 2013) . More recently, implementation scientists have parsed out specific implementation leadership characteristics that are predictive of trainee behavior change with respect to the implementation of EBP in the mental health sector. For example, providers who perceive their supervisors as "humble" are more likely to engage in quality improvement activities (Aaronset al., 2016; Farahnak et al., 2020) . Additionally, implementation leadership characterized by proactivity, perseverance, availability, and support has been associated with provider behavior change following training .

While trainee and supervisor characteristics, and other approaches to training, have been the focus of study, little attention has been given to the attributes of trainers themselves. Given consistent findings from social psychology that demonstrates people form rapid judgments of others that influence their receptivity to new ideas and behavior towards others (Ambady & Rosenthal, 1992; Amodio, 2019) , this is an understudied and potentially promising avenue for implementation research seeking to optimize transfer of inservice training. Moreover, educational research consistently demonstrates that adult learners are more likely to acquire knowledge and skills from instructors with certain favorable attributes (Beavers, 2009 ).

EBP trainers are in a strategic position to leverage their personal characteristics (i.e., trainee-perceived aspects) to influence trainees' attitudes, motivation, and intentions to implement (Ajzen, 1991) , and ultimately increase the likelihood of successful EBP implementation. Indeed, evidence suggests that particular trainer characteristics may have an influence on trainee knowledge acquisition and behavioral intentions following training that could not only influence their implementation behavior, but their use of subsequent implementation supports that may target outcomes such as intervention fidelity. For example, Towler and Dipboye (2001) investigated the effects of trainer expressiveness and organization on recall and problem-solving tests immediately and 2 days following training. Findings showed that participants had the highest recall after receiving training from someone who was expressive and organized. In addition, Yelon and colleagues (2004) found through their qualitative work that trainees' intentions to transfer were related to the way trainers treated them and how they felt towards the trainer during instruction. While researchers continue to call for studies focused on trainer characteristics, few studies to date have examined the impact of trainer attributes on mechanisms of provider behavior change or actual implementation behavior (i.e., adoption and fidelity), specifically with regard to EBP uptake and use.

To address this extant void in research, Boyd et al. (2017) executed a study which aimed to: (1) create and validate a measure of trainer attributes [i.e., the Measure of Effective Attributes of Trainers (MEAT)]; and (2) assess the effects of trainer attributes on intentions to use skills learned in an undergraduate laboratory setting. In their study, Boyd and colleagues found that the MEAT was composed of two underlying factor structures, which described trainer attributes related to "Charisma" and "Credibility." The first factor, labeled "Charisma," contained items related to characteristics that facilitate a positive personal relationship with the trainee (e.g., friendly, warm), and the second factor, labeled "Credibility," contained items related to characteristics that emphasize the qualification of the trainer (e.g., professional, experienced). Boyd et al. found that the MEAT was a valid and reliable measure.

While Boyd and colleagues established the initial construct validity and technical adequacy of the MEAT, it was administered to undergraduate students and not as part of an active implementation effort to change routine practice. Thus, it is important for additional research to determine whether the factor structure, reliability and predictive validity generalize to different people, service contexts (e.g. schools), and real-world implementation efforts. There is a need for additional studies that evaluate the extent to which the original factor structure is confirmed and possesses utility in predicting the impact of training on key mechanisms of behavior change. The current project was designed to conduct such an evaluation to support the implementation of universal behavioral health programming in the education sector. Such research is key to advancing the multidisciplinary field of implementation science by determining whether specific constructs and instruments are context dependent or independent.

Schools are the most common site for the delivery of behavioral health services to children and adolescents in the USA. Indeed, almost 70-80% of youth receiving behavioral health services (i.e., mental health and substance abuse services; Blount et al., 2007) receive their care in schools (Costello et al., 2014; Farmer et al., 2003; Langer et al., 2015) . In the education sector, behavioral health services are provided along a continuum, ranging from universal prevention (e.g., practice classroom management strategies, social emotional learning curriculum) to selected and indicated interventions (e.g., school-based group or individual cognitive-behavioral therapy; Bruns et al., 2016) . Majority of schools delivering a continuum of care for students have a diverse school-based behavioral health workforce (e.g., teachers, behavioral health providers, support staff; Lyon et al., 2018) .

Teachers are often the primary implementers of universal, behavioral health prevention in schools (Forman et al., 2009) , providing care to all students in the school setting. Universal behavioral prevention can be provided through a variety of mechanisms (e.g., social-emotional learning curriculum, school-wide positive behavior interventions and supports). Proactive classroom management is a preventionoriented and intentional approach to promoting high levels of academic engagement as incompatible to classroom problem behaviors, limiting the amount of reactive, negative interactions between teachers and students (Rathvon, 2008) . Prior research has demonstrated the efficacy of several proactive classroom management strategies to promote a range of positive social, emotional, and behavioral outcomes for students Oliver et al., 2011; Simonsen et al., 2008) . Proactive classroom management strategies represent a set of evidence-based, prevention-oriented strategies that are considered low cost and high yield, making them ideal for implementation in low resourced school settings Wehby & Lane, 2009 ).

While the continuous use of proactive classroom management strategies has the potential to improve student outcomes, educators often enter the workforce with limited knowledge of student behavioral health needs or EBPs to enhance student functioning (Christofferson & Sullivan, 2015; Owens et al., 2014 ). To provide system-wide support to deliver EBPs across multiple levels of care, school-based behavioral health consultants are frequently leveraged to act as EBP champions (within school buildings) or intermediaries (across school buildings), who support implementation of behavioral health programs through a variety of activities (e.g., training and coaching for individual schools or entire districts; Lyon et al., 2018) . Given consistent evidence that school-based behavioral health services, while accessible, are unlikely to be evidence-based (Forman et al., 2009; Owens et al., 2014; Rones & Hoagwood, 2000) or delivered with insufficient fidelity to improve student and school functioning (Gottfredson & Gottfredson, 2002) , individuals in this role are critical but understudied as key facets of organizational change.

Like other sectors, research on training in the education sector (i.e., professional development) has focused narrowly on learning strategies (e.g., role play, feedback), follow-up supports (i.e., coaching support and consultation), and organizational factors (e.g., implementation leadership and climate) associated with EBP uptake and use (i.e., attitudes and intentions). Notwithstanding the importance of these factors, school-based behavioral health consultants embedded within school systems serve as potential champions or intermediaries who can facilitate the successful uptake and delivery of EBP via training and consultation. However, no studies in education to date have investigated the attributes of behavioral health trainers that impact key mechanisms of teacher behavior change within the context of EBP training, which is a frequently used implementation strategy in schools regardless of whether follow-up supports are available or provided. Lacking this information, a critical component of training effectiveness could be missed. Moreover, failure to recruit behavioral health consultants who possess specific attributes linked to successful transfer could limit the impact of training as a core implementation strategy and ultimately undermine changes in youth behavioral health outcomes. To further research on the impact of trainer attributes on provider-level outcomes in the education sector, cross-validation of the MEAT in the context of a real-world implementation effort in schools provides a useful next step.

The purpose of this study was to extend research on the MEAT by examining its underlying factor structure and reliability in the context of in-service EBP training for teachers (i.e., structural validity). This study also examined the predictive validity of the MEAT by examining relationships with a measure of teacher intentions to implement EBPs following a standardized training experience (i.e., predictive validity). This study had two primary aims:

1. Explore the underlying factor structure of the MEAT from data collected in an implementation effort in the education sector and gain evidence of structural and internal validity; 2. Examine predictive validity of the MEAT by examining its relationships with intentions to implement EBPs following a standardized in-service training experience.

Twenty-one schools participated in EBP trainings (see Procedures) in the Midwestern and Western USA. Schools participating in the study ranged from small to large with regard to student enrollment (min.: 6790; max.: 37,971). The participating schools had majority White/Caucasian enrollments. Level of eligibility for free or a reduced-price school lunch at participating schools ranged from 24 to 86%, evidencing a wide range of socioeconomic advantage/disadvantage across schools represented in the sample. Trainers ranged in age from 29 to 60 years old (M = 40.95; SD = 8.87) and had an average of 16.95 years of experience (SD = 9.17) as an EBP trainer. Prior to training, five teachers from each unique training experience were randomly sampled to complete the MEAT following training (n = 105). With regard to race and ethnicity, participating teachers identified as White (61%), Asian (16%), African American (8%), Hispanic/Latino (7%), and other (9%). Teachers ranged in years of experience from one to 32 years (M = 11.33; SD = 7.73).

The MEAT (Boyd et al., 2017 ) is a 33-item measure designed to assess the effective attributes of trainers. Items are endorsed on a five-point, Likert-type scale (1 = Very slightly/not at all to 5 = Extremely), and exploratory factor analysis yielded two subscale scores: Charisma and Credibility. Items on the Charisma subscale capture trainee perceptions of characteristics that facilitate a positive personal context, while items pertaining to the Credibility score relate to characteristics that emphasize the credibility and qualification of the trainer. Initial research on the MEAT indicated that the internal consistency of the subscales was excellent (Charisma, α = .98; Credibility, α = .91; Boyd et al., 2017) .

Participants completed a two-item measure following the EBP training, constructed using Ajzen's widely used manual for creating reliable and valid measures based on the Theory of Planned Behavior (TPB; Ajzen, 2002) . The measure captures teachers' behavioral intentions to implement the trained upon evidence-based proactive classroom management practices. The two items consisted of "I intend to implement the proactive classroom management strategies covered in the training" and "I mean to make every effort to deliver the proactive classroom management strategies with fidelity (that is, to implement the intervention step-by-step in the way it has been shown to be effective)." Items were rated on a seven-point scale ranging from Completely Agree to Completely Disagree. The items were summed to create a total intentions to implement score and demonstrated a strong inter-item correlation (r = .91).

School districts contacted the second author to engage in a multi-year, district-wide initiative, to roll out the implementation of multi-tiered approach to facilitate the delivery of a continuum of EBPs targeting the promotion of students' social, emotional and behavioral outcomes. This implementation effort occurred prior to the COVID-19 pandemic; as such, this study was completed when students and teachers were engaged in continuous, in-person instruction without widespread disruption. The implementation effort was designed based on a horizontal (within tiers of support; e.g., integration of universal supports delivered to all students) and vertical (across tiers of support; e.g., integration of targeted services to match identified students to precise services) approach to integrating EBPs (Domitrovich et al., 2010) to facilitate the delivery of a continuum of supports based on student need and data-driven decision making to engage in problem-solving at multiple levels (school-wide, class-wide, and individual student). This model is discussed in greater detail in other sources (Cook et al., 2010; Sprague et al., 2008) .

The implementation initiative was designed as a 3-year project in which participating schools build capacity over time to ultimately build the model to scale (Aarons et al., 2011) . The initiative was grounded in a train-the-trainer model (i.e., experts in specific EBPs identify and provide instruction to identified personnel, who then provide training to others in their organization; McHugh & Barlow, 2010; Powell et al., 2015) , which focused on increasing internal capacity within each school by providing site-based dissemination and implementation teams with ongoing professional development and follow-up coaching over the course of 3 years. External consultants trained site-based dissemination and implementation teams to build internal expertise and capacity within each school building to develop strategic action plans that were tailored to their context and monitor efforts to facilitate data-driven continuous improvement efforts. Most relevant to the current study were the schoolbased behavioral health consultants (i.e., trainers) who were endogenous to each district who were assigned to specific schools (approximately five schools for each consultant) and responsible for supporting site-based implementation planning, problem-solving barriers to implementation and delivery of ongoing and targeted professional development (i.e., training) to school staff in the form of training on specific EBPs. Trainers' professional roles included: school psychologist, school social worker, MTSS coordinator, behavior specialist, special education director. Trainers (n = 21) were unique to each training experience. Slightly more than half of EBP trainers were female (n = 13; 62%) and held a Master's degree (n = 18; 85%).

This study leveraged the training component of consultants' responsibilities specifically to explore whether trainer attributes were associated with teachers' intentions to implement EBPs following training. One of the standardized trainings consultants (i.e., trainers) received support to deliver was on the topic of proactive classroom management, which is an approach to integrate prevention-oriented EBPs to promote student academic engagement and reduce incidents of problem behavior (e.g., Hawkins & Weis, 1985; Lewis & Sugai, 1999; O'Donnell et al., 1995; Simonsen et al., 2008) . The training consisted of three parts: (1) rationale for proactive versus reactive classroom management, (2) presentation of specific evidence-based, proactive classroom management strategies, and (3) opportunities to plan application of strategies and ask questions for clarification. To gather data for this study, trainers identified one of their EBP training sessions with teachers and a researcher randomly selected a minimum of five teachers to complete the MEAT and intentions to implement measures following the training session.

The consultants (i.e., trainers) were provided with handouts (i.e., implementation scripts, fidelity rubrics) and a PowerPoint template to support teachers' delivery of the proactive classroom management training. The training session for consultants lasted 3 h and was designed to model the standardized training they would deliver to teachers at their assigned school sites. The training utilized a tell-showdo-feedback approach in which participants were provided a rationale and description of specific proactive classroom management strategies (i.e., tell), then saw them modeled (i.e., show), and finally were provided with opportunities to engage in role playing (i.e., do) and receive feedback based on their performance (i.e., feedback). The aim of the standardized training delivered to teachers by behavioral health trainers was to provide them with a menu of classroom-based EBPs they could select from to advance the implementation of prevention-oriented strategies in their classroom to create more positive and proactive learning environments to optimize student behavioral health functioning.

The EBPs included as part of the standardized training were a suite of proactive classroom management strategies. Specifically, the training integrated content and supplemental resources to support teachers' selection, adoption and delivery of eight proactive classroom management strategies: (1) positive greetings at the door, (2) behavior specific praise, (3) providing numerous opportunities to respond, (4) teach, review, and reinforce behavioral expectations, (5) precorrection, (6) attention signals, (7) 5-to-1 ratio of positive to negative interactions, and (8) choice making. Table 1 provides descriptions and supporting literature for each EBP.

Analyses for structural validity and reliability were conducted in SPSS; tests were two-tailed with p values set at .05. An exploratory factor analysis (EFA) was employed to determine the latent factors (i.e., subscales of characteristics) that underlie the data (Fabrigar et al., 1999) . Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett's test of sphericity both suggested the suitability of performing an EFA (Kaiser, 1974) . A principal axis factoring analysis was selected with an oblique Promax rotation. Decisions were made to create a simple structure with no crossloading items. According to de Winter et al. (2009) , sample size, number of factors retained, number of items, and factor loading contribute to factor recovery. To maximize the reliability of factor recovery, items were retained only if they had: (1) eigenvalues greater than one; (2) eigenvalues greater than the point at which the slope of decreasing eigenvalues approaches zero on the scree plot; (3) loadings above .60 on the primary factor (Floyd & Widaman, 1995) ; and (4) loadings less than .32 on additional factors (Costello & Osborne, 2005) . Factors that met all four criteria, and were made up of three or more items (Raubenheimer, 2004) , were retained. Parallel analysis was also carried out in R to support decision-making regarding the number of factors to retain.

Cronbach's coefficient (α) was calculated for the derived subscales to assess internal consistency.

Bivariate correlations were used to examine relationships between predictor and outcome variables. Then, a forward selection, stepwise regression was conducted in R to determine the extent to which trainer attributes could explain variance in intentions to implement. Variables were entered into the model according to Akaike information criterion (AIC; Akaike, 1973) , which was utilized to determine best model fit. The model with the lowest AIC value was selected to examine the extent to which trainer attributes could explain the variation in intentions to implement. Finally, the differential effect of each predictor was examined by running a fourth model to calculate changes in adjusted R 2 when controlling for each predictor variable.

Bar tlett's test of spher icity was signif icant (χ 2 (528) = 6185.61, p < .001) and the KMO measure was "marvelous" (KMO = .91; Fabrigar et al., 1999) , deeming EFA to be suitable. A principal axis factoring analysis with an oblique Promax rotation was utilized, reflecting a nonorthogonal approach that allows derived factors to correlate with one another. The first solution resulted in three first order factors, explaining 66.88%, 12.92%, and 2.46% of the variance, respectively. However, the third factor was weak on a few accounts. First, the third factor accounted for a negligible proportion of variance (2.46%). Second, only two unique, highly correlated items loaded onto the third factor (i.e., Organized and Prepared). Third, findings from the scree plot and a parallel analysis (Fig. 1) suggested exclusion of the third factor. Considering that research indicates factors should be made up of three or more items (Raubenheimer, 2004) , these items were excluded from further analyses but warrant further investigation (see "Discussion" section).

The second solution resulted in a two-factor structure with three items that crossloaded on two factors above .32 (i.e., Accessible, Likeable, Sociable). Crossloading items were excluded resulting in a third solution, which included one additional crossloading item (i.e., Approachable). This item was excluded and a fourth solution was obtained. The fourth solution contained one crossloading item (i.e., Respectful), which was excluded, resulting in the final solution. The final solution contained two factors with 13 items loading onto the first factor and 12 items loading onto the second factor ( Table 2 ). The first factor, labeled "Engaging Presentation," contained items Behavior specific praise Contingently and specifically acknowledging and recognizing students for exhibiting specific desired behaviors in class Briere et al. (2015) Numerous opportunities to respond Actively engaging students in any type of instruction by prompting or soliciting responses (e.g., verbal, gestured, written) from students Simonsen et al. (2010) Behavioral expectations Ensuring students clearly understand what expected, appropriate behavior looks like in order for students to be successful in class and recognizing and acknowledging students for doing so Johnson et al. (1996) , Sharpe et al. (1995) Precorrection Frontloading situations, in which there is an anticipation of potential problem that the teacher may react to student, with pre-teaching and encouragement of appropriate, desired behaviors Colvin et al. (1993) Attention signals Prompting students to provide their attention after they have been released to engage in non-teacher directed activities (e.g., small group, independent work, free time) Lewis and Sugai (1999) 5-to-1 positive to negative interactions Keeping track of the ratio of positive to negative interactions between teachers and their students to ensure that they engage in five positive exchanges with students (e.g., positive greeting, empathy statement, praise statement, mutual laughter) for every one negative interaction (e.g., criticism, disapproval statement, discipline for behavior)

Sugai and Horner (2002) Choice making Facilitating student autonomy in the classroom through providing opportunities for students to exert their preferences by selecting specific aspects of how they participate in class Dunlap et al. (1994) Fig . 1 Parallel analysis and scree plot used for factor retention (e.g., Entertaining, Intellectually Stimulating, Humorous) related to trainers' captivating outward expression and engaging presentation delivery style. The second factor, labeled "Welcoming Disposition," contained items (e.g., Humble, Trustworthy, Able to Listen) related to trainers' warm, positive temperament and internal character traits. The engaging presentation subscale accounted for 63.92% of the variance and the welcoming disposition subscale accounted for 15.63% of the variance, together accounting for 79.55% of the variance. Table 2 provides the factor loadings.

The internal consistency of the resulting MEAT subscales was excellent: Engaging presentation, α = .98; welcoming disposition, α = .97.

Bivariate correlations indicated moderate, positive relationships between predictor and outcome variables (Table 3) .

Intentions to implement was positively and moderately related to both trainer welcoming disposition (r = .61) and engaging presentation (r = .50); additionally, the two predictors were moderately related to each other in a positive direction (r = .61). All relationships were statistically significant (p < .01). Next, a forward selection, stepwise multiple regression was conducted in R to determine the extent to which welcoming disposition and engaging presentation could explain variance in teachers' intentions to implement the trained upon EBP. Descriptive statistics for variables that were candidates for entry into each model are provided in Table 4 . Assumptions of linearity, conditional homogeneity of variance, and normal distribution of errors were met (Field, 2009; Gravetter & Wallnau, 2014; Trochim & Donnelly, 2006) . Multicollinearity between the predictors was low, as indicated by the variance inflation factor (VIF = 1.58; Montgomery & Peck, 1992) . AIC was examined to select the optimal regression model. All covariate and predictor variables were included in analyses to test model specification. An empty model was run first, AIC = 175.41, t(104) = 45.34, p < .001. Next, welcoming disposition was added and improved the model considerably, AIC = 129.43, F(103) = 59.67, p < .001. The third model included engaging presentation, which improved the model further, AIC = 126.98, F(102) = 33.03, p < .001. Control variables (Table 4 ) did not further specify the model and were thus excluded from further analyses. Comparisons of AICs suggested that the optimal model included engaging presentation and welcoming disposition alone, AIC = 126.98. The final model indicated that both engaging presentation and welcoming disposition significantly predicted intentions to implement, F(102) = 33.03, p < .001 (Table 5) , explaining approximately 38% of variance in intentions to implement overall.

To examine the differential effect of each predictor, change in adjusted R 2 (Δ adjusted R 2 ) was computed. A fourth model was run to examine the change in adjusted R 2 from a reduced model with Engaging presentation as the only predictor (Table 5) . When controlling for Engaging presentation, welcoming disposition explained 14% of unique variance in intentions to implement beyond that of Engaging presentation. Conversely, Engaging presentation accounted for 2% of unique variance in intentions to implement beyond that of welcoming disposition. Thus, the effect size for welcoming disposition (Δ adjusted R 2 = .14) was approximately 7 times larger than the effect size for Engaging presentation (Δ adjusted R 2 = .02), indicating that welcoming disposition contributed more to the prediction of intentions to implement than Engaging presentation.

Training is considered a core implementation strategy (Fixsen et al., 2009 ), yet limited research has examined attributes of trainers that predict key mechanisms of provider behavior change. This study extended the research on the Measures of Effective Attributes of Trainers (MEAT) by examining evidence of its reliability and validity when administered in the education sector as part of an active implementation effort. Consistent with previous research (Boyd et al., 2017) , this study revealed two first order factors; however, the item loadings and meaning of the factors from the original research (i.e., Charisma and Credibility) differed from those derived in this study (i.e., Welcoming Disposition and Engaging Presentation). This difference in item loadings and factor meaning likely resulted from the study sample, as the original study was completed in a convenience sample of college students with a contrived stimulus, while the current study was conducted in the context of a real-world implementation effort, offering an opportunity to test the measure in the setting it was ultimately intended to be utilized. Results from the current study also suggested that trainer attributes, particularly those related to trainee perceptions of the trainers' welcoming disposition, were significantly associated with trainees' intentions to implement the trained upon EBP. This finding takes on importance in light of research indicating that behavioral intentions are a mechanism of behavior change (Ajzen, 1991) , and research linking behavioral intentions to implementation outcomes (Webb & Sheeran, 2006) . Findings also indicated a potential third factor related to trainee's perceptions of trainer organization and preparation. Indeed, previous research has indicated that trainer expressiveness coupled with their technical abilities, such as organization, may impact retention of important information related to the content being presented (Towler & Dipboye, 2001) . Moreover, findings are consistent with other disciplines that have identified spokesperson affect and disposition as significant predictors of attitudes and intentions to engage in a behavior (e.g., Sharma & Levy, 2003) .

This study has implications for the selection and recruitment of EBP trainers. If certain trainer attributes are found to be associated with desired training outcomes (e.g., knowledge and skill acquisition, intentions to implement, positive attitudes towards EBP), the MEAT could be used to select trainers to enhance the impact of training on adoption and implementation. Specifically, there is a growing interest in "train-the-trainer" approaches in which experts in specific EBPs identify and provide instruction to identified personnel, who then provide training to others in their organization (McHugh & Barlow, 2010; Powell et al., 2015) . In fact, many EBP purveyors offer train-the-trainer options as a way of scaling up implementation (e.g., cognitive-behavioral therapy; Nakamura et al., 2014) , which has been shown to yield superior adherence to treatment than self-study alone (McHugh & Barlow, 2012) . The MEAT could be used as a tool to inform trainer selection efforts or audit and feedback systems that support trainers to cultivate specific characteristics that are likely to influence key mechanisms of provider behavior change across a range of settings. Considering that the welcoming disposition subscale had a stronger association with trainee intentions to implement, EBP purveyors or organizations employing trainers may focus on cultivating trainer skills that foster an optimal learning environment. This could be similar to findings in other fields, such as the business and mental health sector, that underscore the importance of leaders who develop psychological safety in groups (Edmondson, 2018) and therapists who develop strong therapeutic alliances with clients (Lavik et al., 2018) which act as mechanisms for risk taking, behavior change, and learning. Moreover, trainee-perceived characteristics captured by the MEAT could be directly relevant to consultants or coaches who follow-up training with additional implementation supports that pick up where training leaves off. Before the MEAT can be used for these purposes, strong evidence for predictive validity must first be established, as well as a deeper assessment of trainer attributes that influence specific mechanisms related to provider-level implementation outcomes.

Findings from this study also have implications for the continued study of the attributes of key personnel associated with organizational change. More specifically, the results of this study suggest that there may be cross-cutting attributes of individuals who are in facilitative positions, such as trainers, supervisors, and consultants, that may influence providers' EBP implementation. This is aligned with previous literature documenting leadership attributes that are associated with organizational climate, workforce satisfaction, and innovation implementation (Aarons et al., 2016) . Future research should examine whether there are common attributes across personnel in supportive roles, or whether there is divergence in attributes depending on the position (e.g., trainer vs. supervisor) that could differentially inform selection decisions to enhance both better implementation and service recipient outcomes.

Finally, this study has implications for further study of multi-level factors associated with optimal training transfer outcomes. Findings of this study suggested that expanding research on in-service training specifically may yield important information regarding the "who," "what," and "how" of training to determine optimal training conditions that facilitate successful EBP adoption, adherence, and sustainment. Research continually indicates that training as usual produces sub-optimal influences on implementation outcomes (Fixsen et al., 2009; Rogers et al., 2002) . This study's findings, however, indicate that factors not typically investigated may serve as moderators of training success. For example, trainers viewed as boring and as having a cold, unapproachable disposition are likely to be associated with worse implementation outcomes than trainers perceived as engaging and having a warm, approachable disposition. Notwithstanding this implication, it is unclear whether more optimal training conditions are likely to promote better implementation outcomes beyond training-as-usual; this also is an area that is in critical need of further study (Beidas & Kendall, 2010; Boyd et al., 2017; Lyon et al., 2011) . Until researchers, organizational consultants, and EBP trainers find and utilize effective methods of training, organizations, such as schools, may continue to waste limited resources on expensive trainings that yield virtually no changes in provider work behavior.

Results from this study were obtained from 21 trainers and a subset of five teachers who participated in each of the 21 training sessions. Thus, findings are not representative of the full range of trainers and teachers included in each training. However, teachers who completed the MEAT for the purposes of this study were randomly selected within each cohort to improve representativeness. Future studies should attend to the complexity of data collection. For example, numerous trainers need to be identified who can standardize their training in structure and content to create continuity.

Additionally, some conventional wisdom would suggest the present study was underpowered for EFA (i.e., five to ten participants per item; Tinsley & Kass, 1979) . However, both the KMO measure and Bartlett's test of sphericity suggested the appropriateness of performing EFA. Further, based on the number of factors (2), number of items (25), and factor loading (average of .89 on first factor and .85 on the second factor), de Winters et al. (2009) state that the current sample size of 105 was appropriate. However, in order to execute more sophisticated evaluations (i.e., nesting trainees within trainers in hierarchical linear models; EFA followed by confirmatory factor analysis on a randomly split dataset), a larger sample size is recommended.

Further, this study did not include a measure of actual implementation behavior (i.e., observation of treatment adherence). While research has shown that behavioral intentions are a valid predictor of actual behavior under voluntary situations (Webb & Sheeran, 2006) , this is a limitation of the current study's findings. Additionally, it is unclear whether the findings from this study would hold across other types of EBPs, as this study was limited to the implementation of one type of EBP (i.e., proactive classroom management practices). Further, the EBP that was utilized in that study was limited in complexity and required few steps to implement. Thus, trainees may have had higher intentions to implement practices as they were perceived as more feasible and less complex than EBPs that require adherence to a multiple-step, multi-component process (e.g., manualized interventions). Last, this study included only one mechanism of behavior change (i.e., intentions to implement). Thus, it is unclear whether trainer attributes differentially impact other key mechanisms of behavior change (e.g., attitudes, beliefs, self-efficacy).

To further establish evidence for the psychometric properties of the MEAT in the education sector, the factor structure needs to be confirmed with a large, representative sample of teacher trainees through larger training networks and technical assistance centers. Future study of the MEAT will include adding items that may capture a potential third factor focused on perceptions of the trainer's preparation and organization. Future research should also examine multilevel factors (e.g., organization policy, leadership, consultation, provider-level motivation to change) that interact with indicators of quality training to predict successful EBP implementation. For example, training is unlikely to result in effective transfer of knowledge and skills without supervisor support, a strong implementation climate, and followup strategies that provide ongoing implementation support (Beidas & Kendall, 2010; Lyon et al., 2018) . The MEAT could also be utilized in studies focused on the effectiveness of consultation and coaching designed to improve implementation outcomes (i.e., fidelity, sustainability).

Additionally, there is ample room for additional research exploring the causal pathways or theoretical explanations that detail how and why trainer attributes impact implementation outcomes, such as uptake and sustained use of EBPs in practice. Several candidate theories could be utilized in conjunction with the MEAT in larger trials to formalize this work. First, self-determination theory (Deci & Ryan, 2012) , which focuses on the influences of social environments on attitudes, values, motivations, and behaviors may be a powerful explanatory framework to utilize when interpreting and designing studies focused on trainer attributes. For example, self-determination theory, in particular its focus on social ambiance and the cultivation of autonomous motivation, could serve as a helpful theoretical framework that further explains the effect of trainer attributes on cognitive, affective, and behavioral changes predictive of teacher uptake and use of EBPs. Similarly, other social psychological theories focused on social influence, which suggest that individuals who engender positive liking and connection are more likely to have others say "yes" to the ideas or products they endorse (e.g., Cialdini, 2001; Clark & Isen, 1982) , could be utilized to begin mapping the specific causal pathway between trainer attributes and implementation outcomes, or even help refine and solidify the MEAT's construct and nomological validity.

As such, the MEAT should be administered along with other measures capturing: (1) other potential mechanisms of behavior change (i.e., complementary measures and countermeasures explicated by candidate theories) to continue building its validity evidence; and (2) actual implementation behavior to examine its association with more robust measures of training outcomes. Research in this area should also administer additional measures of trainer characteristics to examine convergent and divergent validity of the MEAT to further evidence of its construct validity. It is also worth noting that this study took place prior to the COVID-19 pandemic. Since the onset of the COVID-19 pandemic, EBP training for school-based professionals, such as teachers, has frequently been delivered via teleconferencing platforms. Future study of the MEAT will investigate how results may vary when in-service training is delivered in a teleconferencing format.

To develop and deliver high-quality trainings that facilitate successful EBP implementation, findings from this study suggest that trainer attributes need to be factored into the implementation process. The MEAT has demonstrated evidence as a psychometrically sound measure of trainer attributes for use in examining and promoting the effectiveness of EBP training in the educational sector. There remains considerable room for future research to advance aspects of training to enhance the impact of this core implementation strategy on implementation outcomes. Future research will to attend to socially influential attributes of trainers, the structure, content, and activities of the training itself, and the broader context in which training and implementation is taking place.

The implementation leadership scale (ILS): Development of a brief measure of unit level implementation leadership

The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health

Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and

Organizational culture and climate and mental health provider attitudes toward evidencebased practice

The theory of planned behavior

Constructing a TPB questionnaire: Conceptual and methodological considerations

Information theory and an extention of the maximum likelihood principle

Using teacher greetings to increase speed to task engagement

Effects of teacher greetings on student on-task behavior

Thin slices of expressive behavior as predictors of interpersonal consequences: A metaanalysis

Social Cognition 2.0: An interactive memory systems account

Transfer of training: A review and directions for future research

Teachers as learners: Implications of adult education for professional development

Training therapists in evidencebased practice: A critical review of studies from a systems-contextual perspective

The economics of behavioral health services in medical settings: A summary of the evidence

The creation and validation of the measure of effective attributes of trainers (MEAT)

Increasing new teachers' specific praise using a within-school consultation intervention

Fostering SMART partnerships to develop an effective continuum of behavioral health services and supports in schools

Preservice teachers' classroom management training: A survey of self-reported training experiences, content coverage, and preparedness

The science of persuasion

Toward understanding the relationship between feeling states and social behavior

Precorrection: An instructional approach for managing predictable problem behaviors

Transforming school psychology in the RTI era: A guide for administrators and school psychologists

Positive greetings at the door: Evaluation of a low-cost, high-yield proactive classroom management strategy

A supportive beliefs intervention to facilitate the implementation of evidence-based practices within a multi-tiered system of supports

Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis

Services for adolescents with psychiatric disorders: 12-month data from the National Comorbidity Survey-Adolescent

Building capacity in dissemination and implementation science: A systematic review of the academic literature on teaching and training initiatives

Exploratory factor analysis with small sample sizes

Self-determination theory

Which training method works best? A randomized controlled trial comparing three methods of training clinicians in dialectical behavior therapy skills

Implementation quality: Lessons learned in the context of the Head Start REDI trial

Choice making to promote adaptive behavior for students with emotional and behavioral challenges

The fearless organization: Creating psychological safety in the workplace for learning, innovation, and growth

Assessing the organizational context for EBP implementation: The development and validity testing of the implementation climate scale (ICS)

Evaluating the use of exploratory factor analysis in psychological research

The influence of transformational leadership and leader attitudes on subordinate attitudes and implementation success

Pathways into and through mental health services for children and adolescents

Discovering statistics using SPSS

Core implementation components

Implementation research: A synthesis of the literature

Factor analysis in the development and refinement of clinical assessment instruments

Evidence-based interventions in schools: Developers' views of implementation barriers and facilitators

Quality of schoolbased prevention programs: Results from a national survey

Essentials of statistics for the behavioral sciences

Transformational leadership moderates the relationship between emotional exhaustion and turnover intention among community mental health providers

Changing physicians' behavior: What works and thoughts on getting more things to work

Sustainability of teacher implementation of school-based mental health programs

The social development model: An integrated approach to delinquency prevention

Moderators of trainee therapists' competence in cognitive therapy

Demonstrating the experimenting society model with classwide behavior management interventions

Student achievement through staff development

An index of factorial simplicity

Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care

Mental health service use in schools and non-school-based outpatient settings: Comparing predictors of service use

The first sessions of psychotherapy: A qualitative metaanalysis of alliance formation processes

Effective behavior support: A systems approach to proactive schoolwide management

Closing the gap between research and practice: The importance of practitioner training

Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship

Developing the mental health workforce: Review and application of training approaches from multiple disciplines. Administration and Policy in Mental Health and

Promoting evidence-based medicine in general practice-The impact of academic detailing

The dissemination and implementation of evidence-based psychological treatments: A review of current efforts

Dissemination and implementation of evidence-based psychological interventions

Introduction to linear regression analysis

Developing a systematic evaluation approach for training programs within a train-the-trainer model for youth cognitive behavior therapy

Preventing school failure, drug use, and delinquency among low-income children: Long-term intervention in elementary schools

Teacher classroom management practices: Effects on disruptive or aggressive student behavior

Implementation science in school mental health: Key constructs in a developing research agenda

A refined compilation of implementation strategies: Results from the expert recommendations for implementing change (ERIC) project

Effective school interventions: Evidence-based strategies for improving student outcomes

An item selection procedure to maximize scale reliability and validity

White paper: The power of realization

School-based mental health services: A research review

Organizational climate and culture

Salespeople's affect toward customers: Why should it be important for retailers?

The effects of a sportsmanship curriculum intervention on generalized positive social behavior of urban elementary school students

Evidence-based practices in classroom management: Considerations for research to practice. Education and Treatment of Children

Teaching teachers to use prompts, opportunities to respond, and specific praise

Response to intervention for behavior: Integrating academic and behavior supports

The evolution of discipline practices: School-wide positive behavior supports

The latent structure of the need satisfying properties of leisure activities

Effects of trainer expressiveness, organization, and trainee goal orientation on training outcomes

Atomic Dog

Effectiveness of training methods for delivery of evidence-based psychotherapies: A systematic review

Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation

Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence

Proactive instructional strategies for classroom management

Intention to transfer: How do autonomous professionals become motivated to use new ideas? Performance Improvement Quarterly

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations