key: cord-0726109-z2vmjhee
authors: Lasecke, Meagan; Baeza‐Hernandez, Katerina; Dosovitsky, Gilly; DeBellis, Amanda; Bettencourt, Brianna; Park, Alayna L.; Bunge, Eduardo L.
title: Disseminating online parenting resources in the community during the COVID‐19 pandemic: Lessons learned
date: 2022-01-05
journal: J Community Psychol
DOI: 10.1002/jcop.22788
sha: 6c0f0f81fd40bb901d38a8f90f44fc3dc9fc4952
doc_id: 726109
cord_uid: z2vmjhee

Online parenting programs are an effective way to teach behavioral management skills to parents in the absence of in‐person resources. This community‐engaged study aimed to examine strategies for disseminating online parenting resources in schools. Online resources were disseminated to parents in a Northern California school district. Dissemination strategies were informed by conversations with school principals, teachers, and parents and considered agent, message, and format. A total of 685 parents and teachers clicked on the online resources: 151 parents and 114 teachers attended synchronous classes. The use of dissemination strategies had a compounding influence on the number of synchronous class attendees and clicks. Emails sent by the school district yielded the greatest number of clicks, which was influenced by message content and format. A community–academic partnership (CAP) led to the dissemination of evidence‐based online parenting resources to a large population and led to lessons learned that could inform future research involving CAPs.

Compounding public health, economic, and social stressors from the COVID-19 pandemic have contributed to increased rates of mental illness, with 14% of parents reporting behavioral health concerns for their children (Imran et al., 2020) , and 48% of parents reporting that the pandemic has increased their desire to seek mental healthcare for their child (Martinelli et al., 2020) . Professional mental healthcare can be expensive, but online interventions may be a cost-effective solution for reaching large populations (Muñoz et al., 2016) . Recent studies have demonstrated that online parenting programs hold promise for increasing access to mental health services (Florean et al., 2020) , and are an effective way to teach behavioral management training skills to parents (Florean et al., 2020; Spencer et al., 2020) . Although many parents could benefit from online parenting resources, they may be unaware that such resources exist.

Increased awareness of online parenting resources can be accomplished through the dissemination of innovations, or active efforts to persuade a target audience to adopt an innovation-in this case, active efforts to persuade parents to adopt online parenting resources (Baker et al., 2021; Brownson et al., 2018; Greenhalgh et al., 2004) . Successful dissemination requires consideration of a multitude of factors, including the agent disseminating the innovation, the message being disseminated, and the format in which the innovation is disseminated.

Change agents refer to sources of information who influence a target audience to adopt an innovation, or who are disseminating the innovation (Mendel et al., 2008) . The effectiveness of change agents to influence their target audience depends on the change agent's ability to present information in ways that fit the target audience's values and worldviews, as well as their credibility with the target audience (Greenhalgh et al., 2004) . Given the instrumental role of schools in the lives of school-aged youth and their parents, school staff can serve as natural change agents for promoting online parenting resources (e.g., Baker et al., 2021) . Specifically, school staff, such as principals and teachers, are often embedded and respected within their local community and, accordingly, are well-situated to not only connect with parents but also frame information in ways that will garner parent support.

The message of innovation, or the content being disseminated, may also impact dissemination outcomes. For example, a recent study found that negative framing of email subject lines (e.g., "Threats to the health of various communities") resulted in more email opens than neutral framing (e.g., "New solutions for addressing health differences"; Long et al., 2021) . Other studies have shown that parents prefer "parent-friendly" language (e.g., describing "evidence-based treatments" as "effective treatments" or "treatments that work"; Becker & Midoun, 2016; Okamura et al., 2018) .

Additionally, the format, or how the innovation is being disseminated, may affect the extent to which the innovation is adopted. Formats may include individual correspondences, newsletters, webinars, radio, television, and social media platforms-with recommendations to use multiple formats to extend the reach of dissemination efforts (Baker et al., 2021) . In other words, some audiences may prefer newsletters, whereas other audiences may prefer social media platforms, and using multiple formats can increase the likelihood that the innovation is disseminated in a way that fits the target audience's preferences. For example, a recent study that investigated directto-consumer marketing of evidence-based treatments for substance use found that parents of adolescents with substance use problems were more likely to request additional information when presented with an infographic rather than a standard description, whereas parents of adolescents with substance use and legal problems tended to request additional information regardless of marketing format .

Community-academic partnerships (CAPs) are imperative for facilitating the successful dissemination of innovations (Drahota et al., 2016) . That is, community partners can help identify change agents who are empathetic and credible with the target audience. Community partners can also inform how to frame information to maximize its impact on the target audience. Additionally, they can provide guidance on the format(s) in which the target audience is routinely accessing information.

Schools represent the largest source of behavioral health services for youth and provide a natural avenue for disseminating online parent resources (Williams et al., 2019) . The present study partnered with a school district to disseminate an online parenting program. The goal of this community-engaged research was 1) to explore how a CAP informed efforts to promote online parenting resources in schools; 2) to identify effective strategies for disseminating an online parenting program to the target audience (i.e., parents); and 3) to share "lessons learned" about disseminating online mental health resources.

The research team partnered with a school district in Northern California. Five elementary and middle schools (prekindergarten-Grade 8) in this school district actively collaborated with the research team to inform the dissemination of the online parenting resources.

For the 2020-2021 school year, there were 5382 students enrolled across the school district, with 310 teachers, for a student-to-teacher ratio of 22:1. Fifty-eight percent of families enrolled in the school district identified as ethnic minorities, and 32% were socioeconomically disadvantaged. In terms of enrollment by gender, 49% of students identified as female, and 51% of students identified as male. Thirty-two percent of students were eligible to participate in the federal free and reduced-price meal program, and 32% of students identified as English language learners. See Table 1 Table 2 ). Active dissemination strategies included meeting with principals and hosting synchronous classes for parents (SCPs) and teachers (SCTs). Passive dissemination strategies included sending emails and e-parenting tips.

Subsequently, all materials were reviewed by authors Eduardo L. Bunge and Alayna L. Park before they were distributed.

The 

Three different email messages were tested: 1) a negative frame, 2) a positive frame, and 3) an expert frame.

Emails were sent from agents (i.e., school principals and school district) to parents, teachers, and parent-teacher associations (PTAs) that advertised the Everyday Parenting course. Negatively-framed emails 

Parenting tips from Everyday Parenting were included in weekly school newsletters. Parenting tips featured a description of a behavior-change technique (e.g., praise), an example related to COVID of how to apply each technique, and a hyperlink to Everyday Parenting. Formats of parenting tips included descriptive text, images (i.e., infographics), and videos outlining how to apply the behavior-change technique. 

Synchronous, 1-h live classes were offered to parents at the five participating schools. Various topics and tips (e.g.,

giving effective instructions, praise, attending and planned ignoring, household rules, punishment, problem-solving, and seeking professional help) were provided based on the first two modules of Everyday Parenting. Synchronous classes featured a description of the behavioral parent training technique (e.g., modeling) and then application/ practice (e.g., parents were asked to share how they could apply the technique with their children).

Synchronous live workshops were offered in English for teachers at the participating schools. The topics and tips provided were based on the requests of the principals. The workshops included stress and classroom management (e.g., praise and effective punishment) tips for the teachers. The teachers were encouraged by the research team to provide a link to Everyday Parenting to their student s' parents.

This project was determined to be nonhuman subjects research by the Palo Alto University Institutional Review Board (FWA00010885). This community-engaged project was initiated after a presentation by the research team on youth mental health disorders to the School District Board. Following the presentation, the research team met with school district administrators, who agreed to partner on efforts for disseminating online parenting resources for parents.

In September 2020, the school district sent six emails about Everyday Parenting to parents: two positive-framed emails in English; two positive-framed emails in Spanish; one expert-framed email in English; and one expert-framed email in Spanish. Each email directed parents to click a link for the online behavior management course, and the number of clicks was tracked. The efficacy of each email variation was monitored and used to inform further dissemination strategies.

In October 2020, the research team was introduced to 11 principals in the school district, six of whom contacted the research team to set up a meeting to plan future steps. Five principals agreed that: 1) the research team would introduce Everyday Parenting to the parents in a weekly meeting that principals had with parents; and 2) principals would forward emails from the research team to parents with links to Everyday Parenting; 3) synchronous classes via video conferencing would be offered by the research team to the parents; 4) principals would share eparenting tips developed by the research team via email to parents. The research team explored the efficacy of these dissemination strategies by examining click-tracking via unique URLs.

In May 2021, once the relationship was established with the principals and school communities, the research team was asked to provide SCTS. SCTs were offered in three schools on topics requested by the principals (e.g., behavioral management in the classroom and stress management for teachers). To support behavioral management in the home, teachers were given information about Everyday Parenting to share with the parents of the students in their classrooms.

3 | RESULTS 

For a timeline of the dissemination strategies used in each school see Figure 1 . As an example, the research team utilized 15 dissemination strategies (one principal meeting, one info meeting, two SCPs, four emails in total sent by principal and PTA, five e-parenting tips, one SCT, and one email sent by the teacher) in school one, which yielded 70 synchronous class attendees, and resulted in a ratio of 4.67 attendees to strategy. When all the strategies utilized in each school were combined, it was observed that the school that resulted in the highest number of attendees was School 3 (136, ratio of attendees to strategies 7.56), followed by School 1 (70, ratio of attendees to strategies 5.84),

and School 4 (46, ratio of attendees to strategies 3.54). SCPs were offered in four schools. The school with the highest attendance rates was School 1 (n = 50), followed by School 4 (n = 46). The researchers offered SCTs in three schools. School 3 participated in a classroom behavior management and stress management for teachers workshop with 93 attendees, and School 1 and School 2 participated in the stress management workshop with a total of 20 and 1 attendees, respectively.

When all the strategies utilized in each school were combined, it was observed that the school that resulted in the highest number of clicks was School 1 (144), followed by School 3 (107) and then School 2 (101; see Table 3 ). When the average number of clicks per school population was analyzed, a similar pattern was observed with School 1 having the highest average (40%) followed by School 2 (34%), School 3 (18%), and School 4 (18%; see Table 3 ).

F I G U R E 1 Strategies timeline and number of attendees per school. Number of attendees represents the total number of participants across meetings, not unique participants. The ratio is estimated by dividing the number of attendees by strategies. Emails include those sent by the principals and parent-teacher associations. Synch, synchronous.

T 17.65% 0% 6.63%

Analyses are based on researchers' efforts and the outcomes of these efforts but researchers do not have confirmation of how many times the school district, principals, or PTAs sent out the information. Abbreviation: PTA, parent-teacher association.

When comparing the different dissemination strategies, the impact of the different change agents were analyzed.

Regarding emails promoting Everyday Parenting, the ratio of number of clicks to number of email campaigns was 258:4 (mean clicks = 64.5; SD = 28.94) for the school district; 105:14 (mean clicks = 7.5; SD = 6.95) for the principals;

26:4 (mean clicks = 6.5; SD = 11.7) for the PTAs; and 55:4 (mean clicks = 13.75; SD = 18.6) for teachers.

The average number of clicks was used to compare the efficacy of three message types (i.e., positive, negative, and expert) sent by the school district. The percentage of clicks per recipient across all three types was 1.72% (n = 258/ 14978). The positive frame email had a higher percentage of clicks per recipient at 2% (n = 145/7232), followed by the negative frame at 1.70% (n = 90/5321) and the expert frame at 0.95% (n = 23/2425).

A series of e-parenting tips were sent with a ratio of the number of clicks and number of e-tips sent across four schools at 134:17 (mean clicks = 7.9, SD = 8.14). Four schools received e-parenting tips on praise, modeling, and the positive opposite. The total number of clicks for the e-parenting tips was 21 for praise, 32 for modeling, and 43 for the positive opposite. Three schools received the antecedents e-parenting tip, which resulted in 21 clicks (see Table 4 ). Two schools received the shaping e-parenting tip, which resulted in 17 clicks. The number of clicks per strategy depended on the number of e-parenting tips received and the number of schools the e-parenting tips were distributed to. The e-parenting tip that had the highest number of clicks was the positive opposite which included text, image, and video.

The current study involved a CAP between the research team and a school district in northern California to disseminate online parenting resources in schools during the COVID-19 pandemic. As proposed by Baker et al.

(2021), the dissemination strategies are intended to communicate information about parenting strategies to parents and teachers. These strategies considered the schools' and parental needs during the pandemic (e.g., provided asynchronous online resources and evening SCPs). Provided resources considered the characteristics of the target audience (e.g., synchronous meetings were delivered in English and Spanish and by Latinx facilitators) and utilized different messages (e.g., positive-frame, negative-frame, expert-frame) and communication formats (e.g., emails, newsletters, and live presentations).

The process started in the fall of 2020, in which more than half of principals in the school district responded to an introductory email about partnering with the research team to offer online parenting resources. Five principals ultimately partnered with the research team on this endeavor, with three schools requesting additional mental health resources (i.e., SCTs on stress and classroom management). The five principals who did not reply to the introductory email were not contacted again. This decision was made so that the research team could focus the available time, efforts, and resources on the schools that expressed interest. It is possible that a follow-up and personalized email to those principals who did not initially respond would have led to further collaborations.

However, it is notable that the majority of principals from a local school district were interested in partnering with the research team during a global pandemic that created unprecedented challenges for schools. Such collaboration resulted in several efforts over the following months that ultimately resulted in evidence-based online parenting resources reaching a large population. Overall, a total of 685 parents and teachers clicked on the Everyday Parenting course, 151 parents attended synchronous parenting meetings, and 114 teachers attended synchronous classes on behavior management in the classroom and stress management. This CAP led to a set of lessons learned.

The first lesson learned was that building strong CAPs with motivated stakeholders can lead to successful outcomes (see Table 5 ). Partnering with school principals provided an opportunity for researchers to understand how to best support schools, particularly during the constantly evolving COVID-19 pandemic. For example, in the first meeting with one of the principals, they suggested starting with a presentation to parents about the online resources and then offering synchronous classes. This strategy was later discussed with the other principals who then requested synchronous parenting classes for their schools. Stakeholder engagement varied across and within schools, likely due to a variety of factors; however, stakeholder engagement appeared to be strongest among change agents (i.e., schools, principals, and teachers) that had pre-existing relationships with the research team.

Researchers aiming to partner with community stakeholders should prioritize building strong CAPs.

Findings also showed that the schools that received the most dissemination strategies were the schools with the highest number of attendees and the greatest ratio of attendees by strategies. These findings suggest a positive association between the number of dissemination strategies and reach (i.e., Lesson Learned #2) which supports findings in the literature suggesting that the more dissemination strategies used, the more likely that one may fit the stakeholder (i.e., innovation-system fit; Baker et al., 2021; Greenhalgh et al., 2004) . As such, researchers should consider reaching stakeholders through various messages and formats, as different stakeholders may have different preferences. For instance, in this study, one stakeholder may prefer the positive message and another may prefer the negative message; or one stakeholder may prefer the email and another may prefer the synchronous class. Lastly, in relation to the positive association between the number of dissemination strategies and reach, future researchers may want to explore at what point there might be diminishing returns. From the present study, the synchronous classes required a considerable amount of the researchers' efforts and reached fewer individuals, but these synchronous classes may have benefited the participants more than those that just clicked on a link to the course.

Additionally, results showed a positive association between the use of dissemination strategies and the number of clicks for the Everyday Parenting course. It is possible that the use of various dissemination strategies contributed to greater interest in online parenting resources and/or that some schools had greater parent involvement, which contributed to both stakeholder engagement and interest in online parenting resources. Researchers disseminating interventions should assess and, if possible, contribute to the culture (e.g., perceived organizational support and job satisfaction) and climate (e.g., shared norms and values) of the partnering organization, as these factors have been shown to influence successful implementation, openness to organizational change, receptivity to the intervention, and intervention sustainment (Lesson Learned #3; Aarons et al., 2015 Aarons et al., , 2017 Aarons & Sawitzky, 2006; Mendel et al., 2008; Novins et al., 2013; Williams et al., 2019) .

When considering the sustainability of dissemination strategies, two factors that should be considered are the role of user-centered design (UCD) and the dynamic and developmental nature of systems. UCD is an approach to intervention development that is grounded in integrating information about the users into the development of the intervention itself (Lyon & Koerner, 2016) . This study included synchronous classes, e-parenting tips, and emails The more the merrier Overall, reaching large communities require using dissemination strategies with various messages, formats, and agents to best maximize dissemination effort efficiency 3

Understand the organizational culture Work to understand the overall social context and organizational culture of the school community to maximize the fit between the innovation and the context in which it is being disseminated 4 Consider effectiveness, fit, and sustainability

Examine both intervention outcomes (e.g., effectiveness) and dissemination outcomes (e.g., acceptability, appropriateness, feasibility, sustainability) for each individual school may allow for the greatest personalization, one large synchronous class for the school district may be a more sustainable option that balances the needs of the community and the resources of the researchers. When disseminating interventions, it is important to consider not only intervention effectiveness but also implementation outcomes including intervention fit and sustainability (Proctor et al., 2011; Lesson Learned #4) .

Findings showed that the change agent (i.e., district, principals, PTAs, and teachers) influenced the level of parental involvement with the online parenting resources. More specifically, emails sent by the school district yielded the greatest number of clicks, but a relatively small percentage of email recipients clicked on the Everyday Parenting link; whereas emails sent by teachers yielded a relatively low number of clicks, but a large percentage of the email recipients clicked on the link. These results suggest that different stakeholders and change agents played differently, yet important functions in these dissemination efforts (Baker et al., 2021; Kreuter & Wang, 2015) . For example, while teachers have a strong influence on parents, it may be difficult for teachers to participate in the dissemination of mental health resources given their high number of competing demands. Thus, using multiple change agents is helpful and the effectiveness of change agents is largely based on the agent's relationship with the target audience, and stakeholders may have different relationships with principals, teachers, PTAs, and the school district's office. When selecting stakeholders, it is important to consider the objective. If the objective is to reach as many people as possible, then the school district or principals could be a good channel. If the objective is to have people engage with the innovation, then teachers may be the more effective agent. The main lesson learned from this analysis is that different agents have different impacts (Lesson Learned #2; Mendel et al., 2008; Novins et al., 2013) , particularly in school-based contexts.

Furthermore, although emails sent by the school district or principals may have resulted in a large number of clicks, they may not have reached parents who are not familiar with digital means. Access to stakeholder groups, availability of stakeholder groups, organizational leadership, and openness to innovation were facilitators within our CAP, but time, personnel, and budgetary resources were dissemination barriers. Each dissemination strategy has barriers and facilitators, and researchers need to consider these determinants to successfully and sustainably disseminate their products (Baker et al., 2021) . Furthermore, research shows that different contextual factors (i.e., organizational structure, resources, policies, networks, connections, change agents, norms, and attitudes) influence the ability and willingness of agents to implement and sustain new interventions (Brownson et al., 2018; Mendel et al., 2008) . Future researchers should consider identifying key change agents when disseminating innovations and explore the influence of contextual factors in school settings.

Results from the study also showed that parents' clicks to Everyday Parenting were influenced by message content and format. The positively-framed email (i.e., Making Life with Kids EASIER!) had the highest percentage of clicks per recipient, followed by the negatively-framed email (i.e., Parenting problems during a pandemic) and then theexpert framed email. While this is not enough data to conclude that positively-framed messages should be used, it There were two additional lessons learned throughout the CAP that were based on the process, rather than specific outcomes. One was that data management in a community-based participatory research project is less structured and more challenging than in regular experimental studies. Thus, data management of a project that is, not exclusively under the control of the research team will have limitations and require careful organization. It is important to have a systematic way of organizing and collecting data. Researchers should consider having a designated person assigned to this task who is able to regularly track and monitor data collection (Lesson Learned #5). Additionally, CAP projects should utilize ongoing evaluation to facilitate the goal of continuous quality improvement which involves having defined aims and measures to attain improvement, adapting interventions to fit the needs of the target recipients, and using an "iterative development and testing process" (Lesson Learned #6; Rubenstein et al., 2014) . Ongoing evaluation has numerous benefits including ensuring that the needs of the community are met, promoting positive outcomes, and informing improvements to CAP processes and procedures.

The present study had several limitations. First, researchers initially planned to track enrollment, in addition to clicks, for

Everyday Parenting with the help of a third party but, due to staffing changes, the third party was no longer able to provide the research team with enrollment data. The ability to track enrollment would have been a more direct and quantifiable way of measuring parental engagement in comparison to solely website visits (i.e., clicks); however, clicks still served as a quantifiable way to measure the effectiveness of dissemination strategies. Second, although the researchers made efforts to have systematic and consistent ways of tracking and organizing data, community-engaged research presents challenges to data management. Consequently, it was sometimes difficult to confirm the number of attendees for synchronous classes and track emails and clicks. It is common in community-engaged research to encounter these difficulties and thus, it is important to have streamlined procedures (e.g., swimlane procedures) and clearly defined tasks for each individual that outlines their role in the project (Hattab and Farook 2013; Jeyaraj & Sauter, 2014) .

Third, when comparing dissemination strategies used, the numbers were so small that no statistical analyses were run, thus limiting the possibility of making relevant statistical inferences. To note, these are initial findings that could be considered as preliminary evidence. Lastly, being that the context of the study took place during the pandemic, content in the synchronous classes was adapted from the original Everyday Parenting course to make it most relevant to the parents (e.g., examples of how to give effective instructions "Please go back to your [virtual] class") and teachers (e.g., strategies for managing behavioral issues in the classroom). Thus, researchers prioritized external validity over internal validity which is a common limitation in community-engaged work.

The current study shows that a CAP between a research team and a school district to disseminate online parenting resources in schools led to hundreds of parents receiving information about evidence-based behavior change techniques. Researchers aiming to implement CAPs with schools can benefit from the many lessons learned in this project such as building solid relationships and utilizing as many dissemination strategies and channels as possible.

The COVID-19 pandemic has shown the potential of utilizing online resources and highlighted that finding effective ways of delivering online parenting resources to school communities could contribute to reducing parental stress and improving children's wellbeing.

Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation

Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: A cluster randomized trial study protocol

Organizational culture and climate and mental health provider attitudes toward evidence-based practice

Dissemination science in school mental health: A framework for future research

Moderators of parent responses to direct-to-consumer marketing materials: A randomized trial

User-informed marketing versus standard description to drive demand for evidence-based therapy: A randomized controlled trial

Effects of direct-to-consumer advertising on patient prescription requests and physician prescribing: A systematic review of psychiatry-relevant studies

Getting the word out: New approaches for disseminating public health science

Structuring the collaboration of science and service in pursuit of a shared vision

Community-academic partnerships: A systematic review of the state of the literature and recommendations for future research

The efficacy of internet-based parenting programs for children and adolescents with behavior problems: A meta-analysis of randomized clinical trials

Diffusion of innovations in service organizations: Systematic review and recommendations

Information flow comparison between traditional and BIM-based projects in the design phase

Mental health considerations for children & adolescents in COVID-19 pandemic

Validation of business process models using swimlane diagrams

From evidence to impact: Recommendations for a dissemination support system: From evidence to impact. New Directions for Child and Adolescent Development

Rapid-cycle experimentation with state and federal policymakers for optimizing the reach of racial equity research

User-centered design for psychosocial intervention development and implementation

2020 Children's mental health report: Telehealth in an increasingly virtual world

Interventions in organizational and community context: A framework for building evidence on dissemination and implementation in health services research. Administration and Policy in Mental Health and Mental Health Services Research

Massive open online interventions: A novel model for delivering behavioral-health services worldwide

Dissemination and implementation of evidence-based practices for child and adolescent mental health: A systematic review

Insights in public health

Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research

How can we recognize continuous quality improvement?

Do online parenting programs create change?: A meta-analysis

Increasing the appeal of CBT: Latent profile analysis of reaction to different ways of describing CBT. 54th Annual Convention Virtual Convention

Organizational culture and climate profiles: Relationships with fidelity to three evidence-based practices for autism in elementary schools

Disseminating online parenting resources in the community during the COVID-19 pandemic: Lessons learned

The authors would like to express gratitude for the following individuals that put time and effort into helping this project come together: Shirin Aghakhani, Francesca Pedraza, Jessica Lin, Alyssa Herman, and Jenna Kim. They LASECKE ET AL.