key: cord-0726643-gdree9xx authors: Armstrong, Anna Torrens; Noble, Charlotte A.; Azeredo, Juliana; Daley, Ellen; Wilson, Roneé E.; Vamos, Cheryl title: An Overview of an Undergraduate Diversity MCH Pipeline Training Program: USF’s Train-A-Bull date: 2022-01-03 journal: Matern Child Health J DOI: 10.1007/s10995-021-03332-y sha: cf182d3ac698bceb4561870f6e2c6caeacf19724 doc_id: 726643 cord_uid: gdree9xx PURPOSE: To describe an undergraduate pipeline training program (PTP) designed to guide underrepresented minorities (URM) trainees into MCH-related health professions, ultimately contributing to a diverse maternal and child health (MCH) workforce that can improve health outcomes for all women/mothers, children, and their families, including fathers and children with special healthcare needs. DESCRIPTION: Three cohorts with 35 total undergraduate trainees were recruited to participated in the 2 years USF MCH PTP program where they were mentored, trained, guided, and supported by program faculty/staff. Students were recruited early in their education track, and the program was individually tailored based on trainees’ educational discovery stages. Key program components included seminars, summer institutes, public health courses, mentorship, internship, experiential learning opportunities, and professional networking opportunities. ASSESSMENT: The majority of the undergraduate participants were diverse URMs including Hispanic/Latino (37.1%), Black/African American (31.4%), Asian (20%), and American Indian/Alaskan Native (5.7%) trainees. Out of all the cohorts, 51.4% were first-generation college students and 74.3% had economic hardships (i.e., PELL Grant, FAFSA). Resulting from the program, all cohorts increased in educational discovery stages, one-third enrolled in health-related graduate studies and half joined the MCH workforce. CONCLUSION: Recruitment in pipeline programs should be intentional and meet students where they are in their education discovery stage. The use of educational discovery stages within a pipeline program are useful in both tailoring curriculum to individuals’ needs and assessment of progression in career decision-making. Mentoring from program staff remains an important component for pipeline programs. The dissemination of the outcomes from MCH diversity pipeline training programs contribute to the knowledge and evidence base of best practices for future programs to consider. Educational pipelines in MCH are acknowledged as an essential part of building a diverse future MCH workforce, but little is known about the various programs that have been implemented. This article highlights the use of the educational stages of discovery within an MCH diversity pipeline program as both a method of assessment and tailoring of program components. With the challenges of the evolving healthcare system and the urgent need to address health disparities and inequalities through a social determinants of health lens in Florida and the nation, we must ensure availability of a competent, passionate, and reflective MCH workforce to improve health outcomes for MCH populations. Highquality MCH training programs that recruit, mentor, and engage the future MCH workforce are critical to achieving this goal. This future workforce must reflect members of historically underrepresented minorities, in the quest of reducing health disparities in MCH. An AHRQ report (2018) reinforces the need for a diverse workforce to provide high-quality, culturally and linguistically responsive care to MCH populations. Diversity among healthcare providers improves patient access, satisfaction, and quality of care (Moy & Freeman, 2014) . With health workforce shortages, increased attention on recruiting and training a diverse workforce is imperative (Duffus et al., 2014) . The U.S. workforce among 30 health occupations consists of a white majority (64.4%), followed by: Hispanics (16.1%); Black/African Americans (11.6%); Asians (5.3%); American Indians/Alaskan Natives; (0.6%) and Native Hawaiians/Other Pacific Islanders (0.2%) (HRSA, 2015) . Among the 47,000 public health professionals who participated in the public health workforce interests and needs survey, few are Black/African American (16%), Hispanic/Latino (14%), or Asian (5%) (de Beaumont Foundation, 2017) . Florida also lacks a diverse workforce and faces widening cultural gaps, as Hispanics and African Americans comprise almost 25% and 17% of the population, but only 17.6 and 5.5% of licensed physicians, respectively (AHRQ, 2018; Desantis & Rivkees, 2020; Florida Department of Health, 2016; U.S. Census Bureau, 2016) . Pipeline programs have shown to increase diversity across healthcare; thus, the dissemination of program strategies and outcomes are important (Kuo et al., 2015) . Given that MCH pipelines are developed based on overarching proposal guidelines, and result in unique, contextualized approaches, there is added value in disseminating program overviews. Additionally, there are very few publications providing such description (Kuo et al., 2015) . By sharing these programs, we can move towards a common model of the critical elements that should be included in all MCH pipeline programs of the future (Petersen, 2019) . Our program presents a student-centered approach that attempts to lower barriers for a population of students that have multiple responsibilities, allowing the program to meet their needs to maximize their level of engagement. This paper describes a pipeline training program (PTP) designed to guide URM trainees into MCH-related health professions. We also include a brief overview of the methods used to evaluate the program emphasizing the outputs, initial and intermediate outcomes based on the program logic model (Table 1) , along with challenges and lessons learned. To address the need for a highly skilled and diverse MCH workforce, the University of South Florida (USF) MCH PTP aimed to recruit, train, mentor, and provide enriching experiences with the goal of guiding URM undergraduate trainees into MCH-related health professions to improve levels of representation, reduce health disparities, and increase access to health care for vulnerable, underserved populations. This PTP had four overarching goals: (1) recruit and support cohorts of undergraduate students from economically and educationally disadvantaged and racially or ethnically diverse backgrounds into a high-quality training program that prepares them for a successful trajectory into MCH graduate training and professions; (2) implement an innovative training program through didactic, experiential, mentoring and peer exchange modalities that expose students to MCH leadership, broad public health perspectives, interdisciplinary training and practice, and cultural and linguistic competencies; (3) collaborate broadly with MCH Long-Term Training Programs, HRSA and Title V stakeholders and other MCH community members; and (4) develop and disseminate innovative curricula and research to advance MCH training and practice. Program components were developed within the framework of the stages of educational discovery to maximize program success (Arnold et al., 2015) . These stages include: (1) decider: expresses interest in health science; exploring degree options; (2) explorer: declared health science; exploring double majors/minors and discussing links with coursework and other opportunities; (3) maximizer: in middle of program; seeking practice opportunities; and (4) mover: approaching graduation; preparing for post-graduate training or workforce (Arnold et al., 2015) . Trainee stage data was assessed throughout the program for evaluation purposes. Program staff formed strong connections with trainees and engagement continues post-program, supporting ongoing insights into their educational and professional trajectories. We recruited three cohorts of 12 trainees into the MCH PTP. Support for trainees included stipends (provided after completing summer institutes) and tuition waivers for required coursework. A dynamic, student-centered (i.e., focused on student needs and involvement rather than program staff expectations) plan was developed using a combination of informal and formal recruitment avenues. The two-tiered approach included: (1) recruitment from within identified feeder programs (e.g., student support services, program advisors, one-on-one recruitment at events, program announcement to instructors and program administrators); and (2) recruitment from additional informal and formal methods By the end of the program: -At least 80% of the trainees who graduate from the program will be from underrepresented racial groups. -At the end of the project period, at least 90% of trainees will be from underrepresented racial groups. -At least 80% of the trainees will report an increase in knowledge in MCH public health pracƟce, research and leadership. -At least 80% of the trainees will report an increase selfefficacy, skills, and abiliƟes to apply to graduate schools and jobs, and work in MCH-related fields. -All trainees will have exposure to at least one MCH long-term training program. -At least 80% of Pipeline graduates will apply for admission into MCH-related graduate programs. -At the end of the project, at least 85% of Pipeline graduates will enter into MCH-related graduate programs. -At least 80% of MCH Pipeline graduates will work in MCH professions after completion of graduate school. Training/mentoring in an interdisciplinary setting, collaboration with diverse MCH partners, and linkages to MCH programs will result in effective MCH graduate school preparation. -Increase diversity in MCH workforce -Improve health and reduce healthcare disparities TARGET POPULATION: Undergraduate students with economically or educationally disadvantaged and ethnically/racially diverse backgrounds in Florida and strategies (e.g., online university platforms, social media, career fairs, meet and greets). Recruitment strategies were implemented during Spring semester and focused on undergraduate students in any major in the second half of their freshmen (first) year and/ or beginning of their sophomore (second) year. Trainees participated in the program for two years starting in the Spring semester and completed the program in the first half of their fourth year. Completing the program in their fourth year was optimal in the context of the educational discovery stages as it allowed mentorship and guidance in the graduate school application process. Eligibility criteria included: current undergraduate enrollment (full-time) at USF; U.S. citizenship or permanent resident visa; demonstrate economically or educationally disadvantaged status (e.g., financial aid eligible); first generation student; family hardship (e.g., foster family); descendants of immigrant or migrant family or multi-language proficiency; self-identify as URM; and a demonstrated interest in MCH and/or pursuing graduate training. The USF MCH PTP curriculum had six components: (1) seminars; (2) summer institutes; (3) public health courses; (4) mentorship; (5) internship; and (6) professional networking opportunities (see Table 2 ). The first year of the program emphasized MCH knowledge through coursework, seminars and the summer institute. During the second year, seminar and summer institute topics focused on graduate school preparation and practical application through the development of a research proposal poster. This scaffolded approach allowed for distributed practice of content and refinement of program components based on trainee educational discovery stage. The program commenced with monthly seminars to foster key comprehension and application of relevant MCH topics. The seminars, hosted and facilitated by program faculty and staff, included guest lectures or discussions from other diverse health professional faculty and MCH community leaders and professionals (e.g., Florida Perinatal Quality Collaborative, Florida Covering Kids and Families, USF Student Affairs, USF Writing Studio). Some seminars were also co-hosted with USF's Center of Excellence in MCH encouraging a continuous trajectory of learning and mentoring. Example seminar topics included: program orientation; MCH 101 part 1 and part 2; community-based research in infant and child health; quality improvement in MCH; graduate/doctoral program student panel; professionalism; social networking; systems approach; MCH advocacy and policy; emotional intelligence; and communication/health literacy. Trainees participated in two virtual, self-guided institutes delivered as modules (five per summer) via the Canvas Learning Management System. The first institute introduced trainees to the field of MCH, research skills, and internship opportunities. The topics by module included: Research 101 Part 1; research 101 Part 2; research ethics; interdisciplinary practice; and conflict resolution and negotiation. The first summer institute included activities such as discussion boards, CITI training, and research activities. A more detailed description is provided later. The second institute expanded trainees' professional development skills with an emphasis on graduate school programs and applications. Module topics included: family-centered care; cultural and linguistic competency; resumes, CVs, and finding your dream job; and personal statements, cover letters, and finding the right graduate school. The second summer institute included case studies with open discussion boards, resume, personal statement and cover letter peer review sessions, and guided activities on conducting research to find dream jobs and graduate programs. All modules contained pre and post-tests to assess changes in knowledge. Trainees were required to take two undergraduate public health courses during the program, Foundations of Maternal and Child Health and Introduction to Health Disparities and Social Determinants of Health. The first course, Foundations of MCH, served as an introductory course, providing an overview of MCH issues and trends. The objectives of this course are organized around the knowledge of health assessment and interventions for families and children. The second course, Introduction to Health Disparities, uses a social ecological framework to provide a broad overview of health disparities in the U.S. and multi-level factors influencing those disparities. It also examines social and cultural determinants of health including race/ethnicity, geography, socioeconomic position, gender, sexual orientation, disability status, migration status, age, religion and spirituality. If a trainee had already taken one of these courses, tuition was offered for another public health elective from a comprehensive list including, but not limited to: Women's Health; Nutrition and Disease; or Public Health Education Theory and Behavior. For flexibility, courses could be taken at any time in the program. This MCH PTP assumed that training and mentoring in an interdisciplinary setting, collaboration with diverse MCH partners, and linkages to MCH programs will result in effective MCH graduate school preparation. The stakeholders (e.g., project director, project coordinator, MCH faculty, community partners) and other inputs (e.g., stipends, tuition) led to the development of a series of program components and an evaluation plan to meet initial and intermediate outcomes (Table 1) . Trainees' demographic data were collected during the application and acceptance process. Pre-and post-surveys were administered during seminars and summer institute 1 to identify changes in MCH knowledge and MCH research knowledge. Trainees completed pre-and post-self-efficacy surveys at the start and end of the program. This survey consisted of 15 questions (five point-Likert scale, Strongly Agree-Strongly Disagree) totaling a possible score of 75. The questions focused on long-term career goals, MCH employment/research/funding opportunities, and the graduate school application process. Program staff formed interpersonal relationships with trainees throughout their time in the program, facilitating the collection of longterm data (i.e., admission into graduate programs or workforce). Ongoing follow-up is conducted via email every 6 months to update trainee information and relevant longterm outcomes such as graduate school application, admission or work in MCH. Although trainee educational discovery stage progression was not listed as an outcome in the logic model, it was used in designing program components. We collected this data at three points in the program: start, mid-point, and end. Trainees learned about educational discovery stages at program orientation and completed their first selfassessment. During one-one-one meetings with program staff, trainees re-assessed their stage. The data presented in this manuscript were collected as part of program evaluation and was considered exempt from IRB review and does not include clinical or patient data. By Fall 2020, 35 trainees have completed the MCH PTP. Program staff communicated with trainees during and after the program to collect data (e.g., student/employment status, e-mail address, five to 10-year goals, social media). One trainee was asked to leave the program for not fulfilling agreed upon responsibilities. Trainees identified as mostly female (91%; 9% male). A few trainees (11.4%) graduated earlier than planned and thus exited the program early. Five trainees were accepted into Title V internships. Positive results across the program presented elsewhere included increases in MCH knowledge, MCH research knowledge and self-efficacy. Recruitment of participants was consistent with the diversity goals of this program. Trainees were asked to self-identify race/ethnicity and first-generation status or economic hardship or both (e.g., PELL Grant) ( Table 3) . Paired samples t-tests were used to assess changes in preand post-mean scores for changes in MCH knowledge and research knowledge within each cohort, while ANOVAs were used to compare across cohorts. MCH knowledge increased within each cohort across all topics: MCH 101 part 1 (topics included foundations of MCH, overview of MCH populations and indicators, health disparities); MCH 101 part 2 (topics included overview of MCH theoretical frameworks, social determinants of health, socioecological model, lifecourse and systems theory); health literacy (topics included significance, at-risk populations, role in policy, research and practice); and quality improvement (topics included role in improving outcomes, Plan-Do-Study-Act cycle, implementation barriers and success) ( Table 4) . While the differences observed were significant (p < 0.05), this must be interpreted carefully due to the small sample size. There were no significant differences observed across cohorts. MCH research knowledge increased within each cohort across all topics: Research 101 part 1 (topics included purpose, process, approaches, funding for research); Research 101 part 2 (topics included qualitative and quantitative methods, data collection and analysis, literature reviews); research ethics (topics included history of human subjects research, principles in Nuremberg code and Belmont report, CITI training); interdisciplinary practice (topics included definitions and benefits/barriers to disciplinary, multidisciplinary and interdisciplinary work, elements of effective interdisciplinary teams); and conflict resolution (define sources and types of conflict, strategies to resolve conflict, skills for successful negotiation) (Table 4 ). Again, we observed significant differences in MCH research knowledge across all MCH topics covered (p < 0.05; with one exception of cohort 3 showing not significant for conflict resolution). There were no significant differences observed across cohorts. Findings from the pre-and post-self-efficacy surveys suggest the MCH PTP may have had an impact on trainee selfefficacy in applying to health-related graduate studies and positions in the MCH workforce. Significant differences in self-efficacy (p < 0.05) were observed across all cohorts. Survey questions showing the highest average increase preand post-focused specifically on self-efficacy as it applies to identifying funding or scholarship opportunities for graduate school and research activities supporting graduate school application. An expected outcome of the MCH PTP was for trainees to enroll in a health or MCH-related graduate program or enter the MCH workforce directly. The curriculum supported the graduate school application process and program staff encouraged trainees to request recommendations for their applications. Program faculty provided a total of 31 recommendations for trainees for graduate school, employment, internships, fellowships, and awards. Currently, 40% of all trainees have been admitted to graduate or professional schools, with most in a health-related area (see Table 5 ). The last cohort completed the program in 2020, therefore follow-up will continue. Approximately half of trainees are in MCH-related workforce positions (Table 5 ). The remaining trainees are in various stages of completing their undergraduate degrees or applying to graduate programs. A positive average increase in educational discovery stage progression was shown among all three cohorts (Table 6) . A change of one or greater indicates a transition to the next stage (e.g., decider to explorer; maximizer to mover). Some trainees fell between two discovery stages. While this outcome demonstrates trainees transitioned to subsequent stages, several limitations should be taken into consideration including sample size, and threats to internal validity such as history and maturation. A student-centered recruitment plan led to a group of diverse undergraduate trainees with an interest in the field of MCH. The comprehensive, scaffolded program using complimentary curriculum components exposed trainees to relevant MCH knowledge, content and career opportunities. Mentorship on various levels paired with support and training in the graduate school application process resulted in success among trainees in their application and admission to a variety of health-related and MCH graduate programs. Engaging in the development of a research proposal and the experience of presenting research provided the opportunity for application of content, and a competitive advantage to graduate school applications. The use of educational discovery stages is an effective approach to raise awareness and build interest in MCH careers among URM students. Repeated assessment of trainee educational discovery stage allowed program staff to refine program components to meet trainee needs, such as, identifying relevant and appropriate internship experiences. For example, several students who presented as "maximizer" (e.g., seeking practice opportunities) were encouraged to apply for the Title V internship program. We encountered multiple challenges throughout program implementation. As a new initiative, raising awareness of a program was labor intensive; however, recruitment self-perpetuated after year one. Trainees reported a perceived overlap in content between the didactic program components. To prevent this redundancy, minor adaptations were made (e.g., excluded topics from seminars and summer institutes). Retention issues arose primarily due to unanticipated early graduation (n = 5) based on a university-system policy related to excess credit hours, with only minimal program dropout (n = 1). Several strategies were added to address program retention, including a memorandum of understanding, ongoing mentorship, and frequent touch points. Additionally, trainee schedules were assessed each semester to address needs and identify barriers to participation. Maintaining contact with trainees' post-program is also challenging. Establishing meaningful relationships with trainees during the program and maintaining engagement post-program are key strategies for follow-up. Delivering the program under the evolving circumstances of the COVID-19 pandemic required program staff to quickly make necessary modifications to ensure trainees received the same quality program. Given part of the program was online via Canvas, trainees were already accustomed to some online program engagement. Due to the nature of grant-funding, a sustainability plan helps ensure continuity. A five-year grant cycle provides time for initial successes that may earn university recognition and garner support. Program modifications will likely need to be made, but sustaining key program elements is critical to build a diverse MCH workforce. Program staff used multiple methods to collect trainee feedback to identify strengths and areas of improvement including pre-and post-surveys, focus groups, and exit interviews. An emergent theme across cohorts was the presence of a consistent program coordinator who trainees identified as a mentor. Trainees reported benefiting from a program coordinator offering support for URM and first-generation students, providing a safe space for open discussion. These benefits, noted by program staff, led to an open-door policy for trainees. Future pipelines should consider this policy in their staffing plan, as it created a sense of connectedness among trainees. Relatedly, convening a culturally congruent mentoring match with graduate students provides meaningful relationships, facilitates student success and increases the likelihood of trainee engagement and commitment to MCH careers (Wyatt & Belcher, 2019) . Program staff identified specific supports for graduate student mentors. The addition of a mentor orientation paired with a small stipend resulted in positive mentor-trainee satisfaction based on the results of a post-survey for mentees. The post-mentorship evaluation included questions on the description and function of the relationship; mentorship quality; mentorship components (i.e., academic and career development, research component); and overall satisfaction. Given that this was only a post-survey, care interpreting these results must be taken. Pipeline programs should also encourage various forms of engagement within the MCH field (e.g., Title V internships, community-engaged learning) as it has been shown to increase confidence in future career decisions (Healthy People 2030, 2020; Kolb & Fry, 1975) . According to trainees, participation in these types of activities solidified their understanding of MCH. Continued use of the educational discovery stages for assessment and tailoring will build the evidence to support its use in pipeline programs. Despite trainee success in a program, limitations still persist. The mentorship component was originally intended to be a triad relationship between trainee, MCH faculty, and graduate student mentor. Challenges with scheduling arose leading to adaptations to this component. Graduate students were matched as mentors to support trainees in developing their research proposal poster and program staff provided ongoing mentorship. Although a faculty member was not assigned to all trainees, they were encouraged to connect with those faculty with aligning research interests. Given the recent completion of this program, long-term outcomes are not available. Outcomes are expected to change over time based on the continued trajectory of trainees' education and career paths. Existing health disparities and a lack of diversity in the MCH workforce continues to drive the need for a pipeline of well-trained and reflective future MCH professionals. A diverse MCH workforce supports patient access and satisfaction in addition to quality of care. Continued funding for pipeline programs is important and should look to support the dissemination of program results to improve overall outcomes and strengthen future efforts while building the evidence base. For example, dissemination should include the ascertainment of critical pipeline components that lead to success, such as mentoring and tailoring to student educational discovery stages. Our approach to a layered, dynamic curriculum that maintained a focus on trainee educational discovery stage helped guide student success in graduate school admission. Trainees moved through the educational discovery stages within our program, demonstrating our curriculum was successful in supporting their journey while also providing a metric for change. National healthcare quality and disparities report Advising undergraduate public health students: A phased approach Public health workforce interests and needs survey Physician workforce annual report Training racial and ethnic minority students for careers in public health sciences FloridaCHARTS: Infant deaths per 1000 live births 3-year rolling rates Expand public health pipeline programs that include service or experiential learning-PHI-R02 Towards an applied theory of experiential learning Creating an MCH pipeline for disadvantaged undergraduate students Federal investments to eliminate racial/ethnic health-care disparities Channeling our legacy into our future: The importance of the MCH pipeline training program Sex, race, and ethnic diversity of US health occupations Establishing the foundation: Culturally congruent mentoring for research scholars and faculty from underrepresented populations Acknowledgements The views expressed in this article are those of the authors and do not necessarily reflect the official policies of HHS or HRSA, nor does mention of the department or agency names imply endorsement by the U.S. Government. Author Contributions Not applicable.Funding The activities described in this article were funded by the U.S. Department of Health and Human Services (HHS), Health Resources and Services Administration (HRSA) under Grant (or cooperative agreement) Number T16MC29833.Data Availability Not applicable. The authors declare that they have no conflict of interest. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.