key: cord-0902155-5p7gog07 authors: Stevens, Elizabeth A.; Murray, Christy S.; Scammacca, Nancy; Haager, Diane; Vaughn, Sharon title: Middle school matters: examining the effects of a schoolwide professional development model to improve reading comprehension date: 2022-03-07 journal: Read Writ DOI: 10.1007/s11145-022-10271-9 sha: b6d4a41e2cd3a79fa35bd209c562f9a7b5ad282e doc_id: 902155 cord_uid: 5p7gog07 Many middle school students perform below grade-level standards in reading (National Center for Education Statistics, Washington, 2019), and recent observation studies demonstrate middle school teachers’ limited use of reading comprehension practices within content area instruction (e.g., science and social studies; as reported by Greenleaf (in: Hinchman (ed) Adolescent literacies: A handbook of practice-based research, Guilford Press, 2017)). In this experimental pilot study, we aimed to boost middle schoolers’ reading comprehension outcomes by providing schoolwide professional development (PD) on integrating reading comprehension practices within content instruction for English language arts, social studies, and science teachers. Six schools were matched into pairs and randomized to the schoolwide PD or a business-as-usual (BAU) condition. Content area teachers in schools assigned to the PD condition received distributed PD resources to support implementation, and coaching in one reading comprehension practice in the fall (i.e., get the gist) and one in the spring (i.e., asking and answering questions). Contrary to traditional PD, this PD was implemented across three content areas, was narrow in scope but long in duration (one practice per semester), focused on practices that could feasibly be integrated into content area instruction, and included ongoing coaching in content area teams. Students in schools assigned to the PD condition significantly outperformed those in the BaU condition on a measure of main idea generation (ES = 0.29) but not on measures of asking and answering questions (ES = 0.11) and general reading comprehension (ES = − 0.09). Findings suggest promise for implementing schoolwide approaches embedded within content area instruction to improve reading comprehension performance for middle school students. Despite large-scale efforts to boost reading comprehension, many students perform below a proficient level in reading (National Center for Education Statistics, 2019). The Institute of Education Sciences, through the Reading for Understanding Initiative, invested more than $100 million to improve knowledge about instructional practices associated with improved reading comprehension for students of all grade levels. The focus of the Reading for Understanding Initiative was to address three areas: (a) underlying processes that contribute to reading comprehension, (b) the development and evaluation of intervention and professional development (PD) programs to improve reading comprehension, and (c) the development of reading comprehension assessments (Institute of Education Sciences, n.d.) . This effort resulted in an extensive, interdisciplinary, and collaborative network of institutions and scholars working to better understand reading comprehension for different populations of students across content areas (Institute of Education Sciences, 2016). These studies provided many insights about reading comprehension across grade and achievement levels, but all sites expressed challenges with actually improving students' reading comprehension outcomes (e.g., Connor et al., 2018; Goldman et al., 2016; Pearson et al., 2020; Scammacca et al., 2016; Vaughn et al, 2013; Wanzek et al., 2013) . There are several possible explanations for the limited improvement in middle grade students' reading comprehension outcomes. First, the literacy expectations increase in the middle grades. Students learn new content knowledge in multiple ways, including by reading text. Readers' background knowledge may be limited because the content is less familiar, resulting in difficulty making inferences and connections (Goldman et al., 2016) . Another possible explanation is that observation studies have demonstrated middle school teachers' limited integration of reading comprehension practices within content area reading (Greenleaf & Valencia, 2017; Swanson et al., 2016) . Classes devoted to reading instruction are rare or nonexistent in middle school, so there may be advantages for content teachers integrating reading comprehension instruction into their teaching routines. However, many teachers assume that middle school students have the necessary skills to comprehend grade-level text and thus do not embed reading comprehension practices when using reading selections (Boardman et al., 2015; Swanson et al., 2015) . Finally, many content area teachers (i.e., social studies and science) do not use text reading as a vehicle for teaching content (Murray et al., 2021; Swanson et al., 2016) ; instead, teachers use other instructional delivery formats (e.g., lecture). When teachers use text reading, they may provide key information orally to students so that students bypass reading the text altogether (Greenleaf & Valencia, 2017) . This approach results in limited practice opportunities for students to engage with text and to develop advanced literacy skills. Content area teachers rarely receive PD focused on improving reading comprehension (Shanahan & Shanahan, 2008; Swanson et al., 2016) , but when they do, it is infrequently research based, ongoing, contextualized, or deep (Brownell et al., 2009; Garet et al., 2001; Grant et al., 1996) . Thus, PD programs fall short and do not always produce reading gains for students, especially those with reading difficulties (Garet et al., 2008) . Many challenges are associated with traditional PD models. First, these models typically include a one-time workshop, which usually occurs outside the school day (Garet et al., 2001) . This type of PD is often ineffective because it is disconnected from the classroom, the duration is not sufficient to facilitate change in teachers' practice, it does not allow for collective participation among groups at the same school, and it lacks ongoing coaching and support for teachers to hone their use of the practice over time (Darling-Hammond et al., 2017; Desimone, 2009; Garet et al., 2001) . Findings from the Reading for Understanding Initiative that indicate that content knowledge is integral to improving comprehension (e.g., Pearson et al., 2020) , coupled with students' persistently poor reading performance, underscore the need for different PD models. Such models must support middle school content area teachers' effective integration of reading comprehension practices while teaching content, increasing opportunities for students to practice reading and learning from text. Few studies have examined the effects of teacher PD on student outcomes, and most have been conducted with students in the elementary grades (e.g., Gersten et al., 2010; Simmons et al., 2010; Swanson et al., 2021) . colleagues (2013, 2017) and Vaughn et al., (2015a Vaughn et al., ( , 2015b examined the effects of middle school teacher PD on the reading comprehension and content knowledge outcomes of middle grade students. Teachers learned a set of instructional practices to improve content knowledge and reading comprehension within social studies instruction, yielding positive outcomes for students' content knowledge and reading comprehension. This PD was ongoing in nature (i.e., included an initial workshop and inclass support throughout implementation), and it focused solely on the integration of effective comprehension and vocabulary practices within social studies instruction. To build upon this model and increase students' opportunities to practice reading and learning from text across the school day, we implemented an alternative, schoolwide approach that (a) provided PD on implementing reading comprehension practices across content areas (English language arts [ELA], social studies, and science), (b) included sufficient practice opportunities for students to read text across the school day, and (c) included instructional practices that are feasible and sustainable for middle school teachers. Across 2 years, we developed a schoolwide PD model-the Middle School Matters (MSM) PD model-optimized for helping middle school science, social studies, and ELA educators implement evidence-based reading practices associated with positive academic outcomes for middle grades students. During the first year, we worked collaboratively with sixth-through eighth-grade science, social studies, and ELA teachers to develop instructional materials for embedding evidence-based reading practices into content area instruction. Our work was informed by Desimone's, 2009 framework and Garet et al.'s (2001) components of effective PD. The model was embedded within teachers' existing school settings, occurred during content area professional learning committees (PLCs) to allow for collective learning and reflection, and provided ongoing coaching and support for the duration of the school year. We purposefully selected high-impact instructional practices that were easy to embed within existing lessons using content area texts and that didn't require teachers to overhaul their content delivery. We started with PD and implementation of get the gist (Klingner et al., 1998) to generate main ideas in the beginning of the academic year and then added asking and answering questions in the middle of the year (Palincsar & Brown, 1984; Rosenblatt, 1978) . The rationale for using one practice per semester was that we wanted teachers to have adequate time to acquire proficiency with one practice before introducing the second practice. We selected get the gist and asking and answering questions because these practices are theoretically grounded and associated with improved reading comprehension outcomes for adolescent readers (e.g.., Goldman, 2012; Kamil et al., 2008; Palincsar & Brown, 1984; Rosenblatt, 1978; Stevens et al., 2019; Vaughn et al., 2011) . Additionally, these practices can be integrated in teachers' existing instruction and used across a variety of content-area texts. Both practices promote active engagement during reading, support monitoring for meaning, and provide opportunities for students to think critically and integrate information across sections of text. We discuss these practices in more detail in the Method section. The MSM PD model consisted of the following components: • Attend a half-day training to learn the reading comprehension practice and plan initial modeling of the practice in content area, grade-level teams • Observe an MSM coach teach a model lesson in a science, social studies, or ELA class • Participate in bimonthly PLCs with an MSM coach to reflect on the implementation of the practice, identify challenges, problem-solve collectively to address those challenges, and set goals for future implementation and use of the practice During the second year, the model was implemented throughout the fall semester for get the gist and again in the spring for asking and answering questions. Researchers collected feedback from teachers as they implemented the two comprehension practices to determine feasibility and usefulness of the model and refined the model as necessary. Given the lack of prior experimental research on the MSM PD model, we conducted an experimental pilot study in six middle schools, examining the effects of the teachers' participation in PD on students' reading comprehension outcomes. We asked the following research questions: (1) Do students in schools where teachers participated in MSM PD significantly outperform those in business-as-usual (BaU) schools on a main idea measure? (2) Do students in schools where teachers participated in MSM PD significantly outperform those in BaU schools on a measure of asking and answering questions? (3) Do students in schools where teachers participated in MSM PD significantly outperform those in BaU schools on a measure of general reading comprehension? We conducted a pilot cluster-randomized trial (CRT) during the 2019-2020 school year in six near-urban middle schools from two southwestern U.S. school districts. We randomly assigned schools to the treatment condition (i.e., MSM PD; n = 3) or the BaU condition (n = 3). Randomization occurred at the school level because the MSM PD is designed to be delivered schoolwide across all core content areas (i.e., ELA, science, and social studies). Because no prior experimental research on the MSM PD model existed, an adequately powered CRT was cost prohibitive due to the number of schools needed. Promising results from this pilot study would support the need for a fully powered efficacy study in the future. Prior to randomization, we matched the six schools into pairs to improve the likelihood of a balanced randomization with baseline equivalence on all pretest measures. We paired the schools based on the available school-level information on variables likely to affect baseline equivalence for reading comprehension: percentage of English learners; percentage of economically disadvantaged; and percentage of students who mastered gradelevel standards, met grade-level standards, or approached grade-level standards on the state-mandated reading assessment from the prior school year. In MSM PD and BaU schools, all teachers who taught the same content area had the same content coverage expectations and standards. Treatment classes differed only in the integration of the reading comprehension practices targeted in the MSM PD (i.e., main idea generation and asking and answering questions). Thus, the focus of this study was the delivery of these reading comprehension practices embedded within teachers' existing content area instruction and materials; we did not provide teachers with content area curricula, texts, or lessons to be used throughout the study other than the initial model lesson. Table 1 provides the demographic data for the six participating schools. All ELA, science, and social studies teachers at participating schools were eligible to participate. Per our Institutional Review Board approval, teachers received an explanation of the study and were invited to participate. A total of 131 teachers (100%) consented to participate in the MSM PD (n = 72) and BaU (n = 59) schools (Table 2 ). MSM PD teachers received a $150 stipend in the fall and spring semesters for completing surveys, attending PD sessions, and planning for implementation of the practices. BaU teachers received $50 stipends in the fall and spring semesters for completing the project surveys. A subset of teachers at each school (MSM PD and BaU conditions) were recruited to assist with collecting consent forms and administering student assessments three times per year; these teachers received an additional stipend of $100 per semester. The participating teachers taught sixth-, seventh-, and eighth-grade students in ELA, science, and social studies classes. All students (N = 4699), regardless of consent status, received instruction from their core ELA, science, and social studies teachers. Data were collected only from students for whom we obtained parent consent and student assent (N = 1952; 42%). During district-provided in-service days in August 2019, science, social studies, and ELA teachers at MSM PD schools participated in a 3-h initial PD workshop on implementing get the gist with content area texts. Teachers received a second PD workshop focused on asking and answering questions in January 2020, after get the gist was (presumably) well-integrated in teachers' classrooms. Specifically, the PD included a brief overview of the study; a description of the research design; an explanation of each instructional practice, how to implement the target practice within content area instruction, and approaches to providing ongoing practice opportunities for students; a model introductory lesson showing teachers how Range 0-29 0-32 to teach the practice; an explanation of data collection procedures; and facilitated planning time with content area teams to plan model lessons connected to relevant content topics and content area texts. A member of the research team (i.e., a Ph.D.level researcher) led individual PD workshops for each MSM PD school but used the same materials and agenda so all participants received the same information. We recorded the PD workshops for those teachers unable to attend the sessions live; as such, all participating teachers received the initial PD training for both practices. Get the Gist Main idea generation (the target practice for the fall PD) is identified as an effective practice for supporting students' reading comprehension (Stevens et al., 2019; Goldman, 2012; Kamil et al., 2008) . Informed by the construction integration model of comprehension processing (Kintsch, 1988) , get the gist is a paraphrasing process that supports students with developing a coherent situation model by reading initially brief sections of text, identifying the most important information, and integrating that information with previously processed main ideas to develop a deeper understanding of the text (Klingner et al., 1998) . Students stop at predetermined points during reading and use a three-step process to write their gist (main idea): (1) Who or what is this section about? (2) What is the most important information about the "who" or "what"? and (3) Combine the information from steps 1 and 2 to write the gist. Students record their gists on learning logs. Asking and answering questions Questioning (the target practice for the spring PD), rooted in reader-response theory and reciprocal teaching (Palincsar & Brown, 1984; Rosenblatt, 1978) , is a reading comprehension practice that facilitates active engagement with text, comprehension monitoring, and literal and inferential comprehension. During reading, students pause to write questions about important information in the text and answer those questions, citing textual evidence. Students generate two types of questions (i.e., specific and wide) and record the questions, answers, and evidence on question logs. Specific questions can be answered in one word or sentence; wide questions can be answered using information from (a) multiple places in the text, (b) the text and the reader's prior knowledge, or (c) the text and another text (Stevens et al., 2020a) . This practice helps students think about what is being communicated and how information relates across paragraphs. Researchers provided materials, customized for each content area, to all participating teachers to support the PD and implementation of the research-based practices. In addition to paper copies, teachers received links to private webpages containing electronic versions of all materials. Researchers ensured the materials' usability and feasibility by conducting focus groups with middle school science, social studies, and ELA teachers during the PD model development phase. Practice guides As part of the initial PD, researchers provided participants with practice guides. The guide provided a description of the research base for the practice, implementation steps, sample model lessons for each content area (see next section), and frequently asked questions. Participating teachers received sample model lessons, which included a content area text notated with example language for thinking aloud while modeling the practice with that text. The lesson packet also included an "answer key" (acceptable gists for each text; sample questions, answers, and textual evidence for each text) and a student copy of the text. These samples helped researchers explain and demonstrate how to introduce each practice and model its use with students. Teachers could use the sample lesson to deliver their initial model lesson in their classroom or as an "exemplar" from which to develop their own model lesson using a different text. Teacher and student materials Participants received physical copies of all materials needed to implement the two practices. Specifically, teachers received classroom posters, student cue cards listing the steps for each practice, get the gist learning logs, and question logs. As a follow-up to PD, MSM coaches provided ongoing coaching during brief PLC meetings (i.e., 15-min) every 2 weeks to help teachers refine implementation of the reading practices. MSM coaches conducted PLC meetings via Zoom or in person using the following: (1) Share experiences using the practice, (2) brainstorm solutions to potential implementation challenges, and (3) set goals for using the practice during the upcoming two weeks. As mentioned previously, the study was conducted during the 2019-2020 school year in which schools closed due to the COVID-19 pandemic. Access to schools and study implementation was not impacted by the pandemic because, by design, we completed the spring PD, implementation, and ongoing coaching sessions before school closures occurred. However, we were unable to collect posttest data on the asking and answering questions subtest for the seventh graders at one school (See the Missing Data section for further explanation). We asked participating teachers to provide main idea instruction for 10 weeks during the fall semester and questioning instruction for 9 weeks during the spring semester. During Week 1, following the initial PD, teachers observed the MSM coach deliver a get the gist lesson in teachers' classrooms across content areas. The lesson modeled how to (a) set a purpose for learning (i.e., reasons using get the gist is important and helpful), (b) introduce get the gist, and (c) demonstrate get the gist with a text while thinking aloud. Debriefing occurred after the lessons so that teachers could ask follow-up questions. During Week 2, teachers delivered their own get the gist model lesson to students. Beginning in Week 3, teachers were asked to provide at least two practice opportunities using get the gist each week. Teachers did not receive scripted lessons or specific texts to use when teaching these practices; researchers asked teachers to integrate the practice within their existing content area instruction twice per week, meaning a given middle school student would have at least six opportunities weekly to practice main idea generation across social studies, science, and ELA classes. To support teachers with embedding the practice within their own lessons rather than relying on scripted lessons, we provided teachers with a "menu" of practice activities. For example, science teachers might ask students to write gist statements at points during a science lab that focus on the hypothesis, procedures, and results of the experiment. Teachers participated in a similar PD and implementation schedule with asking and answering questions in the spring semester. All teachers who attended the initial PD workshops rated the quality of the PD. Teachers rated their understanding of the practice, preparedness to implement the practice, willingness to implement the practice, and campus and district support from 1 to 4, with 1 being strongly disagree and 4 being strongly agree. Table 3 presents the results of the PD ratings for each practice. Teachers rated the PD positively (M range 3.49 to 3.68 for get the gist and 3.32 to 3.55 for questioning), indicating they had a high level of understanding of the practice, willingness to use the practice, and district/campus support. Researchers collected implementation fidelity data from MSM PD teachers via an online survey every 2 weeks. During initial PD, researchers provided teachers with implementation goals (i.e., dosage) for using get the gist or questioning during each cycle (i.e., 2-week implementation period). Teachers reported the number of times they implemented the practice during each cycle via the online survey. The dosage fidelity data and implementation goals for each cycle are presented in Table 4 . Overall, teachers reported implementing get the gist between 1.62 and 3.05 times, on average, during each cycle, and a range of 39.7% to 95.6% of teachers met the dosage goal per cycle across the semester. For each content area, ELA and Science teachers reported using the practice similarly across cycles 2 to 5, but social studies teachers reported using the practice the most (M range 3.05 to 3.33). Teachers reported implementing asking and answering questions, on average, between 2.33 and 3.01 times per cycle, and a range of 39.7% to 95.6% of teachers met the dosage goal per cycle. For each content area, science teachers reported using the practice the least, followed by ELA teachers. Social studies teachers reported using the practice more than science and ELA teachers across most of the cycles. In addition to dosage data, MSM PD and BaU teachers completed a self-report survey about their use of the get the gist and asking and answering questions last Table 3 Professional development ratings Teachers rated the quality and usefulness of each practice after the PD sessions, with 1 being strongly disagree, year (prior to implementation of the MSM PD model) and currently (after implementation of the MSM PD model). Teachers completed the get the gist self-report survey at the end of the 10-week implementation window in the fall semester and the asking and answering questions self-report survey at the end of the 8-week implementation window in the spring semester. For each survey, teachers rated their use of the instructional practice the previous year versus currently on a scale of 1 to 5, with 1 being never, 2 being rarely, 3 being occasionally, 4 being often, and 5 being routinely. Table 5 Researchers collected social validity data from MSM PD teachers via an online survey at the conclusion of fall (get the gist) and spring (asking and answering questions). Teachers rated statements about the usability of the practices, feasibility of the practices, and the impact of the practices on students' reading comprehension. Teachers responded to each statement on a scale of 0 (disagree) to 2 (agree). The social validity data are presented in Table 6 . Ratings indicated teachers understood how to use the practice, could easily implement the practices within current instruction, and positively affected students' content-area reading comprehension (M range 1.61 to 1.97). Researchers provided teachers with training on procedures for administering measures. MSM PD and BaU teachers administered the following reading comprehension measures to participating students: the Gates-MacGinitie Reading Test reading comprehension subtest (GMRT-RC; MacGinitie et al., 2000) and the Strategy Use Measure -Middle School (SUM-MS; The Meadows Center for Preventing Educational Risk, 2019). Teachers administered the GMRT-RC at the beginning and end of the school year. Teachers administered the SUM-MS in August (pretest for get the gist), January (posttest for get the gist; pretest for asking and answering questions), and March (posttest for asking and answering questions). The GMRT-RC is a group-administered, 35-min assessment of generalized reading comprehension. Students silently read expository and narrative passages that vary in length and answer three to six multiple-choice questions immediately after reading each passage. Reliability estimates for the subtest range from 0.91 to 0.93 for internal consistency and 0.80 to 0.87 for alternate forms. Students were administered Form S at pretest and Form T at posttest. The research team adapted the elementary-grade version of the SUM (The Meadows Center for Preventing Educational Risk, 2008) for use with middle school students. The SUM-MS contained two subtests assessing students' application of two reading comprehension practices: main idea generation and asking and answering questions. For the main idea generation subtest, students read three leveled passages containing social studies, science, and ELA content and then wrote a main idea statement for each passage. Lexile levels for the passages ranged from 810 to 1200. The subtest was administered in approximately 20 min for a total of 18 possible points. Each main idea statement was scored from 0 to 6 points, with 2 points awarded for the correct "who" or "what" (e.g., railroads) and 4 points awarded for key components in the most important idea about the "who" or "what" (e.g., improved transportation and the economy in the United States). For the asking and answering questions subtest, students read three passages and answered multiple-choice items that assessed students' understanding of specific (literal) and wide (inferential) questions and answers specific to each passage. The passages included narrative and expository texts that addressed grade-level science and social studies topics. Lexile levels ranged from 810 to 1000. Although the test was not timed, administration took approximately 20 min. One point was awarded for each correct answer to the multiple-choice questions, for a total possible score of 20 points. In this experimental pilot study, six schools were matched into pairs and randomly assigned within pairs to the MSM PD or BaU condition. As a result, student data were nested at the school level and cross-classified at the teacher level (because students' schedules differed in the teachers' classes to which they were assigned for social studies, science, and ELA). We acknowledge the nested and cross-classified nature of the student data but did not model it in our analyses. The small number of schools that participated would not provide sufficient power for examining our research questions in the context of a multilevel, cross-classified model. Additionally, the purpose of the study was to determine whether MSM PD demonstrated promise as a means of improving reading comprehension in middle school students. If it did show promise, funding an efficacy study that is fully powered to examine effects in an appropriate multilevel model would be warranted. Therefore, to address our research questions, analysis of covariance (ANCOVA) models were estimated for each student outcome. The pretest for each outcome was included as a covariate. Student-level effect sizes and the associated standard errors were calculated using the formula for Hedges' g. Following a description of the missingness of data from our sample, we present the results of the small-scale experimental study of MSM PD by research question. Descriptive statistics across all measures and research questions are presented in Table 7 . The time required to score the SUM-MS and the large number of students involved in the study made it prohibitive to score all protocols. Therefore, we selected a stratified random sample of 780 students from the 1777 students who responded to the SUM-MS main idea generation subtest at both the beginning of the year and middle of the year. The sample was stratified on school and grade level to ensure that a balanced sample was selected. As a result, no students were missing scores on the main idea generation subtest and missing data for the students whose protocols were not sampled were missing by design. It was our intention to score the SUM-MS asking and answering questions subtest for the same sample of 780 students. However, at one middle school the end-of-year data collection for the asking and answering questions subtest was not completed for seventh-grade students before the school closure for COVID-19 mitigation. To preserve our sample size and have adequate representation of seventh-grade students, additional seventh-graders were randomly selected from the remaining sample across the other participating schools. The middle school that did not collect end-of-year data for seventh-grade students was the smallest of the schools that participated in the study. Therefore, to ensure that scores from a sufficient number of students from this school were included, all of the SUM-MS protocols for students in grades 6 and 8 at this school were scored. Additionally, across all participating schools, 14.9% of students in the stratified random sample who responded at the middle-of-year time point did not complete the end-of-year asking and answering questions subtest. These data were missing due to these students being absent on the day the test was administered or having transferred out of the school. As a result, we considered these data to be missing at random and students with missing data were not included in the analyses of the SUM-MS scores. GMRT-RC protocols were machine scored, allowing us to include all students in the analysis of outcomes. Pretest scores were missing for 6.1% of the total sample of students, with 4.3% missing from the BaU schools and 7.1% missing from the MSM PD schools. Posttest scores were missing for 16.3% of these students, with 17.8% missing from BaU schools and 15.7% missing from MSM PD schools. Missing data on the GMRT-RC were due to student absences on the day of testing or students transferring into or out of the participating middle schools. We considered the missing data on the GMRT-RC to be missing at random. Students with missing data were listwise deleted from the analyses. Before analyzing posttest data to address our research questions, Hedges' g effect sizes for pretest scores from students in each condition were calculated to determine whether the groups were equivalent at baseline. Using criteria from the What Works Clearinghouse, baseline equivalence was defined as an effect size below 0.25. For the SUM-MS, the effect size for the difference between groups at pretest favored students in the MSM PD condition on the main idea generation subtest (g = 0.05; SE = 0.07) and the asking and answering questions subtest (g = 0.11; SE = 0.07). For the GMRT-RC, the effect size for the difference between groups at pretest favored students in the BaU condition (g = − 0.11; SE = 0.05). In addition to being below the 0.25 criteria, none of the effect sizes for pretest scores differed significantly from 0. Therefore, we determined that the groups were equivalent at baseline and included the pretest scores as covariates in the analytical models to adjust for the small pretest differences. For Research Question 1, we asked whether students in schools where teachers participated in MSM PD significantly outperformed those in BaU schools on a main idea measure. To address this research question, posttest scores from the SUM-MS main idea generation subtest were analyzed in an ANCOVA model where pretest scores were included as a covariate. Results indicated that students in schools where teachers participated in MSM PD scored significantly higher at posttest than students in schools where teachers did not participate in MSM PD, F(1, 777) = 15.78, p < 0.001. The effect size for the difference in posttest scores was 0.29 (SE = 0.07; 95% CI = [0.15, 0.43]). For Research Question 2, we asked whether students in schools whose teachers participated in MSM PD significantly outperformed those in BaU schools on an asking and answering questions measure. Posttest scores from the SUM-MS asking and answering questions subtest were analyzed in an ANCOVA model with pretest scores as a covariate. There was no statistically significant difference between students in schools where teachers did and did not participate in MSM PD, F(1, 744) = 0.13, p = 0.72. The effect size was small and did not differ significantly from 0, g = 0.11 (SE = 0.07; 95% CI = [− 0.03, 0.25]). For Research Question 3, we asked whether students in schools whose teachers participated in MSM PD significantly outperformed those in BaU schools on a measure of general reading comprehension. We addressed this research question by analyzing GMRT-RC posttest extended scale scores in an ANCOVA model with pretest scores used as a covariate. The results showed no statistically significant difference between students in schools where teachers did and did not participate in MSM PD, F(1, 1610) = 0.10, p = 0.92. The effect size for the difference between groups was not significantly different from 0, g = -0.09 (SE = 0.05; 95% CI = [− 0.19, 0.01]). Although most middle school teachers recognize the importance of using effective reading comprehension practices, finding mechanisms for enhancing reading comprehension within the "content-oriented" curriculum of the middle grades is challenging. Our team was interested in supporting middle school teachers with implementing feasible reading comprehension practices associated with improved outcomes across content area learning. With this in mind, we designed a PD model at the school level that could be implemented within social studies, science, and ELA content instruction. This study reports the findings from an experimental pilot study testing the effects of this schoolwide PD model on the reading comprehension performance of middle school students. Students in the MSM PD and BaU conditions received the same content area instruction in science, social studies, and ELA. For students in schools assigned to the MSM PD condition, their teachers embedded specific reading comprehension practices (i.e., get the gist and asking and answering questions) during content area instruction. This provided students multiple practice opportunities with each reading comprehension practice across content areas and throughout the semester. The findings demonstrated that students in schools assigned to the MSM PD significantly outperformed students in schools assigned to the BaU condition on a student measure of main idea performance but not on measures of asking and answering questions and general reading comprehension. We interpret these findings as showing initial promise for implementing schoolwide PD to improve the reading comprehension performance of middle school students. Results from the schoolwide implementation indicated that students in the MSM PD condition demonstrated statistically significant positive outcomes on main idea development (ES = 0.29). There are several reasons why we consider this impact promising. First, content area teachers provided all of the comprehension instruction within the context of their own content area instruction. There was no "set-aside time or class" to teach these comprehension practices. Further, classroom teachers provided these comprehension practices without the benefit of scripted lessons or researcher-selected texts. Teachers received PD on the comprehension practices with guidance on how they might be integrated within their existing content instruction and received ongoing support via coaching throughout the semester. We think this finding is relevant, as it suggests a transfer of learning from the PD to teachers' own teaching routines. It also suggests that students' main idea understanding can be significantly improved with relative ease. The SUM-MS asking and answering questions subtest assessed students' skill in identifying and answering different question types (i.e., literal and inferential) related to reading and understanding texts. Posttest scores from the SUM-MS asking and answering questions subtest revealed no statistically significant difference between students in schools where teachers did and did not participate in MSM PD (ES = 0.11). However, teachers' self-reported use of the practice increased from the previous school year and their reported dosage was greater than that of get the gist. There are several possible explanations for these findings. First, the implementation cycle for asking and answering questions was slightly shorter than for main idea generation (9 versus 10 weeks), meaning students had less exposure and practice opportunities with the various question types. The shortened implementation cycle was due to the need to schedule data collection prior to spring break. It may be that students simply did not have enough practice with asking and answering both question types. Additionally, the asking and answering questions subtest of the SUM-MS contained multiple-choice items, unlike the written responses generated on the main idea subtest. It may be that the multiple-choice format was not sensitive enough to detect improvement in students' questioning understanding. Future research of the MSM PD model might include an open-ended assessment of asking and answering questions; this format may be more sensitive to detecting students' change in questioning understanding. Finally, we assessed the impact of the MSM PD on students' general reading comprehension performance. There were no statistically significant differences between students in the MSM PD or BaU condition on the GMRT-RC (ES = − 0.09). This finding is consistent with prior PD studies (e.g., Swanson et al., 2021; Vaughn et al., 2017 Vaughn et al., , 2015a Vaughn et al., , 2015b , demonstrating a pattern of null findings on distal measures of reading comprehension. As has been pointed out in previous research (Fuchs et al., 2018; Stevens et al., 2020b) , it may be that the GMRT-RC is a measure of other constructs (e.g., vocabulary and knowledge) and not apt for measuring students' growth in main idea generation and questioning. Also, due to the relatively short period of time between pretesting and posttesting, it may be that there simply wasn't enough time for students' learning of the practices to generalize to their overall reading comprehension skill. We think that this study shows promise for designing feasible PD in ways that provide opportunities for teachers to implement comprehension practices within their content area instruction with potential benefit to students-at least with respect to generating main ideas of text. Our findings are consistent with prior research examining PD for upper-elementary and middle school teachers, which resulted in positive student outcomes on proximal measures of reading comprehension but not distal measures of reading comprehension (Swanson et al., 2021; Vaughn et al., 2017 Vaughn et al., , 2015a Vaughn et al., , 2015b . Though the findings from this study align with prior research, the MSM PD model differs from previous PD models in several important ways that position the findings as promising. First, in spite of the challenges with improving the reading comprehension of middle grade readers, the MSM PD model resulted in significant growth in main idea understanding through PD that was feasible for teachers, narrow in scope (i.e., one reading comprehension practice per semester), and easy to implement within existing school settings. We recognize the many demands on middle school teachers, so our intent was to design a PD approach that was feasible for teachers to apply and also integrate within their instruction. We did not ask middle school teachers to substantially change their instruction but instead to use one practice several times each week during content area text reading. Second, the MSM PD model differs from previous studies in that we provided relatively minimal support to these teachers (e.g., initial PD and coaching every other week of implementation). The MSM PD model provided teachers with practices guides to assist with initial understanding of the practice, but teachers were not given specific texts, lesson plans, or scripted lessons to use during implementation. Third, the MSM PD is unique in that it was implemented schoolwide, targeting science, social studies, and ELA teachers versus occurring within one content area (e.g., social studies; Swanson et al., 2021; Vaughn et al., 2013) . This implementation ensured that students had sufficient practice opportunities with the reading comprehension practices across content areas, across text types, and throughout the school week. We collected implementation fidelity data via dosage and self-report surveys. In the dosage survey, teachers reported how often they used the practice during a two-week time period (Table 4 ). In the self-report survey, teachers reported how often they used the practice last year versus in the current year (Table 5 ). In the latter survey, teachers rated their usage on a scale of 1 to 5, with 1 being never and 5 being routinely. We recognize the limitations to collecting implementation fidelity data using selfreport measures and conducting observations would have been preferable; however, we were unable to do so for several reasons. First, it was not feasible to conduct classroom observations, given the large number of teachers in this study. Second, we did not collect audio recordings because teachers had the flexibility to integrate the practices within their existing instruction, rather than using researcher-provided, scripted lessons on specific days or at specific times. As a result, coding audio-recording data was not feasible because it would require the research staff to sift through hours of audio recordings to locate teachers' use of the target practice. In addition, we felt that asking teachers to audio record their use of the practice once or twice per week would not be successful, as teachers would likely forget to turn on their devices, resulting in missing data. We recognize social desirability may have influenced teachers' responses, but we also position our findings and the implementation fidelity data within the context of an experimental pilot study; our aim was to examine the MSM PD model for initial promise. In future studies examining schoolwide PD models across content areas, it would be important for researchers to collect more extensive implementation fidelity data using in-person observations. In addition, observations would provide useful information on how teachers implement the practices within each content area (i.e., number of practice opportunities, gradual release of responsibility, type of texts used, frequency and quality of teacher feedback). This information would be useful in making additional changes, as needed, to the MSM PD model to address teachers' needs based on the information gleaned from the observations. A second limitation of this initial experimental study of the MSM PD was that while the sample of student participants was quite large, the sample of schools was only 6. We were unable to analyze the data in a way that reflected that students were nested in schools and cross-classified across content area teachers, which is a limitation of our findings. Given that MSM PD students outperformed BaU students on the measure of main idea, a full-scale CRT is warranted with a larger sample of schools to allow the data to be modeled as nested and cross-classified with sufficient statistical power. The estimates of effect size for both practices from the present study can be used in a future power analysis to determine the number of schools, teachers, and students that would be required for a fully powered efficacy study. In the introduction of this paper, we described the large-scale efforts and extensive funding invested to improve reading for understanding and the challenges with improving students' reading comprehension outcomes (e.g., Connor et al., 2018; Goldman et al., 2016; Pearson et al., 2020; Scammacca et al., 2016; Vaughn et al., 2013; Wanzek et al., 2013) . These findings warrant alternative, schoolwide approaches that suggest promise to move the needle for middle schoolers' reading comprehension outcomes. This study indicates that schoolwide, targeted, and ongoing PD woven into content area instruction may hold promise for improving text comprehension (i.e., main idea generation) of middle grade students. The impact of this study might be considered much more promising in light of the feasibility for teachers, especially because many teachers resist "layering practices" onto what they are already doing. We are interested in further pursuing the extent to which schoolwide approaches to improving reading comprehension might be integrated into content area instruction in ways that could be associated with meaningful impact for students. The efficacy of Collaborative Strategic Reading in middle school science and social studies classes. Reading and Writing The role of domain expertise in beginning special education teacher quality Examining the efficacy of targeted component interventions on language and literacy for third and fourth graders who are at risk of comprehension difficulties Effective teacher professional development Improving impact studies of teachers' professional development: Toward better conceptualizations and measures Evaluating a multidimensional reading comprehension program and reconsidering the lowly reputation of tests of near-transfer The Impact of Two Professional Development Interventions on Early Reading Instruction and Achievement. NCEE 2008-4030 What makes professional development effective? Results from a national sample of teachers Teacher study group: Impact of the professional development model on reading instruction and student outcomes in first grade classrooms Adolescent literacy: Learning and understanding content Common themes in teaching reading for understanding: Lessons from three projects Learning to teach mathematics in the context of systemic reform Institute of Education Sciences. (n.d.). Reading for understanding research initiative Reading for understanding research initiative Improving adolescent literacy: Effective classroom and intervention practices (NCEE #2008-4027) The role of knowledge in discourse comprehension: A construction-integration model Collaborative Strategic Reading during social studies in heterogeneous fourth-grade classrooms Gates-MacGinitie reading tests Teachers' text use in middle school contentarea classrooms. Reading and writing Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities Reaping the rewards of the Reading for Understanding Initiative. National Academy of Education Rosenblatt A century of progress: Reading interventions for students in grades 4-12 Teaching disciplinary literacy to adolescents: Rethinking contentarea literacy A comparison of multiple-strategy methods: Effects on fourth-grade students' general and content-specific reading comprehension and vocabulary development Using question generation to improve reading comprehension for middle-grade students Summarizing and main idea interventions for upper elementary and secondary students: A meta-analysis of research from 1978 to 2016 The effects of a paraphrasing and text structure intervention on the main idea generation and reading comprehension of students with reading disabilities in grades 4 and 5 The efficacy of two models of professional development mediated by fidelity on fourth grade student reading outcomes Literacy and text reading in middle and high school social studies and English language arts classrooms. Reading & Writing Quarterly Improving reading comprehension and social studies knowledge among middle school students with disabilities The Meadows Center for Preventing Educational Risk The Meadows Center for Preventing Educational Risk Efficacy of collaborative strategic reading with middle school students Improving content knowledge and comprehension for English language learners: Findings from a randomized control trial Improving reading comprehension for high school students with disabilities: Effects for comprehension and school retention Improving middle-school students' knowledge and comprehension in social studies: A replication Improving reading comprehension and social studies knowledge in middle school Extensive reading interventions for students with reading difficulties after grade 3. Review of Educational Research Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations Acknowledgements This research was supported by the Institute of Education Sciences (IES) through Grant R305A170556 to The University of Texas at Austin. We wish to acknowledge and thank the George W. Bush Institute, with funding from the Meadows Foundation and other private foundations, for their leadership during the initial multiyear work of Middle School Matters prior to IES funding. The opinions expressed are those of the authors and do not represent views of IES or the U.S. Department of Education.Funding This research was supported by the Institute of Education Sciences (IES) through Grant R305A170556 to The University of Texas at Austin. We have no conflicts of interest to disclose.