Washington Striving Readers - Year 1 Evaluation Report

Article Abstract

The Washington Striving Readers program provided intensive in-school reading intervention to 176 middle school students who read significantly below grade level. The teachers who provided the intervention received the intended professional development and in-class coaching, and they delivered the intervention the way it was intended, with one exception: fewer lessons were completed than intended, meaning that students did not receive all of the content they were supposed to receive. This was particularly true for students in Group 1, who started the year with difficulty decoding.  The study was designed to combine results from three years in order to have a larger sample size and be able to detect effects of the intervention. Because Congress eliminated funding after the first year of implementation, our sample size was smaller than planned, making it less likely we would find significant effects. For the most part we found no significant differences between the scores of students in the treatment and in the control conditions. There was, however, a significant positive impact on the MSP. As noted earlier, the size of the impact was similar to that found in other Striving Readers programs. We also know that the effect size of the average annual gain of middle school students in reading is about 0.25 (Hill, Bloom, Black, & Lipsey, 2007), so that the gain made by students in the treatment condition was comparable to about five months’ growth. This improvement reduced the gap between low-performing readers and their peers who read at grade level, but did not close that gap.  Students in the treatment condition still had average MSP scores that put them below the cut point to be considered “proficient” readers. We also noted a much larger effect size for Group 1 on the Woodcock Reading Mastery word attack subtest. This finding, although not significant, is promising, and the impact of the Group 1 treatment (Phonics Blitz and Read to Achieve) on students’ decoding skills may merit further study. Originally the study was intended to continue for two additional years. We had hoped that those additional years would allow us to learn more about the impact on students as well as explore whether implementation changed and lesson completion improved when teachers had more experience with the programs. Cutting the study short meant that we were not able to learn everything we had hoped to about the Washington Striving Readers intervention. Nevertheless, there are meaningful lessons from this one-year study that can have important implications for those implementing similar interventions in the future. For example, we learned that it is possible for teachers to attain a high level of implementation, even when teaching two new programs, within a few months of their introduction to the program.  We also found, however, that it is important to attend not only to the fidelity of program implementation but to the amount of material taught during the year. When teaching new programs, teachers may need additional support to ensure appropriate pacing. The findings also demonstrated that it is possible to make a statistically significant difference in struggling students’ overall literacy achievement in the course of one school year. Students in the Washington Striving Readers intervention performed better on the state reading assessment than did students in the control condition, who did not receive any supplemental reading support.  The gains made, however, were not sufficient to bring middle school students who read substantially below grade level up to a proficient level. In light of these and other findings (Vaughn et al., 2011), it may be that these students need more than a one-year intervention. A summer program and/or a second year in intervention might help students make additional progress.   

Citation

Deussen, T., Scott, C., Nelsestuen, K., Roccograndi, A., & Davis, A. (2012). Washington Striving Readers: Year 1 evaluation report. Education Northwest