By Alanna Bjorklund-Young, Research Fellow
Johns Hopkins Institute for Education Policy
Strong literacy skills are increasingly important for success in college and high-skilled jobs (Murnane et al., 2012). Recent results on the National Assessment of Educational Progress (NAEP), however, show that only 34% of 8th graders read at a level considered “proficient” (USDOE, 2015). When proficiency rates are disaggregated by racial sub-groups, we find that only 16% of African American and 21% of Hispanic students read at the proficient level (USDOE, 2015).
In elementary school, students learn how to read. By middle school, they read in order to learn new content (Goldman, 2012). The demands on reading comprehension thus become increasingly important throughout the secondary years and into college. It is vital to remediate struggling readers in middle school, before weak reading skills negatively impact them across a variety of subjects for the rest of their academic careers.
Many school districts understand the urgent nature of this crisis, and programs do exist that show evidence of success in boosting struggling students’ reading. Implementing them with fidelity, however, proves a perennial challenge. What does research say about teaching literacy skills to struggling adolescent readers?
An Overview of Main Solutions
A variety of different approaches aim to boost adolescent literacy, from computer-based supplementary programs to systematic reforms at the school level. However, a meta-analysis of adolescent literacy programs found that what distinguishes the most effective interventions is that they explicitly change daily teaching practices. This finding is supported by a recent summary of effective adolescent literacy programs, which found that the most effective programs commonly support changes to instructional practices, such as explicit reading comprehension instruction, providing instructional routines, employing cooperative learning, and teachers’ provision of feedback (Herrera et al., 2016). While clearly there are many different interventions that can support changes to a teacher’s daily instruction, curricular changes and professional development are the interventions highlighted below that directly influence student reading. In contrast, computer-assisted instructional programs require little additional training or changes to a teacher’s current instructional practices. These interventions do have a positive effect, but it is not as great as those that alter classroom instruction itself.
The estimated effect sizes and the strength of evidence of interventions are listed below (Slavin et al., 2008):
|Intervention||Description||Number of Studies||Number of Students (approximately)||Effect Size (weighted mean)|
These interventions change the content and format of how literacy is taught.
|9||10,000 students||0.23 standard deviations (SD)|
|Professional development and training||
These interventions provide teachers with extensive training in order to implement specific instructional methods.
|7||9,700 students||0.28 SD|
|Computer assisted instructional programs||
These interventions are computer programs that students use independently, typically for additional reading instruction.
|8||13,000 students||0.10 SD|
|Comprehensive middle school reforms||
These interventions include changes to instruction, curriculum, assessment, classroom management, parental involvement, and so on. Note that middle school reading instruction is only one part of the intervention.
|2||18 schools; over 1,600 students||0.12 SD|
To better understand the practical importance of the effect sizes, recall that the studies cited above seek to evaluate the effect of an intervention on student learning. Researchers, therefore, compare student learning of two similar groups of students: one group that received the intervention (the treatment group) and one group that continued with their pre-existing learning environment (the control group). Effect sizes are estimates of additional learning and tell us how much more learning the treatment group gained over the control group. Students on average gain about 0.27 standard deviations (SD) of reading knowledge in each year of middle school (Hill et al., 2008). Therefore, a finding of an average effect size of 0.28 SD means that students gained an additional 0.28 SD than they would have if they had remained in their pre-exiting learning environment, which translates into an additional 187 days of learning when their teachers participated in high-quality literacy professional development.
Specific Programs With Strong Research Evidence
While the estimates above provide important information about which kinds of programs are most impactful, recent research also addresses specific programs that improve adolescents’ literacy skills. The specific programs discussed below, Read 180 and Voyager Passport Reading Journeys (Voyager), use a combination of the general strategies highlighted above, but place the majority of their focus on changing daily teaching practice – primarily through curricular changes. However, both programs also provide teacher training and include computer-assisted instructional components.
READ 180 is an intervention program for students in grades 4 through 12 who are two or more years below grade level in reading. There is strong evidence that Read 180 produces medium to large positive effects for struggling readers in reading comprehension, general literacy achievement, and reading fluency. The program is structured around 90-minute reading sessions that consist of whole-group lessons, small-group rotations, and a whole-class summary at the end. The program consists of both direct and online instruction, includes a student assessment component, and teacher professional development.
Nine recent studies meet What Works Clearinghouse’s (WWC) edibility requirements, three randomized control trials (RCTs) without reservation and six quasi-experimental or RCTs with reservation. Of these studies, six studies estimated the impact of Read 180 in students’ reading comprehension—three of the studies found positive and statistically significant effects and three of the studies found indeterminate effects for reading comprehension. Six of the studies estimated the impact of the program on general literacy achievement—three of the studies found statistically significant positive effects and no studies found statistically significant negative effects. Two studies estimated the effect of Read 180 on reading fluency—one study found positive and statistically significant effects and the other study found indeterminate effects (WWC, 2016).
How do these findings translate into additional student learning? To illustrate the magnitude of the Read 180’s effect sizes, three RCTs estimated the following effects: Swanlund et al. (2012) found that Read 180 increased student learning by 0.14 SD (93 additional days of learning) for 6-9th graders in Milwaukee Public Schools in general literacy achievement; Kim et al. (2010) found that Read 180 increased students’ reading fluency by 0.12 SD (80 additional days of learning) and sight word reading by 0.11 SD (73 additional days of learning) for 4th through 6th grade students in Massachusetts; and Kim et al. (2011) found that the program produced effect sizes of 0.32 SD (213 additional days) in reading comprehension for 4th through 6th graders in an after-school program in Massachusetts (Kim et al., 2011).
The weighted mean effect size of Read 180, which held across eight earlier longitudinal studies, was 0.24 SD (Slavin et al., 2008). Note that it is common for RCTs to find smaller effect sizes than studies than results from non-randomized data. It is therefore unsurprising that Slavin’s estimates are larger than some of the estimates found in the RCTs cited above. What is most important to note is that both sets of results provide strong evidence that Read 180 produces meaningful learning gains for students with below-grade-level literacy skills.
There is some evidence that another program, Voyager, produces positive effect sizes for struggling readers. Voyager is a one-year reading intervention program designed for below-grade-level readers in middle and high school. The program provides explicit, systematic, and scaffolded reading instruction through content selected to be of high interest and an interactive format.
Two recent RCTs found positive to negligible results of the Voyager program. The effect sizes were statistically insignificant amongst 9th graders in Illinois (Dimitrov et al., 2012) and 7th and 8th graders in Virginia (Scheneck et al., 2012). However, researchers found statistically significant effect sizes of 0.27 SD (180 additional days) in comprehension amongst 6th and 7th graders in Louisiana (Vaden-Kiernan et al., 2012). An earlier study of 9th graders with limited English Proficiency also found statistically significant mean effect sizes of 0.17 SD (113 additional days) (Shneyderman, 2006). These results show that Voyager produces large literacy gains within some contexts. However, the range in effect sizes show that the success of the program is context-dependent; it is not well understood how to best support the efficacy of this program.
Clearly, some instructional changes prove to be effective interventions for young struggling readers, and in boosting middle and high school students’ literacy skills by 0.20-0.35 SD in general (Slavin et al., 2008; 2011). However, the programs reviewed in the studies below are not specifically designed for struggling readers. It is therefore unclear how effective these strategies translate for struggling readers; targeting all students with high-quality instruction might raise all students’ learning, but struggling readers might best learn with additional supports.
One instructional program, Student Team Reading (Stevens & Durkin, 1992), is a cooperative learning programs for secondary students. The program incorporates a cooperative learning model in which students work in teams of 4-5 students, engage with high-interest reading material, and receive explicit reading comprehension instruction.
Two high-quality studies have found large to moderate effects for the Student Team Reading Program. Stevens and Durkin (1992) found positive, but not statistically significant, effect sizes of 0.11 SD (73 additional days of learning) for reading comprehension and 0.08 SD (53 additional days) for general literacy achievement in six middle schools in Baltimore City Public Schools. In another study, Stevens (2003) found positive and significant effect sizes of 0.33 SD (220 additional days) in reading vocabulary, 0.25 SD (167 additional days of learning) in reading comprehension, and 0.38 SD (253 additional days of learning) in language expression in five middle schools in a large urban district in grades 6th through 8th. Such outcomes are stunning.
Another program that changes teachers’ instructional practices is Promoting Acceleration of Comprehension and Content Through Test (PACT), which boosts students’ reading comprehension by linking new, text-based learning to students’ prior knowledge. PACT has five research based components: a comprehension canopy that contains an overarching issue or question; key words; knowledge acquisition through text-based instruction and reading; team-based comprehension checks; and team based knowledge application. Teachers receive pre-service training and in-class coaching, and the program includes teacher planning sessions that help implement these instructional practices (Vaughn et al., 2013).
One RCT found that 8th grade students assigned to social studies teachers who used PACT showed statistically significant higher across three outcomes: 0.17 SD (113 additional days) in content acquisition; 0.29 SD (193 additional days) in content-specific reading comprehension; and 0.20 SD (133 additional days) on standardized reading comprehension (Vaughn et al., 2013).
Whole School Solutions
Whole school solutions provide another promising intervention for boosting adolescent literacy. However, this approach appears best for schools that wish to address multiple problems, and is does not produce the largest effect sizes for boosting literacy specifically.
An example of an effective “whole school solution” program is the Talent Development Middle Grades Program. This program aims to increase student achievement and student and teacher expectations through changes to the structure and curriculum of high-poverty, large, urban middle schools. The program reorganizes each school into small interdisciplinary learning communities, where teacher teams share the same students and work within a common planning time. The program introduces evidenced-based curricula that are tied to standards in Language Arts, math, science, and US history. It also offers remedial courses in math and reading (U.S. Dept. of Education, 2013).
A study of 7th– and 8th-graders in 29 urban middle schools in the Northeast found inconsistent results for this program across five years. In one year, one cohort of students showed statistically significant reading improvements, while during other years, other cohorts showed no significant reading improvements (Herlihy and Kemple, 2004). This program is deemed as providing “potentially positive” results for increasing reading comprehension by What Works Clearinghouse (U.S. Dept. of Education, 2013).
The limits of our understanding
The research above provides evidence of which general strategies and specific interventions are effective at increasing the literacy skills of struggling adolescent readers. These results are especially promising because research has shown these strategies and interventions to be effective across multiple studies and therefore among different schools and with different groups of students.
Research has not yet captured the consequences of context upon the effectiveness of a specific intervention, however. Specifically, none of the studies cited above is able to isolate which aspect of a school’s context, such as a school’s leadership, environment or student characteristics, impacted the efficacy of the intervention. Therefore it is unclear why an intervention was so effective in one school but not another. The generalizability of results from specific studies to new contexts is a common issue across many domains of research, and is called external validity. External validity is important for policymakers who need to understand not only the research basis but also the conditions that should be in place in order to maximize the effectiveness of an intervention. For example, did the studies that show the largest effects with Read 180 have the best teacher training or the largest teacher or school buy-in, or was there some other school-level factor that made this program more effective?
While external validity is a concern in virtually all education research, it is particularly salient in this context. Why? Because the estimated effect sizes of the specific programs under consideration vary substantially. For example, Read 180 had estimated effect sizes ranging from 0.14 SD to 0.19 SD, or 93 to 127 days of additional learning, in RCTs. While these differences appear relatively small, they translate into important differences in how long it takes a student to reach grade level. For a student who is two years behind in reading skills, this difference in effect sizes translates into catching up in three years’ time (for an effect size of 0.19 SD), versus catching up in four years’ time (for an effect size of 0.14 SD). It is thus indisputable that context matters in how literacy programs translate into student learning – but how precisely, and concerning which elements of a given context, remain meaningful, real-world questions left unanswered by this research.
Boulay, Beth, Barbara Goodson, Michael Frye, Michelle Blocklin, and Cristofer Price. (2015). Summary of research generated by Striving Readers on the effectiveness of interventions for struggling adolescent readers (NCEE 2016-4001). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved here.
Cheung, A. C., & Slavin, R. E. (2012). How features of educational technology applications affect student reading outcomes: A meta-analysis. Educational Research Review, 7(3), 198-215.
Campuzano, L., Dynarski, M., Agodini, R., & Rall, K. (2009). Effectiveness of Reading and Mathematics Software Products: Findings From Two Student Cohorts. NCEE 2009-4041. National Center for Education Evaluation and Regional Assistance.
Dimitrov, D., Jurich, S., Frye, M., Lammert, J., Sayko, S., & Taylor, L. (2012). Year One Evaluation Report/Impact Study: Illinois Striving Readers. Arlington, VA: RMC Research Corporation.
Dynarski, M. (2007). Effectiveness of reading and mathematics software products findings from the first student cohort: report. DIANE Publishing.
Education Alliance at Brown University (2012). Springfield-Chicopee School Districts Striving Readers (SR) Program: Final Report Years 1-5: Evaluation of Implementation and Impact. Prepared for: Office of Elementary and Secondary Education, U.S. Department of Education Institute of Education
Goldman, S. R. (2012). Adolescent literacy: Learning and understanding content. The Future of Children, 22(2), 89-116.
Hasselbring, T. S., & Goin, L. I. (2004). Literacy instruction for older struggling readers: What is the role of technology? Reading & Writing Quarterly, 20(2), 123–144.
Herrera, S., Truckenmiller, A. J., and Foorman, B. R. (2016). Summary of 20 years of research on the effectiveness of adolescent literacy programs and practices (REL 2016–178). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast.
Herlihy, C. M., & Kemple, J. J. (2004). The Talent Development Middle School Model: Context, Components, and Initial Impacts on Students’ Performance and Attendance. MDRC.
Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2(3), 172-177.
Kim, J. S., Samson, J. F., Fitzgerald, R., & Hartry, A. (2010). A randomized experiment of a mixed-methods literacy intervention for struggling readers in grades 4–6: Effects on word reading efficiency, reading comprehension and vocabulary, and oral reading fluency. Reading and Writing, 23(9), 1109-1129.
Kim, J. S., Capotosto, L., Hartry, A., & Fitzgerald, R. (2011). Can a mixed-method literacy intervention improve the reading achievement of low-performing elementary school students in an after-school program? Results from a randomized controlled trial of READ 180 enterprise. Educational Evaluation and Policy Analysis, 33(2), 183-201.
Murnane, R., Sawhill, I., & Snow, C. (2012). Literacy challenges for the twenty-first century: Introducing the issue. The Future of Children, 22(2), 3-15.
O’Connor, R.E., Bell, K.M., Harty, K.R., Larkin, L.K., Sackor, S.M., & Zigmond, N. (2002). Teaching reading to poor readers in the intermediate grades: A comparison of text difficulty. Journal of Educational Psychology, 94(3), 474–485.
Schenck, A., Jurich, S., Frye, M., Lammert, J., & Sayko, S. (2012). Evaluation Report/Impact Study: Virginia Striving Readers Initiative (VSRII). Arlington, VA: RMC Research Corporation.
Shneyderman, A. (2006). Some results of the Voyager Passport Reading Intervention System in several school districts. Miami, FL: Miami-Dade County Public Schools Office of Evaluation and Research.
Slavin, R. E., Cheung, A., Groff, C., & Lake, C. (2008). Effective reading programs for middle and high schools: A best‐evidence synthesis. Reading Research Quarterly, 43(3), 290-322.
Slavin, R. E., Lake, C., Davis, S., & Madden, N. A. (2011). Effective programs for struggling readers: A best-evidence synthesis. Educational Research Review, 6(1), 1-26.
Stevens, R. J., & Durkin, S. (1992). Using student team reading and student team writing in middle schools: Two evaluations. Part I. Student team reading: A cooperative learning approach to middle school reading instruction (pp. 1–10). Baltimore, MD: Center for Research on Effective Schooling for Disadvantaged Students.
Stevens, R. J. (2003). Student team reading and writing: A cooperative learning approach to middle school literacy instruction. Educational Research and Evaluation, 9(2), 137–160.
Swanlund, A., Dahlke, K., Tucker, N., Kleidon, B., Kregor, J., Davidson-Gibbs, D., & Hallberg, K. (2012). Striving Readers: Impact Study and Project Evaluation Report: Wisconsin Department of Public Instruction (with Milwaukee Public Schools). Naperville, IL: American Institutes for Research.
Vaden-Kiernan, M., Caverly, S., Bell, N., Sullivan, K., Fong, C. & Atwood, E. (2012). Louisiana Striving Readers: Final Report. Austin TX: Southwest Educational Development Laboratory.
Vaughn, S., Swanson, E. A., Roberts, G., Wanzek, J., Stillman‐Spisak, S. J., Solis, M., & Simmons, D. (2013). Improving reading comprehension and social studies knowledge in middle school. Reading Research Quarterly,48(1), 77-93.
U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2009). Adolescent Literacy intervention report: Read 180. Retrieved here.
U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2011). Adolescent Literacy intervention report: Student team reading and writing. Retrieved here.
U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2013). Adolescent Literacy intervention report: Talent Development Middle Grades Program. Retrieved here
U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2015) National Assessment of Educational Progress (NAEP), various years, 1990–2015 Reading Assessments. Retrieved here
U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2016, November). Adolescent Literacy intervention report: READ 180®. Retrieved here
 Studies are weighted by sample size.
 A later large-scale randomized control trial found no statistically significant increases in reading scores in schools that used computer assisted instruction (Dynarski, 2007; Campuzano et al., 2009).
 The conversion into “days” of learning assumes a school year of 180 days.
 Note that a recent study, “Summary of 20 years of research on the effectiveness of adolescent literacy programs and practices” (Herrera et al., 2016), highlights 12 programs that demonstrate “positive or potentially positive” effects. We highlight several of these programs below, however their complete listing can be found here.
 Note that the sight word effect size is not statistically significant at a 10% level.