First Outcomes from the National Summer Learning Study

Photo by Christopher Futcher/iStock

Many students lose knowledge and skills over the long summer break, and research suggests that low-income students fall further behind over the summer than their higher-income peers. Voluntary summer learning programs may provide an opportunity to stem summer learning loss and give struggling students additional learning opportunities.

The overarching question addressed in this research is whether voluntary, district-run summer programs that include academic and enrichment activities improve low-income students' outcomes.

The Wallace Foundation is funding a five-year demonstration project in five urban school districts in Boston, Dallas, Duval County (Florida), Pittsburgh, and Rochester (New York). These districts have been pioneers in offering full-day voluntary programs for five to six weeks free of charge to large numbers of struggling elementary students, not just those facing grade retention. The districts all provide at least three hours of academic instruction in math and reading by certified teachers, along with a range of enrichment activities (e.g., art, music, tennis, swimming), many of which are provided by community-based organizations that partner with the district. The districts vary in their approach to programming — for example, how they manage their sites, when in the summer they offer the program, and the specific curricula used in both academic and enrichment offerings.

Over 5,000 third-grade students applied to participate in the district programs and were randomly selected to participate in two summers of programming (summer 2013 and summer 2014). This study, commissioned by The Wallace Foundation, is the first randomized controlled trial to test the effectiveness of large-scale, district-run, voluntary summer learning programs. In a series of reports, RAND researchers will describe whether such programs benefit low-income elementary students and what program features are associated with positive outcomes. The first results, presented in Ready for Fall? Near-Term Effects of Voluntary Summer Learning Programs on Low-Income Students' Learning Opportunities and Outcomes, describe the effects of the 2013 summer programs in the autumn after the programs ended.

Early Outcomes

We found that there was strong demand for these programs among low-income children and their families, and that these programs appeared to provide opportunities that these children would not have had otherwise.

The programs had a significant positive effect on students' mathematics achievement. The average effect size across the five school districts was 11 percent of 1 standard deviation. This number reflects the spread in scores between students who participated in the program and those who did not, and not the growth in learning from the beginning of the summer to the end for either group. The effect is reasonably large for a five-to-six-week program. To set it in context, the average growth in mathematics achievement between the spring of third grade and the spring of fourth grade is about 52 percent of 1 standard deviation.

The researchers found no similar effect for reading skills: The difference was just 1 percent of 1 standard deviation, which was not statistically significant. This finding is somewhat surprising, given the improvements in math, but one explanation may be that it is more difficult to improve reading comprehension skills in a short program.

Students in the program also showed no difference in social-emotional competencies between the treatment group and the control group. Although some district leaders hypothesized that their programs might have a positive effect in this area, only one district explicitly designed a program with this outcome in mind, and while it did have a higher effect size than the other districts, the estimate was not statistically significant.

Five factors that may help improve outcomes

Another focus of the study is how to best implement programs of this kind. This non-experimental analysis examined a number of program features to see if they were related to positive student outcomes. Of seven factors examined, five had a statistically significant association with mathematics or reading outcomes.

Photo of boy by Christopher Futcher/iStock; photo of girl by DigitalVision/Thinkstock; icons by Anatoliy Babiy/iStock.

Next Steps

The analysis described here, based on the second report from the study, is just one piece of a larger set of evidence that will emerge over the next two years. The next report will explore the effects of the 2013 summer learning programs on students' grades, attendance, and spring state test scores during the 2013–2014 school year. A fourth report will assess the outcomes of two years of voluntary summer programming. Finally, we will publish a fifth report focused on research-based strategies for designing and implementing summer programs. Together, these findings will enhance our understanding about how to design and implement summer learning programs, what kind of outcomes to expect from these programs, and whether district investment in these programs is cost-effective.

Available for Download

Download PDF

Topics

• Academic Achievement
• After-School Programs
• Education Administration
• Educational Program Evaluation
• Summer Learning
• Underserved Students