Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.8 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief

Photo by Christopher Futcher/iStock

Many students lose knowledge and skills over the long summer break, and research suggests that low-income students fall further behind over the summer than their higher-income peers. Voluntary summer learning programs may provide an opportunity to stem summer learning loss and give struggling students additional learning opportunities.

The overarching question addressed in this research is whether voluntary, district-run summer programs that include academic and enrichment activities improve low-income students' outcomes.

The Wallace Foundation is funding a five-year demonstration project in five urban school districts in Boston, Dallas, Duval County (Florida), Pittsburgh, and Rochester (New York). These districts have been pioneers in offering full-day voluntary programs for five to six weeks free of charge to large numbers of struggling elementary students, not just those facing grade retention. The districts all provide at least three hours of academic instruction in math and reading by certified teachers, along with a range of enrichment activities (e.g., art, music, tennis, swimming), many of which are provided by community-based organizations that partner with the district. The districts vary in their approach to programming — for example, how they manage their sites, when in the summer they offer the program, and the specific curricula used in both academic and enrichment offerings.

Over 5,000 third-grade students applied to participate in the district programs and were randomly selected to participate in two summers of programming (summer 2013 and summer 2014). This study, commissioned by The Wallace Foundation, is the first randomized controlled trial to test the effectiveness of large-scale, district-run, voluntary summer learning programs. In a series of reports, RAND researchers will describe whether such programs benefit low-income elementary students and what program features are associated with positive outcomes. The first results, presented in Ready for Fall? Near-Term Effects of Voluntary Summer Learning Programs on Low-Income Students' Learning Opportunities and Outcomes, describe the effects of the 2013 summer programs in the autumn after the programs ended.

Early Outcomes

We found that there was strong demand for these programs among low-income children and their families, and that these programs appeared to provide opportunities that these children would not have had otherwise.

The programs had a significant positive effect on students' mathematics achievement when compared to students in the control group.

The programs had a significant positive effect on students' mathematics achievement. The average effect size across the five school districts was 11 percent of 1 standard deviation. This number reflects the spread in scores between students who participated in the program and those who did not, and not the growth in learning from the beginning of the summer to the end for either group. The effect is reasonably large for a five-to-six-week program. To set it in context, the average growth in mathematics achievement between the spring of third grade and the spring of fourth grade is about 52 percent of 1 standard deviation.

The researchers found no similar effect for reading skills: The difference was just 1 percent of 1 standard deviation, which was not statistically significant. This finding is somewhat surprising, given the improvements in math, but one explanation may be that it is more difficult to improve reading comprehension skills in a short program.

Students in the program also showed no difference in social-emotional competencies between the treatment group and the control group. Although some district leaders hypothesized that their programs might have a positive effect in this area, only one district explicitly designed a program with this outcome in mind, and while it did have a higher effect size than the other districts, the estimate was not statistically significant.

Five factors that may help improve outcomes

Another focus of the study is how to best implement programs of this kind. This non-experimental analysis examined a number of program features to see if they were related to positive student outcomes. Of seven factors examined, five had a statistically significant association with mathematics or reading outcomes.

Five factors that may help improve outcomes

Photo of boy by Christopher Futcher/iStock; photo of girl by DigitalVision/Thinkstock; icons by Anatoliy Babiy/iStock.


For mathematics, strong attendance and more hours of instruction were linked to better outcomes.

1 Consistent attendance

2 Hours of instruction


For reading, teacher grade-level experience, site orderliness, and instructional quality were associated with better outcomes.

3 Teachers with grade-level experience

4 Orderliness of summer sites (clear procedures for student behavior and discipline)

5 Instructional quality

Next Steps

The analysis described here, based on the second report from the study, is just one piece of a larger set of evidence that will emerge over the next two years. The next report will explore the effects of the 2013 summer learning programs on students' grades, attendance, and spring state test scores during the 2013–2014 school year. A fourth report will assess the outcomes of two years of voluntary summer programming. Finally, we will publish a fifth report focused on research-based strategies for designing and implementing summer programs. Together, these findings will enhance our understanding about how to design and implement summer learning programs, what kind of outcomes to expect from these programs, and whether district investment in these programs is cost-effective.

Building on what we learn: Schedule for release of RAND findings


  • Impact of one summer on school-year grades, attendance, and state tests


  • How to implement high-quality summer programs
  • Impact of two summers of district programs

This report is part of the RAND research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.