An Apple a Day

How New York City Schools Turn Grade Retention into Good Medicine

By Jennifer Sloan McCombs, Sheila Nataraj Kirby, and Louis T. Mariano

Jennifer Sloan McCombs is a RAND policy researcher specializing in education reform. Sheila Nataraj Kirby is a RAND senior economist, and Louis Mariano is a RAND statistician.

New York City Mayor Michael Bloomberg announced in August that he would extend the grade retention policy of the city’s schools to 4th and 6th grades, meaning that all New York City students from 3rd through 8th grades now face being held back if they fail to meet promotion criteria on standardized tests or to demonstrate their knowledge by alternative means. The New York policy of ending social promotion is one of many such policies now in effect in several U.S. cities, from Houston to Philadelphia to Long Beach.

Maria Baez, 11, sways as she sings with the Public School 22 Chorus of 5th graders during a building dedication at the Staten Island University Hospital.
AP IMAGES/BEBETO MATTHEWS
Maria Baez, 11, far right, sways as she sings with the Public School 22 Chorus of 5th graders during a building dedication at the Staten Island University Hospital in New York City on June 26, 2009. The chorus has become a cyber-phenomenon.

Grade retention is the practice of keeping low-achieving students at the same grade level for an additional year to give them extra time to catch up, as opposed to social promotion, the practice of promoting students regardless of whether they have mastered the grade content. As part of an emphasis on standards and accountability, many U.S. districts use standardized test scores as one of the main criteria for grade retention.

However, grade retention is a contentious and hotly debated policy. Proponents of grade retention argue that it gives students an additional year to master the academic content that they failed to master the previous year, putting the students at less risk for failure when they do proceed to the next grade. Opponents of grade retention point to research that shows it disproportionately affects low-income and minority children and is associated with low self-esteem, problem behaviors, and an increased risk of dropping out of school.

Indeed, our review of 91 studies of retention policies showed that, in general, retention does not appear to benefit students academically. In fact, most of the studies indicate negative relationships between retention and subsequent academic achievement. On the other hand, a few studies have found academic improvement in the immediate years after retention. But even so, these gains are often short-lived and tend to fade over time. Moreover, the retained students are shown to have a significantly increased risk of eventually dropping out of school.

The policy used by New York has served as both a threat and a promise.

But the policy used by the New York City Department of Education, the largest school district in the United States, has served as both a threat and a promise. In New York, the policy aligns with best practices set out by the National Research Council, placing emphasis on identifying struggling students early in the school year, providing them with additional instructional time (tutoring, Saturday school, summer school), and monitoring their progress. The emphasis on early identification of students who are struggling academically and on providing them extra learning opportunities is a critical factor in what we found to be the success of the city’s policy.

We examined the implementation of the New York policy and its short-term effects on the academic and socioemotional outcomes of students who were in the 5th grade between 2004 and 2007. By tracking the performance of 240,000 students, surveying more than 7,000 students in each of three years, surveying school administrators, conducting case studies of schools, interviewing district officials, and analyzing demographic and achievement data, we found that the policy and supportive services had positive effects on student performance in the 5th grade that continued into the 7th grade.

Few students were retained under the policy. However, for those students who were retained, we found the additional year in the same grade improved student performance into the 7th grade, compared with what it would have been without the policy, and that retention did not result in negative effects on sense of school belonging or academic confidence.

Our top recommendation for New York is to continue identifying at-risk students early and providing them with academic support services. This approach sets New York apart.

Our top recommendation for New York is to continue identifying at-risk students early and providing them with academic support services. This approach sets New York apart from those districts that use test results to make promotion decisions without providing early educational support services to help students improve.

But two questions remain for New York as well as other districts: whether the short-term positive effects of a retention policy like that in New York will persist over the longer term and whether the benefits will outweigh the costs. New York’s policy has not been in existence long enough to address these questions. We have followed the retained students only to grade 7.

How the Policy Took Effect

New York implemented its policy for students in grade 3 during the 2003–2004 school year, extending it to grade 5 in 2004–2005, grade 7 in 2005–2006, and grade 8 in 2008–2009. As a result, general education students in these grades must score at or above Level 2 on a four-level performance scale on the state English language arts and mathematics assessments in order to be promoted to the next grade. Performance at or above Level 3 is considered “proficient” under the federal No Child Left Behind Act and is a higher standard than the city’s promotion benchmark.

Students in need of extra learning opportunities are identified early based on the previous year’s assessments, teacher recommendations, or being previously retained in grade. These students receive academic intervention services, such as tutoring, during the school day. Schools can also offer services outside the school day, such as Saturday school, throughout the academic year. Students who do not meet promotion standards in the spring are enrolled in summer school.

Students who fail to pass the spring assessments, in either English or math, are given multiple opportunities to meet the promotion criteria. The students can be promoted based on (1) a review of a portfolio of their work in the spring, (2) performance on the summer standardized assessment, (3) a review of a portfolio of their work in August, or (4) an appeal process.

Many Students Needed Services During 5th Grade, but Few Were Retained

Many Students Needed Services During 5th Grade, but Few Were Retained
SOURCE: Ending Social Promotion Without Leaving Children Behind, 2009.

In each of the three academic years between 2004 and 2007, about 20 percent of students were categorized as needing assistance upon entering the 5th grade. However, as shown in the figure, few students were retained under the policy. The percentage of retained students dropped over time, from 2–3 percent in the first two years to 1 percent in the third year (621 students out of 57,762 students). More of the students needed assistance in English language arts than in mathematics, but students were more likely to be retained for failing to meet promotion criteria in mathematics.

Schools seemed to provide academic intervention services to as many students as they had the capacity to serve. Schools with larger percentages of in-need students were less likely to serve all students needing services, compared with other schools, even though they were significantly more likely than low-need schools to have additional resources for students, such as mathematics specialists, reading coaches, and mathematics coaches.

Across the three survey years (2006–2008), almost all schools reported serving students in need through small-group tutoring during the school day. Fewer schools (64–80 percent) were able to offer one-on-one tutoring during the school day. This is unfortunate, because, similar to findings from other studies, our analyses found that one-on-one tutoring was associated with improved student achievement. We also found that services were not offered consistently, because instructors were pulled away for other duties. So, for instance, students who should have received services three times a week may have typically received services only twice a week.

How the Policy Has Helped

The policy had positive effects on the achievement of students needing services during the 5th grade, raising their assessment scores in both English language arts and mathematics by small amounts over their expected performance, judging by a control group of comparable students. The positive effects were stronger in English language arts than in mathematics. In contrast, more frequent attendance at Saturday classes and summer school in particular was associated with greater improvements in mathematics.

The positive effects continued into the 6th and 7th grades. The benefits included small, positive effects from early identification and intervention; small, positive effects from summer school; and moderate, positive effects from an additional year of instruction due to retention.

In other words, students who needed services at the beginning of the 5th grade scored higher on the 7th-grade assessments than they would have scored without the policy. Students who were required to attend summer school after 5th grade scored higher on English language arts and mathematics assessments in the 6th and 7th grades. And the positive effects of retention on student performance were larger than those found for early support services alone. The positive effects into the 6th and 7th grades were again generally larger in English language arts than in mathematics.

For the small group of students entering 5th grade with the lowest scores on state assessments, the additional services during the school year had little effect on performance relative to how they would have done in the absence of the policy. It may be that we find no promotion policy effects for these students because the services they received under the policy were quite similar to those provided prior to the policy.

Retained students reported a greater sense of belonging than did promoted students.

Schoolchildren listen to a reading of Winnie-the-Pooh.
AP IMAGES/MARK LENNIHAN
Schoolchildren listen to a reading of Winnie-the-Pooh at a new Barnes & Noble on June 19, 2009, in New York City.

The retained students did not report negative socioemotional effects. According to student surveys, retained students did not differ from promoted peers with respect to their sense of school belonging or their confidence in mathematics and reading. In fact, one consistent finding concerned school belonging: Retained students reported a greater sense of belonging than did promoted students, even three years after having been retained. While this is counterintuitive, it is consistent with some prior studies.

School staff tended to be positive about the policy. In surveys and interviews, the majority of principals and teachers agreed that the policy helped focus their instructional efforts and made parents more aware of their children’s progress. However, the majority thought that the policy relied too heavily on state assessment scores and, interestingly, that the policy made it more difficult to retain students who passed the test but who would benefit from being retained.

How the Policy Can Be Enhanced

Our findings lead to several recommendations for school administrators. While targeted to New York City, these recommendations could also apply to other districts and states implementing or considering test-based promotion policies.

Continue early identification of students and the provision of academic intervention services. These components of the policy helped students meet the promotion standards and improved their achievement in future grades. One-on-one tutoring should be expanded when possible.

Enable academic service providers to work consistently with students who need services. The provision of services was inconsistent because providers were often asked to assume other responsibilities, such as substitute teaching, lunch duty, or proctoring assessments.

One-on-one tutoring should be expanded when possible.

Consider the expected duration and participation when designing Saturday programs. Improved performance in mathematics was associated with attending at least seven Saturday sessions. Principals who offer the programs need to ensure frequent attendance to maximize the benefits.

Continue to encourage struggling students to attend summer school. Summer school attendance had a positive relationship with summer assessment outcomes, especially in mathematics, and the positive effects of summer instruction continued into grades 6 and 7.

Analyze student-level data to evaluate the effectiveness of interventions. Several instructional strategies hold promise for helping struggling students. Schools have considerable autonomy over the services provided and would benefit from analyses of the effects of specific instructional strategies and programs on student achievement.

Continue to monitor the longer-term effects of retention. Two of the most important questions are whether the short-term positive effects of the policy persist over the longer term and whether the policy is cost-effective, compared with alternatives. We could not answer these questions.

The effectiveness of the promotion policy will ultimately be judged by whether its benefits outweigh its costs in the long term. This question has two components. First, from an individual student’s point of view, the question is whether the short-term gains will persist into high school and result in an improved probability of graduation and in higher proficiency at graduation. Second, from a societal point of view, the question is whether the economic and other benefits associated with implementing a promotion policy and holding students back a year outweigh the costs in terms of financing an extra year of education and any delay in entering the workforce. Although the New York City policy has not been in place long enough to address these larger questions, the near-term benefits at least hold the possibility of longer-term benefits as well. square

Related Resources

Ending Social Promotion: Examining the Effects of NYC’s 5th-Grade Promotion Policy, Jennifer McCombs, Lou Mariano, 2009.
Ending Social Promotion Without Leaving Children Behind: The Case of New York City, Jennifer Sloan McCombs, Sheila Nataraj Kirby, Louis T. Mariano, eds., RAND/MG-894-NYCDOE, 2009, 308 pp., ISBN 978-0-8330-4778-6.
Retaining Students in Grade: Lessons Learned Regarding Policy Design and Implementation, Julie A. Marsh, Daniel Gershwin, Sheila Nataraj Kirby, Nailing Xia, RAND/TR-677-NYCDOE, 2009, 111 pp.
Retaining Students in Grade: A Literature Review of the Effects of Retention on Students’ Academic and Nonacademic Outcomes, Nailing Xia, Sheila Nataraj Kirby, RAND/TR-678-NYCDOE, 2009, 142 pp.