RAND Review News for Winter 2011-2012

Pay-for-Performance Fails to Improve NYC Student Achievement

A New York City program to boost student performance by offering bonuses to schools that met annual schoolwide performance targets failed to improve student achievement at any grade level or to affect school progress reports, researchers from RAND and Vanderbilt University have found. As a result of the research, published in July, the city has discontinued the program.

Overall, the researchers found no statistically significant difference in average mathematics and English language test scores between elementary school, middle school, and K–8 students in schools randomly selected to participate in the program and students in similar schools not selected (the control schools). The researchers also found no statistically significant differences between the overall scores of schools assigned to the program and the overall scores of the control schools. And there were no differences between the reported practices, effort, and attitudes of teachers in schools selected for participation and those in the control group.

The New York City Department of Education and the United Federation of Teachers jointly implemented the Schoolwide Performance Bonus Program for the first time in the 2007–08 school year. Schools meeting their targets received bonuses equal to $3,000 for each full-time, union-represented staff member (teachers, counselors, secretaries); at each school, a four-person committee determined how to distribute the bonus among staff. About 200 schools participated in the program over a three-year period. The researchers used test scores, surveys, and interviews to assess the performance of the program.

“Bonuses alone have not proven to be the answer to bettering student achievement,” said Julie Marsh, an adjunct researcher at RAND and visiting associate professor at the University of Southern California, who led the study. “Some educators didn’t understand how the program worked, while others did not perceive the bonus as having tremendous value. Still others felt the bonus criteria relied too heavily on test scores. Many also reported that other factors, such as accountability pressures and intrinsic motivators, were more salient than financial bonuses.” The figure shows examples of these other motivators.

Teachers Report That Factors Other Than Bonuses Motivated Them to Work Hard

Teachers Report That Factors Other Than Bonuses Motivated Them to Work Hard

SOURCE: A Big Apple for Educators, 2011.

The researchers suggested that the lack of improvement might also have resulted from a lack of capacity or resources, such as school leadership, expertise, instructional materials, or time to bring about improvement. It is also possible, as suggested by a 2010 RAND and Vanderbilt University study of a pay-for-performance program for teachers in Nashville, that the theory underlying such programs may be flawed. Even though teachers in New York reported that the bonus was desirable and motivating, they also reported not changing their teaching practices in response to the program. “Thus, a desirable award might not be enough to change behavior,” said Marsh. square

For more information:

A Big Apple for Educators: New York City’s Experiment with Schoolwide Performance Bonuses: Final Evaluation Report, RAND/MG-1114-FPS, 2011.