May 2, 2004
This is one in a series of policy briefs on key education issues prepared by the RAND Corporation for the Obama administration. A pay-for-performance system of educator compensation has been proposed to improve teacher effectiveness and thus improve student achievement. While there is some evidence that such systems work, there are numerous challenges to adopting them in the United States. RAND recommends policy actions and investments to increase the likelihood of success for pay-for-performance and identify the practices most likely to succeed.
Improving the quality of teaching in the nation's schools is critical to promoting high levels of student achievement and reducing racial/ethnic and socioeconomic performance gaps. One proposal to improve teacher effectiveness is a shift from a uniform salary schedule to a pay-for-performance (P4P) system, which would attach financial rewards to student achievement on standardized tests, possibly in combination with other data, such as graduation rates or measures of educators' practices.
With pilot programs underway in several districts, the debate over P4P continues. Advocates argue that the system will motivate educators, encourage those who are successful to remain in the profession, and attract more good candidates, while detractors worry that it will lead to negative effects on morale, collegiality, and student learning. Compared with other reforms, evidence about the effectiveness of P4P is limited. This policy brief summarizes what we know about P4P, what we need to know, and the implications of such information for education policy.
P4P programs have led to higher student achievement in some cases, but the research is limited. There are few studies of the effects of P4P on student achievement, and some of the best evidence comes from other countries and therefore may not apply to education in the United States. For example, studies conducted in Israel, Kenya, and India have found that teacher P4P has led to higher student test scores, but some of these studies also found negative outcomes, such as manipulation of test scores or "gaming" of the system. Overall, there is insufficient evidence to support claims that P4P will improve achievement in the United States.
Attaching stakes to test scores leads to changes in curricula and instruction. When tests are used to reward or penalize schools or individual educators, teachers and administrators tend to respond by focusing more on tested material and less on material not tested. These changes may be positive if educators spend more time on more-important topics, but the changes may also be undesirable if they take time away from valued subjects or result in excessive focus on a narrow set of test items.
The data needed to measure achievement gains or estimate added value are not always available. Most P4P programs evaluate teachers or principals using student test scores while controlling for prior achievement. Researchers are developing more and better methods for evaluating the value of P4P programs, but the measures work only with annual, consecutive-grade testing. Adopting P4P in early elementary grades, high school, and subjects other than reading and mathematics tends to be much more difficult because of a lack of such testing.
The contribution of a single teacher is difficult to measure. In most schools, multiple teachers and other staff interact with students and may influence their performance, which is one reason school-level incentives are sometimes preferred over individual teacher incentives. In addition, it is important to recognize that factors outside the school also influence student achievement.
Measures of teaching or leadership practices may lead to better professional development and increase educators' acceptance of the program. Many recently adopted P4P programs have combined student achievement data with evaluations of educators' practices. Findings from the limited studies available suggest that measuring teaching or leadership practices allows supervisors to provide useful feedback to teachers and principals, and it may increase educators' support for the program.
We know little about the effects of P4P on school climate and staff morale. Research suggests that teacher and principal support for a reform, as well as an environment of collaboration and trust, is important to its success. However, there is little evidence about how P4P affects the school environment, and the effects are likely to depend on such decisions as whether to provide incentives to individual staff or to the entire school.
We still lack evidence-based guidance for designing effective P4P programs. A number of choices must be made when adopting P4P. These include what subjects to include, how to address teachers who do not teach tested subjects, how (and whether) to reward principals in addition to teachers, and whether to reward groups (e.g., a school's entire staff) or individuals. There are also questions about how best to deal with students for whom existing tests may not provide valid information—e.g., English-language learners and students with disabilities.
Use high-quality achievement measures that are aligned with curricula. Because of the power of high-stakes tests to shape instruction, the measures in any high-stakes accountability system must be designed to assess the full range of knowledge and skills we want students to learn, and they must resist manipulation. The test content should not be predictable from one year to the next, and the tests should reflect a range of formats rather than relying exclusively on multiple-choice items. The measures should also include diagnostic tools to help educators evaluate their own strengths and weaknesses.
Supplement test scores with other measures of instruction or student outcomes. Using multiple measures reduces the likelihood that teachers or principals will focus narrowly on the content that is included in achievement tests. It also sends a clear message about the range of objectives the school or district values. In addition to test scores, P4P programs could include measures of teacher practices or knowledge and patterns of student course-taking, such as the percentage of students enrolled in college-preparatory or Advanced Placement courses.
Provide educators with resources to improve their practices. Districts and schools using P4P should provide their principals and teachers with resources—such as professional development, coaching, and collaboration tools—so that these educators can use the performance feedback to build on their strengths and address their weaknesses.
Invest in high-quality, longitudinal data systems that link students to their teachers. P4P for teachers requires linking student outcomes to the educators responsible for the instruction in the relevant subjects. Therefore, states or districts using P4P should implement identification systems that allow students to be followed over time, even as they transfer across schools or districts.
Encourage and fund innovation and evaluation. Because we know so little about designing effective P4P programs, the U.S. Department of Education should encourage and fund the development of a variety of models and build in rigorous evaluation to compare the options, such as individual versus group incentives. These initiatives should be accompanied by a national-level evaluation to gather consistent data across sites, and the findings should be widely disseminated to policymakers, practitioners, and researchers. In addition, policymakers should fund a long-term evaluation to understand how redesigned pay policies affect districts' ability to recruit and retain high-quality teachers and administrators.
Well-designed P4P programs may eventually help improve teacher and principal effectiveness and, in turn, promote higher levels of student achievement. However, because our knowledge about designing effective P4P programs is so limited, funding for further piloting and evaluation will be especially important. In addition, P4P must be combined with other reforms, including the implementation of high-quality standards and curriculum materials, well-designed pre-service training and professional development, and assistance to educators to promote continuous improvement in educational practices.