To recognize and reward teachers for contributing to students' learning, states and districts are retooling their teacher evaluation systems to incorporate measures of student performance. These efforts are aided by improved access to longitudinal data systems that link teachers to students and by improved statistical models for associating teachers with their students' test score gains. But at least two important challenges remain: generating valid estimates of teachers' contributions to student learning and including teachers who do not teach subjects or grades that are tested annually.
Given these challenges, a RAND Corporation study analyzed the systems of three districts (Denver, Colorado; Hillsborough County, Florida; and Washington, D.C.) and two states (Tennessee and Delaware) that have begun or are planning to incorporate measures of student performance into their teacher evaluations. The study focused on what is and is not known about the quality of various student performance measures that the systems are using and how the systems are supplementing existing measures—such as scores from state accountability tests—with other student performance measures and other information about teachers' effectiveness.
The study found that policymakers should demand adequate score reliability (freedom from errors of measurement) and also collect and review evidence about the validity of inferences drawn from value-added estimates (the accuracy and appropriateness of score interpretations). Policymakers may also wish to select student assessments that are vertically scaled so that scores fall on a comparable scale from year to year.
As for additional student performance measures that states or districts might use, the study found that locally developed assessments can be well aligned with local curricula, but such assessments need to be developed, administered, and scored in ways that are comparable from teacher to teacher and from year to year. Commercially produced interim assessments are more comparable across classrooms and over time, but, like locally developed assessments, they are often not designed for use in high-stakes contexts. Using aggregate student performance measures, such as school- or department-wide test scores, to evaluate teachers in nontested subjects or grades allows school systems to rely on existing measures but may create a two-tiered system in which some teachers are evaluated differently from others. For the most part, none of these measures has been validated for use in teacher evaluations, which may carry high stakes for teachers in terms of future course assignments, compensation, employment, and so forth. Policymakers must also consider how teachers will be held accountable for students who receive instruction from multiple teachers in the same subject in a given year.
When incorporating student achievement measures into teacher evaluation systems policymakers should (1) create comprehensive evaluation systems that incorporate multiple measures of teacher effectiveness; (2) choose measures for which there is strong evidence of reliability and validity, and attend to how the assessments are being used in high-stakes contexts; (3) if teachers are allowed to choose supplemental performance measures, set clear parameters to promote consistency among classrooms; (4) use multiple years of student achievement data in value-added estimation and, where possible, average teachers' value-added estimates across multiple years; and (5) find ways to hold teachers accountable for students they teach who are not included in value-added estimates.
READ THE RESEARCH BRIEF:
RAND Education conducts research on a wide range of topics in education. Our mission is to bring accurate data and careful, objective analysis to the national debate on education policy. To access all of our research, please visit our website at www.rand.org/education.html.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND focuses on the issues that matter most such as health, education, national security, international affairs, law and business, the environment, and more. As a nonpartisan organization, RAND operates independent of political and commercial pressures. We serve the public interest by helping lawmakers reach informed decisions on the nation's pressing challenges. Learn More »
Copyright © 2011 RAND Corporation. 1776 Main Street, Santa Monica, CA 90401-3208. RAND® is a registered trademark.