The Consistency of Composite Ratings of Teacher Effectiveness
Evidence From New Mexico
Published in: American Educational Research Journal, Volume 56, No. 6, pages 2116–2146 (December 2019). doi: 10.3102/0002831219841369
Posted on rand.org Feb 17, 2021
Contemporary teacher evaluation systems use multiple measures of performance to construct ratings of teacher quality. While the properties of constituent measures have been studied, little is known about whether composite ratings themselves are sufficiently reliable to support high-stakes decision making. We address this gap by estimating the consistency of composite ratings of teacher quality from New Mexico's teacher evaluation system from 2015 to 2016. We estimate that roughly 40% of teachers would receive a different composite rating if reevaluated in the same year; 97% of teachers would receive ratings within ±1 level of their original rating. We discuss mechanisms by which policymakers can improve rating consistency, and the implications of those changes to other properties of teacher evaluation systems.