A number of challenges are encountered when evaluating a large-scale, multisite educational reform aimed at changing classroom practice. The challenges include substantial variability in implementation with little information on actual practice; lack of common, appropriate outcome measures; and the need to synthesize evaluation results across multiples study sites. This article describes an approach to addressing these challenges in the context of a study of the relationships between student achievement and instructional practices in the National Science Foundation’s Systematic Initiatives (SI) program. The authors gathered data from 11 SI sites and investigated relationships both at the site level and pooled across sites, using a planned meta-analytic approach. They found small but consistent positive relationships between teachers’ reported use of standards-based instruction and student achievement. The article also describes how the authors addressed the challenges discussed, as well as a number of additional obstacles that need to be addressed to improve future evaluations of large-scale reforms.
Originally published in: Educational Evaluation and Policy Analysis, v. 25, no. 1, Spring 2003, pp. 1-29.
This report is part of the RAND Corporation Reprint series. The Reprint was a product of the RAND Corporation from 1992 to 2011 that represented previously published journal articles, book chapters, and reports with the permission of the publisher. RAND reprints were formally reviewed in accordance with the publisher's editorial policy and compliant with RAND's rigorous quality assurance standards for quality and objectivity. For select current RAND journal articles, see External Publications.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.