Controlling for Changes in Test Conditions when Estimating Education Intervention Effects

Matthew D. Baird, John F. Pane

Published May 29, 2018

Education evaluations that analyze student achievement typically use baseline scores to control for unobserved ability. These analyses rely on the assumption that the conditions under which the test was administered are consistent from pretest to posttest. We caution evaluators to examine this assumption, provide a framework for evaluation under changes in test conditions, and demonstrate an application of this framework. We examine a case in which treatment status is correlated with the changes in test durations. We test three estimation strategies to account for this, each finding bias in estimation of treatment effects when not accounting for the changes in test conditions. We also discuss what to do in absence of such information.

Topics

Document Details

Citation

RAND Style Manual
Baird, Matthew D. and John F. Pane, Controlling for Changes in Test Conditions when Estimating Education Intervention Effects, RAND Corporation, WR-1245-BMGF, 2018. As of September 11, 2024: https://www.rand.org/pubs/working_papers/WR1245.html
Chicago Manual of Style
Baird, Matthew D. and John F. Pane, Controlling for Changes in Test Conditions when Estimating Education Intervention Effects. Santa Monica, CA: RAND Corporation, 2018. https://www.rand.org/pubs/working_papers/WR1245.html.
BibTeX RIS

This research was conducted by RAND Education.

This publication is part of the RAND working paper series. RAND working papers are intended to share researchers' latest findings and to solicit informal peer review. They have been approved for circulation by RAND but may not have been formally edited or peer reviewed.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.