Using Structured Classroom Vignettes to Measure Instructional Practices in Mathematics

Published in: Educational Evaluation and Policy Analysis, v. 28, no. 2, Summer 2006, p. 101-130

Posted on RAND.org on January 01, 2006

by Brian M. Stecher, Vi-Nhuan Le, Laura S. Hamilton, Gery W. Ryan, Abby Robyn, J. R. Lockwood

Large-scale educational studies frequently require accurate descriptions of classroom practices to judge implementation and impact. However, such descriptions are difficult to obtain in a timely, efficient manner. To address this problem, we developed a vignette-based measure of one aspect of mathematics instructional practice, reform-oriented instruction. Teachers read contextualized descriptions of teaching practices that varied in terms of reform-oriented instruction and rated the degree to which the options corresponded to their own likely behaviors. Responses from 80 fourth-grade teachers yielded fairly consistent responses across two parallel vignettes and moderate correlations with other scales of reform-oriented instruction derived from classroom observations, surveys and logs. The results suggested that the vignettes are measuring important aspects of reform-oriented instruction that are not captured by other measurement methods. Based on this work, it appears that vignettes can be a useful tool for research on instructional practice, but cognitive interviews with participating teachers provided insight into possible improvements to the items.

This report is part of the RAND Corporation external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.