Large-scale educational studies frequently require accurate descriptions of classroom practices to judge implementation and impact. However, such descriptions are difficult to obtain in a timely, efficient manner. To address this problem, we developed a vignette-based measure of one aspect of mathematics instructional practice, reform-oriented instruction. Teachers read contextualized descriptions of teaching practices that varied in terms of reform-oriented instruction and rated the degree to which the options corresponded to their own likely behaviors. Responses from 80 fourth-grade teachers yielded fairly consistent responses across two parallel vignettes and moderate correlations with other scales of reform-oriented instruction derived from classroom observations, surveys and logs. The results suggested that the vignettes are measuring important aspects of reform-oriented instruction that are not captured by other measurement methods. Based on this work, it appears that vignettes can be a useful tool for research on instructional practice, but cognitive interviews with participating teachers provided insight into possible improvements to the items.
Stecher, Brian M., Vi-Nhuan Le, Laura S. Hamilton, Gery W. Ryan, Abby Robyn, and J. R. Lockwood, Using Structured Classroom Vignettes to Measure Instructional Practices in Mathematics. Santa Monica, CA: RAND Corporation, 2006. https://www.rand.org/pubs/working_papers/WR336.html.
Stecher, Brian M., Vi-Nhuan Le, Laura S. Hamilton, Gery W. Ryan, Abby Robyn, and J. R. Lockwood, Using Structured Classroom Vignettes to Measure Instructional Practices in Mathematics, Santa Monica, Calif.: RAND Corporation, WR-336-NSF, 2006. As of June 15, 2021: https://www.rand.org/pubs/working_papers/WR336.html