Using Artifacts to Describe Instruction
Lessons Learned from Studying Reform-Oriented Instruction in Middle School Mathematics and Science
Published in: CSE Report, no. 705 (Los Angeles, CA : CRESST, National Center for Research on Evaluation, Standards, and Student Testing, UCLA Center for the Study of Evaluation, Jan. 2007), 137 p
Posted on RAND.org on January 01, 2007
It is important to be able to describe instructional practices accurately in order to support research on "what works" in education and professional development as a basis for efforts to improve practice. This report describes a project to develop procedures for characterizing classroom practices in mathematics and science on the basis of collected classroom artifacts. A data collection tool called the "Scoop Notebook" was used to gather classroom artifacts (e.g., lesson plans, instructional materials, student work) and teacher reflections. Scoring guides were developed for rating the Notebooks (and observed classroom behaviors) along ten dimensions of reform-oriented practice in mathematics and science. Field studies were conducted in middle school science and mathematics classrooms to collect information about the reliability, validity, and feasibility of the Scoop Notebook as a measure of classroom practice. The studies yielded positive results, indicating that the Scoop Notebooks and associated scoring guides have promise for providing accurate representations of 1 Many people contributed to the success of this project. Although they are unnamed in this report, we would like to acknowledge the importance of the 96 middle school mathematics and science teachers who assembled Scoop Notebooks as part of our pilot- and field-studies. We asked more of them than is typically required in educational research, and they responded thoughtfully and graciously. Many researchers were part of our team during the five years of the project. In addition to the named authors on this report, the following people contributed directly to the research effort: Alicia Alonzo, Daniel Battey, Victoria Deneroff, Elizabeth Dorman, Sherrie McClam, Shannon Moncure, Joi Spencer, and Alice Wood. Linda Daly from RAND produced the formatted Scoop Notebook pages, sticky notes and other project materials. We are also grateful for the patient and reliable assistance provided by Donna White and Lisa Loranger. Finally, we wish to acknowledge the support and encouragement of Eva Baker and Joan Herman, who were more than just CRESST administrators, but were interested and provocative colleagues.