Evidence on the Validity, Reliability, and Usability of the Measuring and Improving Student-Centered Learning (MISCL) Toolkit
Jun 8, 2020
|PDF file||1.2 MB||
Use Adobe Acrobat Reader version 10 or higher for the best experience.
Photo by Jacob Lund/Adobe Stock
This research brief describes the contents of the MISCL Toolkit and how the RAND team developed and tested it for use in high schools.
Many high schools across the United States are striving to provide student-centered learning (SCL) by engaging students with a meaningful curriculum that supports students' individual needs and allows them to make some decisions about their learning. But how can schools know that they are reaching their SCL-related goals or whether all students in the school have access to the same SCL opportunities?
To help high schools and school systems answer such questions, RAND Corporation researchers developed the Measuring and Improving Student-Centered Learning (MISCL) Toolkit. The MISCL Toolkit is designed to help school practitioners and other school-support providers measure and improve the extent of SCL in high school classrooms. The MISCL Toolkit also helps users understand the degree to which SCL opportunities are equitably distributed across their schools. It is not intended to be used for evaluation or accreditation. Instead, it offers a one-stop shop through which school staff can easily collect, examine, and reflect on data from students, teachers, and district and school leaders about SCL.
The Toolkit defines student-centered learning as, "Students' deep engagement in learning opportunities that are designed to address their goals and interests while at the same time providing appropriate supports and challenge according to their learning needs."
The MISCL Toolkit allows users to measure SCL in their high schools according to five SCL strategies:
Following the MISCL User Guide, users can collect data about their high schools' engagement in SCL from six different data collection instruments:
Three key criteria guided the development, testing and revision of the MISCL Toolkit, which took place from 2017 to 2019.
The MISCL Toolkit is not an assessment or evaluation tool. The MISCL Toolkit was designed to collect formative data through a collaborative process. The data from the Toolkit can be used for professional learning, reflection, and informing school improvement strategies related to SCL. The validity, reliability, and usability evidence collected support these uses.
The Toolkit is intended for use in high schools. The survey questions are designed to capture elements of SCL implementation and SCL instructional approaches that most often occur in high schools. The Toolkit might not be appropriate for use in other grade levels. The student survey and focus group questions are specifically written at a level appropriate for high school students, and the questions might not be accessible to younger students.
Surveys are only one source of information and need to be interpreted carefully. Although the evidence gathered suggests that the MISCL Toolkit surveys provide good and useful information about SCL, the Toolkit survey questions might not always work as intended. During the process of testing the Toolkit, researchers gathered suggestive evidence that those completing the Toolkit surveys may have felt compelled to indicate that they were engaged in more SCL practices than they actually were using or might have rated their practices at lower levels if they were not sure what a survey question meant. To balance the surveys, the Toolkit includes other ways of collecting data about SCL, such as student focus groups and classroom walkthroughs.
More testing is needed. The RAND team was able to gather evidence that the surveys were reliable and valid by administering them to many different groups. Participants included school staff, leaders, and students who participated in pilot tests; teachers from a national sample; and district and school leaders across New England. This work provided preliminary evidence that the Toolkit can differentiate between schools with high SCL and those with low SCL. That said, the MISCL Toolkit as a whole was tested in only four high schools with relatively small numbers of students, teachers, and leaders. More testing in schools with a variety of different characteristics and experiences with SCL is needed to provide conclusive evidence on how well the MISCL Toolkit can measure whole-school SCL.
This report is part of the RAND Corporation Research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.