In 2013, the National Research Council identified 14 indicators for tracking the nation's progress toward improving science, technology, engineering, and mathematics (STEM) education in the United States. This report explores approaches to measuring indicator 5, classroom coverage of content and practices in the Common Core State Standards for mathematics and the Next Generation Science Standards.

Creating an Indicator of K–12 Classroom Coverage of Science, Technology, Engineering, and Math (STEM) Content and Practices

Laura S. Hamilton, Brian M. Stecher, Kun Yuan

ResearchPublished Mar 28, 2017

In 2013, the National Research Council identified 14 indicators for tracking the nation's progress toward improving science, technology, engineering, and mathematics (STEM) education in the United States. This report focuses on indicator 5, classroom coverage of content and practices in the Common Core State Standards for mathematics and the Next Generation Science Standards.

One of the primary limitations of most existing methods is their focus on what the teacher does rather than what students experience. This focus on the teacher can be particularly limiting in classrooms in which different students are engaged in different learning activities simultaneously. These within-classroom differences can result from efforts to differentiate instruction to meet the needs of individual students and are likely to be especially prevalent in classrooms that rely on technology-based, personalized-learning approaches. This report describes the rationale for examining new approaches to measuring students' exposure to standard-aligned content and practices, summarizes what is known about currently available measures, and explores innovative approaches that might be adopted to create new measures.

Key Findings

Surveys Appear to Be the Most Plausible Method for Measuring Exposure to STEM Content and Practices Across the United States at This Time

  • Surveys are relatively inexpensive and can be deployed fairly easily for large-scale data collection.

Technology-Based Learning Systems, Particularly Simulations, Have Potential to Support Future STEM Indicator Measurement Efforts

  • New applications of instructional technology offer the possibility of novel data-collection approaches that could gather detailed information on what students are doing and how much time they spend doing it.

The Use of Technology-Based Learning Systems for Large-Scale Measurement Is Likely to Be Limited Because of Variability Across Schools in the Computer-Based Tools and Other Instructional Materials That Are Adopted and Used

  • One major factor that will hinder large-scale adoption is the substantial diversity of software products used across schools, along with inconsistency in how data are collected and how the software reports them.

STEM Practices, Such as Those Identified in the Common Core State Standards for Math and the Next Generation Science Standards, Are Enacted Differently in Different Content Areas

  • For a large-scale indicator system, measures of these practices are likely to require sampling across classrooms or schools to capture different content areas without overburdening respondents.

Some Opportunities to Engage in STEM Content and Practices Occur Outside of Traditional Courses

  • Out-of-school opportunities to engage in STEM learning are common, particularly in such areas as robotics.

Recommendations

  • Create a working group to inform indicator development.
  • Use multiple measures to collect evidence related to indicator 5.
  • Begin by building on existing data-collection tools and systems.
  • Design the measures to support longitudinal comparisons.
  • Consider incorporating measures of student knowledge into the broader indicator system.
  • Avoid attaching stakes to the measures.
  • Continue to conduct research on STEM teaching and learning to inform future indicator efforts.

Topics

Document Details

Citation

RAND Style Manual
Hamilton, Laura S., Brian M. Stecher, and Kun Yuan, Creating an Indicator of K–12 Classroom Coverage of Science, Technology, Engineering, and Math (STEM) Content and Practices, RAND Corporation, RR-1913-NSF, 2017. As of October 10, 2024: https://www.rand.org/pubs/research_reports/RR1913.html
Chicago Manual of Style
Hamilton, Laura S., Brian M. Stecher, and Kun Yuan, Creating an Indicator of K–12 Classroom Coverage of Science, Technology, Engineering, and Math (STEM) Content and Practices. Santa Monica, CA: RAND Corporation, 2017. https://www.rand.org/pubs/research_reports/RR1913.html.
BibTeX RIS

The research reported here was funded by the National Science Foundation and conducted by RAND Education.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.