Creating an Indicator of K–12 Classroom Coverage of Science, Technology, Engineering, and Math (STEM) Content and Practices

by Laura S. Hamilton, Brian M. Stecher, Kun Yuan

Download eBook for Free

FormatFile SizeNotes
PDF file 0.7 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. How can exposure to science, technology, engineering, and math (STEM) content and practices be measured?
  2. How can existing measures of instructional practice be used to measure students' exposure to STEM content and practices?
  3. What new technology-based approaches could be adapted to create measures of STEM content and practices?

In 2013, the National Research Council identified 14 indicators for tracking the nation's progress toward improving science, technology, engineering, and mathematics (STEM) education in the United States. This report focuses on indicator 5, classroom coverage of content and practices in the Common Core State Standards for mathematics and the Next Generation Science Standards.

One of the primary limitations of most existing methods is their focus on what the teacher does rather than what students experience. This focus on the teacher can be particularly limiting in classrooms in which different students are engaged in different learning activities simultaneously. These within-classroom differences can result from efforts to differentiate instruction to meet the needs of individual students and are likely to be especially prevalent in classrooms that rely on technology-based, personalized-learning approaches. This report describes the rationale for examining new approaches to measuring students' exposure to standard-aligned content and practices, summarizes what is known about currently available measures, and explores innovative approaches that might be adopted to create new measures.

Key Findings

Surveys Appear to Be the Most Plausible Method for Measuring Exposure to STEM Content and Practices Across the United States at This Time

  • Surveys are relatively inexpensive and can be deployed fairly easily for large-scale data collection.

Technology-Based Learning Systems, Particularly Simulations, Have Potential to Support Future STEM Indicator Measurement Efforts

  • New applications of instructional technology offer the possibility of novel data-collection approaches that could gather detailed information on what students are doing and how much time they spend doing it.

The Use of Technology-Based Learning Systems for Large-Scale Measurement Is Likely to Be Limited Because of Variability Across Schools in the Computer-Based Tools and Other Instructional Materials That Are Adopted and Used

  • One major factor that will hinder large-scale adoption is the substantial diversity of software products used across schools, along with inconsistency in how data are collected and how the software reports them.

STEM Practices, Such as Those Identified in the Common Core State Standards for Math and the Next Generation Science Standards, Are Enacted Differently in Different Content Areas

  • For a large-scale indicator system, measures of these practices are likely to require sampling across classrooms or schools to capture different content areas without overburdening respondents.

Some Opportunities to Engage in STEM Content and Practices Occur Outside of Traditional Courses

  • Out-of-school opportunities to engage in STEM learning are common, particularly in such areas as robotics.

Recommendations

  • Create a working group to inform indicator development.
  • Use multiple measures to collect evidence related to indicator 5.
  • Begin by building on existing data-collection tools and systems.
  • Design the measures to support longitudinal comparisons.
  • Consider incorporating measures of student knowledge into the broader indicator system.
  • Avoid attaching stakes to the measures.
  • Continue to conduct research on STEM teaching and learning to inform future indicator efforts.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Existing Measures of STEM Instruction

  • Chapter Three

    Promising Digital Data-Collection and Analysis Methods

  • Chapter Four

    Conclusions and Recommendations

Research conducted by

The research reported here was funded by the National Science Foundation and conducted by RAND Education.

This report is part of the RAND Corporation research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.