Quality Start Los Angeles (QSLA) is a county-level quality rating and improvement system (QRIS) that supports center-based and family child care early learning providers serving children from birth to age five. During a developmental evaluation of QSLA, the authors of this report identified strengths, challenges, and lessons learned that could benefit other QRISs as they create or expand data systems.

Cover: Data Use in Quality Rating and Improvement Systems
Read Online

Data Use in Quality Rating and Improvement Systems

Lessons Learned from Quality Start Los Angeles

by Jill S. Cannon, Celia J. Gomez, Anamarie A. Whitaker

Download Free Electronic Document

FormatFile SizeNotes
PDF file 2.5 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. What did an independent developmental evaluation of QSLA reveal about key challenges in data collection and usage?
  2. What are the lessons learned for other QRISs as they create or expand data systems?

Quality Start Los Angeles (QSLA) is a county-level quality rating and improvement system (QRIS) that supports center-based and family child care providers serving children from birth to age five. During a developmental evaluation of QSLA that focused on program coaching, assessment technical assistance, and quality tier rating perceptions, the authors of this report also examined data use at QSLA. In this report, they identified strengths, challenges, and lessons learned that could benefit other QRISs or early learning systems as such organizations create or expand data systems, noting that, even with good intentions and a strong data system infrastructure, data use might not always be implemented as expected.

Key Findings

  • Variable definitions and entry protocols sometimes differed across organizations, limiting the ability to use existing data to fully address several intended research questions.
  • When data are not easy to manipulate or export for staff or evaluator use, analyses are less likely to happen in a timely manner.
  • Most QRIS models evolve and change as new providers are added, new funds become available for service expansion, or program evaluation leads to improvements in the design. But decisionmakers are often interested in comparing implementation measures or program outcomes over time, which can present a challenge if some time periods had different data-entry requirements.
  • The existence of multiple data sources—from multiple partners and changes over time—can make ongoing evaluation more time-intensive and, in some cases, depending on data quality across the sources and linking variables, questions might not be answerable. Creating processes to link distinct data sources collected by different organizations enhances the potential for data use.
  • To know whether quality improvement models are improving provider quality, data systems must be able to link model components—such as coaching or technical assistance—to such outcomes as tier ratings or classroom assessment scores. This linkage requires advance preparation to identify the key goals for data use and research questions of interest and to ensure that the data system is built to address them.

Recommendations

  • Establish common data definitions and entry protocols for all staff who enter data.
  • Make data usable for ongoing monitoring and evaluation.
  • Ensure data systems can adapt to partnership or QRIS model changes.
  • When multiple administrative data sources are necessary, ensure that sources can be easily linked.
  • Ensure data systems can connect quality improvement supports to outcomes.

Research conducted by

This study was commissioned by First 5 Los Angeles and undertaken by RAND Education and Labor.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.