As a first step in a larger study of Delaware Stars, a voluntary quality rating and improvement system (QRIS) for early learning and care programs, RAND researchers examined prior QRIS validation research, analyzed Delaware Stars administrative data, conducted key stakeholder interviews and focus groups, and implemented a virtual pilot test using national data to identify relationships between program quality and child developmental outcomes.

Cover: Evaluation of Delaware Stars for Early Success
Read Online

Evaluation of Delaware Stars for Early Success

Year 1 Report (2014)

by Heather L. Schwartz, Lynn A. Karoly, Vi-Nhuan Le, Jennifer T. Leschitz, Claude Messan Setodji

Download eBook for Free

FormatFile SizeNotes
PDF file 1.8 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. What lessons from prior research can inform the design of an evaluation of Delaware Stars for Early Success, Delaware's voluntary system for rating the quality of early learning and care programs?
  2. What is the extent of participation in Delaware Stars among home- and center-based providers, and what do the Delaware Stars ratings reveal about dimensions of program quality?
  3. What are the experiences of system administrators, providers, and families with Delaware Stars? What aspects of the system are working well, and what challenges do stakeholders identify?
  4. What can existing national data reveal about the relationship between the dimensions of quality in the Delaware Stars quality rating and improvement system and child developmental outcomes?

Delaware was in the first group of states to receive a federal grant in 2012 to improve early care and education services and increase the number of infants, toddlers, and preschool-age children in high-quality programs. One component of the state's grant is a rigorous validation process for Delaware Stars for Early Success, a voluntary quality rating and improvement system (QRIS), including a study to assess the relationship between the quality ratings and the developmental outcomes of children. To support Delaware's efforts to ensure that the QRIS is working as intended, RAND researchers examined prior validation research, analyzed Delaware Stars administrative data, conducted a series of key stakeholder interviews and focus groups, and implemented a virtual pilot test of Delaware Stars using national data to identify relationships between program quality and child developmental outcomes. The results provide a baseline understanding of the types of child care providers that participate in Delaware Stars, their Delaware Stars quality ratings, how quickly they progress to higher quality levels and which standards they are meeting, how enrolled children are distributed across participating and nonparticipating programs, and the experiences of administrators, providers, and families with the current system. Although this initial study was not intended to provide definitive recommendations for improving Delaware Stars, the findings point to ways to reinforce initiatives already under way in Delaware Stars or that could be considered for the future.

Key Findings

Delaware Stars Has High Rates of Participation, Especially for Center-Based Programs

  • As of January 2014, 72 percent of licensed early care and education (ECE) centers participated in Delaware Stars, the providers first targeted for enrollment. Enrollment of home-based child care providers, the more recent entrants, was 21 percent.
  • Delaware Stars ECE providers served about two out of three children enrolled in licensed programs. Participation was even higher for children in center-based programs (77 percent).

Delaware Stars Providers Have Varying Levels of Quality

  • A plurality of providers (38 percent) in Delaware Stars were rated at the Star 2 level as of January 2014, the second of five rating tiers. About one in three providers (32 percent) was at the Star 4 or Star 5 level.
  • Almost half of participating children (but just 27 percent of those with Purchase of Care subsidies) were in Star 4 or Star 5 programs.
  • Delaware Stars uses a points-based provider choice model, so top-rated providers do not meet a consistent set of quality standards, including some that are expected for high-quality programs.

Stakeholders Are Committed to Improving Delaware Stars

  • Systems strengths, according to stakeholders, included engagement from all parties, strong participation by providers, and extensive technical assistance for providers.
  • Identified areas for improvement included prioritizing quality standards, ensuring financial sustainability, and improving parent awareness.

Prior Research Shows the Need for QRIS Validation Research

  • Previous validation studies show that QRISs, as currently designed, do not necessarily capture meaningful differences in program quality.
  • Research shows that it is challenging to identify measures of program quality that are highly predictive of children's early learning.

Recommendations

The research activities documented in this first-year report were not designed to produce definitive recommendations, but they do provide support for initiatives under way in Delaware Stars or suggest future directions for the QRIS.

  • Delaware Stars should continue its efforts to recruit small and large family child care providers and address the unique circumstances facing home-based providers.
  • Expansions to quality improvement supports already under way should help lower-rated programs progress to higher ratings more quickly. Delaware Stars should consider targeting Star 2 programs, which serve a large share of children with Purchase of Care subsidies.
  • Upcoming Delaware Stars enhancements should also generate further quality improvement, especially for hard-to-attain standards, such as those pertaining to developmental screening and a comprehensive curriculum, where even higher-rated programs can benefit. The enhancements will also introduce more required standards at the higher levels, ensuring consistency in the quality of the highest-rated programs.
  • There is room for improvement in how Delaware Stars is marketed to parents and the general public. Awareness could be improved by making Delaware Stars part of a statewide campaign to promote healthy child development.
  • The Delaware Stars administrative database should be improved to make it more useful for tracking programs and conducting analyses. Delaware Stars should revise its system to capture data in a uniform way and to make it accessible to parents, providers, and other stakeholders.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Overview of Delaware Stars

  • Chapter Three

    Participation in Delaware Stars and Rating Results

  • Chapter Four

    Stakeholders' Experiences with Delaware Stars

  • Chapter Five

    A Virtual Pilot of Delaware Stars

  • Chapter Six

    Conclusion and Implications

  • Appendix A

    Additional Documentation for Delaware Stars

  • Appendix B

    Data Sources for Administrative Data Analyses

  • Appendix C

    Interview and Focus Group Protocols

  • Appendix D

    Virtual Pilot Data, Measures, and Methods

The research in this report was produced within RAND Education and RAND Labor and Population, divisions of the RAND Corporation. The research was sponsored by the Delaware Office of Early Learning.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.