Better child outcomes are the ultimate goal of early care and education (ECE) quality improvement (QI) efforts, but assessing these outcomes is difficult and rarely done. This study identifies five strategies for incorporating child assessments into the design, implementation, and evaluation of QI initiatives such as quality rating and improvement systems. The study assesses the merits of each strategy and offers guidance for its use.

Moving to Outcomes

Approaches to Incorporating Child Assessments into State Early Childhood Quality Rating and Improvement Systems

Gail L. Zellman, Lynn A. Karoly

Expert InsightsPublished Feb 28, 2012

Many studies have shown that higher-quality early care and education (ECE) predicts positive developmental gains for the children who experience it. However, much ECE in the United States is not of sufficiently high quality to produce these benefits. Quality rating and improvement systems (QRISs) attempt to improve practice and care quality in ECE settings; both are expected to improve child functioning. However, these systems rarely assess children to determine their effects because of the high costs and assessment burdens involved. Yet including child assessments in the design, implementation, and evaluation of QRISs or other quality improvement (QI) efforts could improve practice and raise care quality. The authors identify five strategies for states to consider for incorporating child assessments into QRISs or other QI approaches. Two of the strategies use assessments to inform classroom practice and to support program improvements. The remaining three use assessments to measure the effects of participating in a given classroom, program, or ECE system on child functioning. The authors' analysis of these strategies relies on research about measuring child functioning and methods for determining the contribution of ECE to developmental trajectories. It relies as well on new research concerning how states have included child assessments in their QRISs. Guidance is offered about when and how to incorporate the five approaches into a QRIS; the value of these approaches depends on the questions to be answered, the stage of the QRIS, and the availability of the resources needed to implement assessments and mount a rigorous research design.

Key Takeaways

Quality Rating and Improvement Systems (QRISs) Rarely Directly Assess Children to Determine System Effectiveness

  • Despite the cost involved in accurately measuring child functioning, and the difficulty in identifying the contribution of any given early care and education (ECE) setting to a particular child's developmental trajectory, QRISs should incorporate child assessments, at least to some extent, because they can help to improve practice and the quality of the QRIS.

States Have Options for Incorporating Child Assessments into Their QRISs

  • The authors identify five approaches for incorporating child assessments into state ECE quality improvement efforts. Some focus on child assessments at the micro level — the development of an individual child or group of children in a classroom; some have a macro focus — the performance of a QRIS. Each strategy is analyzed in terms of its current use in states, the types of questions it can address, the resources required, the stage of a QRIS when it can best be deployed, and the benefits and barriers to its use.

Child Assessments Can Be Used in Different Ways

  • Child assessments can serve multiple purposes. They can inform caregiving and instructional practice and help identify needs for staff professional development and other program quality enhancements. They can contribute to assessments of the effectiveness of technical assistance or other interventions. They can be used to validate the rating portion of a QRIS. And they also can be used to determine if a QRIS or other improvement effort leads to better child developmental outcomes.

Recommendations

The authors offer the following guidance to policymakers and QRIS designers regarding which strategies to use for incorporating child assessments into QRISs and other quality improvement efforts and in what circumstances:

  • Promote the use of child assessments, either as a required element at specified rating tiers of a QRIS or by reinforcing best practice through licensing, regulation, or accreditation. Either approach can be implemented at relatively low cost, since teachers often learn to use key assessment tools in their training.
  • Collect independent measures of child outcomes to assess QRIS validity, which can be done fairly easily when piloting a QRIS and can be redone periodically once the QRIS is implemented at scale. Although the methodology is complex, periodic implementation of this approach produces high return on investment with limited assessment burden on children.
  • Use independent measures of child functioning to evaluate the ability of the ECE system or specific ECE programs to improve child outcomes regardless of whether a QRIS exists. Although the methodology for this approach is also complex, the resulting evidence of system effects can justify spending and guide quality improvement efforts.
  • Carefully weigh the benefits versus the costs of collecting independent measures of child outcomes to rate specific classrooms or programs as part of a QRIS. The methodology for using this approach is not sufficiently developed for use in high-stakes QRISs. Given the current state-of-the-art, this approach is costly to implement for uncertain gain.

Topics

Document Details

  • Copyright: RAND Corporation
  • Availability: Web-Only
  • Year: 2012
  • Pages: 80
  • Document Number: OP-364-PF

Citation

Chicago Manual of Style

Zellman, Gail L. and Lynn A. Karoly, Moving to Outcomes: Approaches to Incorporating Child Assessments into State Early Childhood Quality Rating and Improvement Systems. Santa Monica, CA: RAND Corporation, 2012. https://www.rand.org/pubs/occasional_papers/OP364.html.
BibTeX RIS

The research described in this report was conducted jointly by RAND Education and RAND Labor and Population, units of the RAND Corporation. Funding was provided by the David and Lucile Packard Foundation.

This publication is part of the RAND occasional paper series. RAND occasional papers were products of RAND from 2003 to 2013 that included informed perspectives on a timely policy issue, discussions of new research methodologies, essays, papers presented at a conference, and summaries of work in progress.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.