AUGUST 2008

Providing High-Quality Child Care: The Stakes Are Rising

With so many families relying on child care providers these days, there is growing concern that the care provided be of “high quality.” But what constitutes high-quality child care and how can we measure it? Building off decades of work in the child care field—including a large body of work assessing child care quality issues in the military and extensive work examining the costs and benefits of child care—RAND researchers are leveraging their experience to address this question.

Queries about QRISs

Quality Rating and Improvement Systems, or QRISs, are an increasingly popular approach to assessing and improving child care quality. QRISs include multi-component assessments that produce an overall quality rating designed to make child care quality easily understood by consumers. The “improvement” part of the QRIS offers participation incentives and hands-on technical assistance and quality improvement resources to participating providers to improve the level of quality they offer.

While QRISs are popular, there is little information about how well they actually work—an important issue as more and more states implement these systems. Many of the quality measures that are part of the provider ratings were designed for “low-stakes” environments like research studies or quality improvement efforts. Of late, however, the stakes are getting higher; a number of QRISs reward higher-quality child care programs with higher per-child subsidies and other incentives. In such contexts, quality measures designed for low-stakes settings need to be validated for high-stakes use.

Qualistar’s QRIS in Colorado

RAND researchers have recently completed an assessment of Qualistar Early Learning’s QRIS (or, Q-QRIS). Qualistar, a Colorado-based nonprofit, was one of the first organizations to create a QRIS, implementing it in 1999. The Q-QRIS assesses five components that experts generally agree are associated with quality of care: classroom environment, child-staff ratios and group size, staff and director training and education, parent involvement, and accreditation.

To assess the Q-QRIS, RAND researchers examined 65 child care centers and 38 family child care providers using data from Q-QRIS and two other measures of quality. Researchers collected data on over 1,300 children in the first wave of data collection and administered the same instruments over two additional waves approximately 12 months apart.

Mixed Support for the Q-QRIS

The assessment focused on answering a series of questions, including how closely the five Q-QRIS component measures were correlated with each other and with the other measures of quality and whether there was a relationship between the Q-QRIS components and child outcomes. According to the logic model underlying QRISs, an improved child care environment, characterized by more responsive care-giving and enriched content, will lead to better outcomes for children, including improved school readiness, better cognitive skills, and such non-cognitive outcomes as social skills development and creativity.

Study findings provided mixed support for the Q-QRIS and its components as measures of provider quality. On the plus side, Q-QRIS and its component measures correlated moderately with each other and showed some relationships with one of the two process measures of child-adult interaction we used as quality benchmarks. But on the minus side, the research team found little evidence that the Q-QRIS ratings predicted child outcomes, even for subgroups which typically have shown the greatest gains from quality child care.

What Have We Learned?

Although it is hard to generalize the findings to other QRISs because of important study design and implementation limitations—including very high child attrition—the assessment did identify some important lessons, one of which is that building a QRIS takes time and should probably be done incrementally. Each rating system construct should be clearly articulated, designed, tested, and validated in the context in which it will be used. Once the components are well-measured, an iterative, evidence-based validation of the QRIS as a whole can begin.

The lack of relationships to child outcomes raises the broader question of which, if any, child outcomes should be expected as quality increases. For example, early childhood educators are more generally interested in children’s capacity to regulate emotions, develop trusting relationships with adults, and approach learning in a more efficacious way than focusing exclusively on pre-academic skills. Alternatively, until we can build a stronger empirical basis for quality measures, it may be appropriate to ignore longer-term child outcomes entirely, focusing instead on program outputs, such as children’s engagement in developmentally appropriate tasks in a safe and supportive environment.

INTERVIEW

QRISs: Where Do We Go from Here? Gail L. Zellman is a social and clinical psychologist at RAND, where much of her research has focused on child care issues. In particular, she has conducted five major research studies of military child care over the last 20 years and, as part of a Consortium focused on improving child care quality rating systems, has examined QRISs in five “pioneer” states. She has a Ph.D. in Psychology from the University of California, Los Angeles. What can we say about QRISs at this point? Although our assessments of the Q-QRIS show mixed results, QRISs are still the most encouraging policy tool to improve child care quality to come down the pike in a long time. Child care in this country is generally mediocre; QRISs hold substantial promise for improving that situation. Why are they so appealing now? I think QRISs are particularly appealing because they resonate with the nation’s current focus on accountability, which is the driving force behind the No Child Left Behind legislation. QRISs create accountability by measuring quality and making it transparent. They also offer providers improvement incentives and support. As evidenced by our own just-completed examination of QRISs in five states, enormous state-level energy has been put behind QRISs. In fact, the state efforts have even gotten Congress’s attention. Both Senator Casey and Senator Dodd are pursuing federal legislation; Senator Dodd’s bill would mandate a QRIS in every state. But there are issues with QRISs? Our research shows that QRISs need a more solid foundation. There is little empirical research to support the cut scores and weighting schemes used in these systems, and the measures used in the ratings themselves have generally not been validated in the high-stakes settings that characterize QRISs. Such validation is critical. Why is this such an important issue? As the stakes attached to ratings increase, this issue will become more prominent. Recently, a Florida child care provider filed a lawsuit challenging the rating it received. And there are other problems as well. For example, subsidies are generally set at 75 percent of market rate, which means that providers serving lower-income parents cannot possibly achieve higher ratings, because quality improvements are generally costly. So continued research is a must? Absolutely. Along with a push for QRISs must go a push for empirical research about them. A number of stakeholders have gotten together and are working to develop a QRIS Consortium that would devote resources to sharing data and conducting the many research studies required to provide a strong empirical basis for QRISs. Such research would make these systems more defensible, would enable system developers to create more efficient measures of the key components that underlie these systems, and would focus attention on attainable QRIS outcomes. Funding for such research should be included in any QRIS legislation.

For monthly email updates on all new RAND child policy publications and research projects, sign up for the Child Policy mailing list at http://www.rand.org/research_areas/children/.

The RAND Corporation is a nonprofit research organization providing objective analysis and effective solutions that address the challenges facing the public and private sectors around the world.

Learn More »

Copyright © 2008 RAND Corporation. 1776 Main Street, Santa Monica, CA 90401-3208. RAND® is a registered trademark.

We respect your privacy. If you do not wish to receive this periodic newsletter, please email ocr@rand.org or call (703) 413-1100 x5395.

This newsletter is also available on the RAND Congressional web site »