Independent Evaluation of California's Race to the Top-Early Learning Challenge Quality Rating and Improvement System

In 2011, California successfully submitted a Race to the Top–Early Learning Challenge (RTT-ELC) grant application to the U.S. Department of Education that would move the state toward a locally driven Quality Rating and Improvement System (QRIS) or set of systems. The state proposed building a network of 17 Early Learning Challenge Regional Leadership Consortia that had already established — or were in the process of developing — QRIS initiatives in 16 counties. These Consortia, comprised of local First 5 commissions, county offices of education, and other key stakeholders, represent counties that together have more than 1.8 million children ages birth to five. This locally based approach sets some common goals for workforce development, program assessment, and child assessment for school readiness, but allows for some flexibility in quality benchmarks. The counties participating in the RTT-ELC Regional Leadership Consortia have voluntarily adopted a Hybrid Rating Matrix that allows considerable local autonomy in some tier requirements, the rating protocol, and supports and incentives. The goal in this analysis was to collect information on the economic costs associated with the main types of QI strategies employed by local Consortia in California. Such information may be of interest in its own right to understand the resources required to provide various QI supports in the context of a QRIS. In addition, such cost information may provide the basis for undertaking a cost-effectiveness analysis to understand which QI activities produce the largest impacts on program quality improvement or children's developmental gains for every dollar spent. Coaching was the only QI activity that showed consistently positive associations with children's developmental outcomes. Thus, we could calculate a cost-effectiveness ratio for coaching but not for other QI activities. Without multiple ratios, a comparative cost-effectiveness analysis cannot be conducted. To our knowledge, our analysis provides one of the first efforts to estimate the cost of various QI activities on a per-unit basis. We provide such estimates in this chapter for up to 10 of the local Consortia. Given the limitations on the cost information provided by the Consortia, the estimated cost figures should be viewed as approximations of the true economic cost. Furthermore, there is considerable variation across Consortia in the resulting cost estimates. We would expect some variation, given that the Consortia approach the QI activities in different ways and face various costs for labor and other inputs in their local community. But some of the variation may reflect the challenge of calculating such costs in a consistent manner. Future research may seek to understand and document the reasons behind the variations in reported costs by Consortia. This would allow for more meaningful cost estimate comparisons across Consortia and would thus ensure the most reliable cost-effectiveness analyses.