Report
Data Use in Quality Rating and Improvement Systems: Lessons Learned from Quality Start Los Angeles
Sep 9, 2020
RAND researchers conducted a developmental evaluation of Quality Start Los Angeles (QSLA), the voluntary quality rating and improvement system for early learning providers in Los Angeles County. The goal of the evaluation was to determine whether selected components of the QSLA model were feasible, appropriate, and being implemented as designed.
The Quality Start Los Angeles Developmental Evaluation
Research Findings and Lessons Learned
Full Document
Full Document
| Format | File Size | Notes |
|---|---|---|
| PDF file | 8.8 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Technical Appendix
| Format | File Size | Notes |
|---|---|---|
| PDF file | 0.4 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Research Questions
- How much assessment technical assistance do QSLA providers receive? What happens during a typical visit?
- What are the providers' and TAs' perceptions of the technical assistance support? To what extent are TAs perceived to be available and knowledgeable by providers?
- What are the providers' perceptions of the assessment process and their tier ratings?
- How much program coaching do QSLA providers receive? What happens during a typical visit?
- What are the providers' and coaches' perceptions of program coaching? To what extent are program coaches perceived to be available and knowledgeable by providers?
RAND researchers conducted a developmental evaluation of Quality Start Los Angeles (QSLA), the voluntary quality rating and improvement system for early learning providers in Los Angeles County. The goal of the evaluation was to determine whether selected components of the QSLA model were feasible, appropriate, and being implemented as designed. The evaluation focused on two topics: (1) the QSLA assessment process and tier ratings and (2) QSLA coaching. The RAND study team engaged in multiple data-collection and research activities from February 2019 through March 2020, including a survey of early learning providers; administrative data analysis; observations of coaching sessions; and focus groups and interviews with early learning providers, coaches, technical assistants (TAs), and other QSLA stakeholders.
Key Findings
Most providers found the technical assistance support they received as part of the assessment process to be helpful, although there were some challenges
- TAs supported providers on a variety of topics, including how to prepare for the assessment, and what sites' tier ratings meant.
- The majority of surveyed providers found the assessment technical assistance support helpful. A portion of providers expressed some challenges, such as limited time to meet with their TA and having questions that their TA could not answer.
- Forty-one percent of surveyed QSLA providers did not know their tier rating. This lack of knowledge was concentrated among classroom staff.
- Some providers, especially family child care (FCC) providers, perceived the assessment process to be unfair or challenging, while other providers felt the process was manageable.
Similarly, providers were very positive about the coaching support they received, although logistical challenges remained
- The majority of surveyed providers reported that the amount of coaching they received was "about right."
- Providers' busy schedules presented logistical challenges to scheduling coaching sessions.
- Coaches and providers addressed a variety of topics during coaching sessions, and coaches used multiple methods, most commonly observation and feedback, to promote growth and improvement.
- Surveyed providers were overwhelmingly positive about their coaching experience. They described the importance of a positive coaching relationship and reported that coaching helped improve their practice.
- Some coaching challenges emerged, including limited time to meet, providers' perceptions that coaching was not always helpful, and coach and early care and education provider staff turnover.
Recommendations
- Continue to offer quality assessment supports that are tailored to providers' needs. Both providers and TAs felt that the assessment technical assistance helped providers navigate and manage the assessment process.
- Clearly explain the details of the assessment process, including what to expect from assessment technical assistance and the assessment site visit, and create clear channels of communication for providers to learn about the tier ratings.
- Consider the unique experiences of FCCs in the quality assessment process. Some QSLA FCC owners felt that the assessment process and resulting tier ratings may be particularly inaccurate for judging the quality of home-based child care settings. QSLA could consider ways to be more responsive to the FCC home-based settings by addressing their unique strengths and challenges.
- Ensure that the coaching dosage recommendations are feasible and aligned with providers' needs, and establish practices to minimize scheduling burden. Consider ways to ease the scheduling burden by finding ways to cover providers' time to meet with coaches.
- Continue to provide coaching services that are tailored to providers' goals. Providers and coaches cited the provider-driven nature of the coaching process as a strength.
- Consider ways to involve assistant teachers and other support staff in the coaching process. Consider providing more formal guidance to coaches on when and how to involve assistant teachers in coaching sessions.
Related Products
Research conducted by
This study was commissioned by First 5 Los Angeles and undertaken by RAND Education and Labor.
This report is part of the RAND Corporation research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.