- What did an independent developmental evaluation of QSLA reveal about key challenges in data collection and usage?
- What are the lessons learned for other QRISs as they create or expand data systems?
Quality Start Los Angeles (QSLA) is a county-level quality rating and improvement system (QRIS) that supports center-based and family child care providers serving children from birth to age five. During a developmental evaluation of QSLA that focused on program coaching, assessment technical assistance, and quality tier rating perceptions, the authors of this report also examined data use at QSLA. In this report, they identified strengths, challenges, and lessons learned that could benefit other QRISs or early learning systems as such organizations create or expand data systems, noting that, even with good intentions and a strong data system infrastructure, data use might not always be implemented as expected.
- Variable definitions and entry protocols sometimes differed across organizations, limiting the ability to use existing data to fully address several intended research questions.
- When data are not easy to manipulate or export for staff or evaluator use, analyses are less likely to happen in a timely manner.
- Most QRIS models evolve and change as new providers are added, new funds become available for service expansion, or program evaluation leads to improvements in the design. But decisionmakers are often interested in comparing implementation measures or program outcomes over time, which can present a challenge if some time periods had different data-entry requirements.
- The existence of multiple data sources—from multiple partners and changes over time—can make ongoing evaluation more time-intensive and, in some cases, depending on data quality across the sources and linking variables, questions might not be answerable. Creating processes to link distinct data sources collected by different organizations enhances the potential for data use.
- To know whether quality improvement models are improving provider quality, data systems must be able to link model components—such as coaching or technical assistance—to such outcomes as tier ratings or classroom assessment scores. This linkage requires advance preparation to identify the key goals for data use and research questions of interest and to ensure that the data system is built to address them.
- Establish common data definitions and entry protocols for all staff who enter data.
- Make data usable for ongoing monitoring and evaluation.
- Ensure data systems can adapt to partnership or QRIS model changes.
- When multiple administrative data sources are necessary, ensure that sources can be easily linked.
- Ensure data systems can connect quality improvement supports to outcomes.