Final Report on the Hawai i P-3 Evaluation
Oct 7, 2015
How Well Did P–3 Work?
|PDF file||0.5 MB||
Use Adobe Acrobat Reader version 10 or higher for the best experience.
Photo by yangna/iStock
The RAND Corporation conducted a five-year evaluation of the Hawai'i P–3 initiative from 2009 through 2014. This brief summarizes the key findings, focusing on how the P–3 initiative was implemented and whether it improved third grade reading scores.
A range of evidence suggests that public schools are failing to prepare students for success in work and life. Several education reforms have emerged over the past decade to address these concerns. The "preschool through third grade" or "P–3" education reform movement aims to improve students' kindergarten readiness and sustain high-quality early education experiences through elementary school. Proponents of P–3 note that many students enter kindergarten lacking the basic skills needed to succeed in school and that those who enter kindergarten with such deficits tend to stay behind. They also note that many early education gains fade out after children enter elementary school.
With support from the W. K. Kellogg Foundation, the Hawai'i P–20 Partnerships for Education — a collaborative overseen by state leaders in early, K–12, and higher education — launched the state's P–3 initiative in 2007, one of the first comprehensive P–3 initiatives. The initiative's goal was for every child in Hawai'i to read at grade level by third grade. Five demonstration sites were selected to implement the initiative in their communities, and P–20 also implemented state-level work.
Through interviews and document reviews we determined that the state-level and demonstration site work included core P–3 features, that most of the elements of system change had been implemented, and that the work will contribute to long-term change in Hawai'i's P–3 landscape. Lacking, however, is planning for future work. State-level efforts are likely to yield lasting system change because they have been institutionalized in new standards and programs. Demonstration site efforts, more dependent on the support of local leaders with uncertain tenure, may be less likely to endure.
This is the first P–3 evaluation to analyze whether P–3 reforms produced improvements in third grade performance by analyzing reading score data — in this case, data drawn from the Hawai'i State Assessment (HSA) reading assessment. Specifically, we compared the scores of students in the five sites with those of students in schools not exposed to P–3. The analyses included all 192 elementary schools with available test score data for seven years — spring 2008 to spring 2014 — for a total of 98,909 students.
Our evaluation finds evidence that more years of participating in the P–3 initiative raised student reading scores modestly but significantly and increased the likelihood of scoring proficient on the state reading test. The P–3 initiative targeted low-performing schools in the state; thus, relative to other schools, demonstration site (P–3) schools started with a sizable reading proficiency gap (see figure below). In 2008, 64 percent of students in schools that were not part of the P–3 initiative were proficient in reading, while 53 percent of students who were eventually part of the initiative were proficient, an 11 percentage point gap. By 2014, that gap had narrowed to 7 percentage points — 67 percent versus 60 percent.
Viewed another way, over the seven-year period covered by the data, the gap in scores between schools that never participated in P–3 and those that did narrowed by four points on the HSA reading score. This impact is comparable to estimates of the effects of nine additional weeks of schooling and is higher than an estimate of the average effect size for elementary school interventions for mainstream students.
Although activities other than P–3 were occurring in the sites that may have contributed to these impacts, the findings are encouraging in showing a narrowing of the reading score test gap by the end of the P–3 initiative.
Photo by Eugene Tanner/AP Photo
Determine in advance an appropriate balance between standardization and site-specific needs and resources, rather than relying on one-size-fits-all approaches.
Consider contracts that specify outcomes rather than activities — increasingly recognized as a valuable approach for education and social service contracting.
Establish measurable, standardized outcomes for the work to make monitoring possible and suggest mid-course corrections if the work is not producing expected outcomes.
Plan explicitly for unanticipated changes in policy and personnel.
Consider sustainability from inception.
Require explicit agreements between high-level early education and K–12 administrators to engage in collaborative work to support strong coalition-building.
This report is part of the RAND Corporation Research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.