This evaluation describes the Regional Choice Initiative (RCI), a large-scale initiative in 17 school districts in Pennsylvania. The RCI sought to expand school choice and offer more-rigorous courses for students in grades 7 to 12. RAND evaluated the performance of the individual programs and the RCI overall. The authors also identified aspects of implementation that facilitated or hindered the performance of the RCI.

Evaluation of the Regional Choice Initiative

Andrea Phillips, Kun Yuan, Shannah Tharp-Gilliam

ResearchPublished Jan 8, 2016

School districts in Beaver County, Pennsylvania, have been working to further improve student achievement by increasing students' exposure to more-rigorous courses. In 2007, the Beaver Valley Intermediate Unit (BVIU), a regional educational-service agency, received funding for a five-year grant under the U.S. Department of Education's Voluntary Public School Choice program. The BVIU responded by developing, implementing, and evaluating the Regional Choice Initiative (RCI), a large-scale initiative designed and implemented in 17 school districts. The RCI sought to expand school choice — as well as provide opportunities for students in low-performing districts to learn in high-performing environments — for students in grades 7 to 12 by offering four programs: Open Seats, Dual Enrollment, Cyber Learning, and Academies for Success. The RCI was implemented for six years — 2007–2008 to 2012–2013.

The BVIU commissioned RAND to conduct a formative and a summative evaluation of the RCI programs. The RAND team took a mixed-methods, quasi-experimental approach to evaluate the implementation and impact of the RCI initiative. The authors reviewed program documents, interviewed a number of stakeholders throughout the region, surveyed parents, and analyzed student RCI participation data and administrative data. The RAND team also worked with the RCI team and partnering superintendents to develop the RCI logic model and set up performance measures for each RCI program to guide the implementation and evaluation of the RCI. RAND evaluated the performance of the individual programs and the RCI overall. The authors also identified aspects of implementation that facilitated or hindered the performance of the RCI.

Key Findings

The Overall Participation in RCI Programs Was Low and Inconsistent

  • In total, 4,357 students participated in four RCI programs. The number of participants increased from 165 in year 1 to 2,063 in the final two years.
  • The number of participants each year was less than 10 percent of the overall enrolled students in grades 7–12 in the first four years of program implementation.
  • Most participants (86 percent) participated in RCI for only one year, and more than 90 percent of participants took courses through only one RCI program.

Although RCI Met Most of Its Objective Measures, It Failed to Meet the Majority of Its Performance Measures

  • The RCI achieved eight out of 21 measures.
  • Five of the eight achieved measures were objective measures. The three achieved performance measures were about the dropout rate of participating students and the number of courses in the core curriculum created by content-area faculty teams.
  • None of the performance measures that required a certain percentage of participating students earning a passing grade in courses enrolled through RCI programs was met.

Data Available for Impact Analysis Were Limited; No Relationship Was Detected Between Participation and Student Engagement and Achievement Outcomes

  • Participating districts were unable to provide all the data required for analysis. Insufficient data limited the statistical power of the analyses and therefore the ability to identify program effects.
  • The analysis based on available data did not find any discernible relationship between participation in RCI programs and student achievement and engagement outcomes at the school or district level.

Multiple Factors Contributed to the Low Level of Participation in the RCI

  • These factors include lack of quantifiable indicators for eligible students; school stakeholders' concerns about the potential impact of the program on students, staff, and the school; lack of compatible schedules across schools; students' unwillingness to attend courses using virtual-learning technologies; and logistical constraints for students to physically transfer among schools to take courses.

Stakeholders Viewed the Impact of the RCI on Participating Districts and Students Positively, and Districts Decided to Sustain Most RCI Programs in the Future

  • Stakeholders reported that the RCI and its programs provided important opportunities to students.
  • Stakeholders reported that the RCI and the work of its administrators improved communication and collaboration between districts and transformed the climate and culture of the county.

Recommendations

  • Test the feasibility of designed program features and the implementation plan in advance. This feasibility study should examine the demand for the expected program, examine the readiness and capacity of participating organizations to implement the program features, and identify problems that might arise during the implementation of the program.
  • Create a strong, multitiered communication strategy to communicate program information to the target student population and their parents. Experts in communication might support the development of effective campaigns that appeal to the target population and encourage recruitment.
  • Use a dashboard system from the beginning of the program to monitor the implementation and facilitate communication with stakeholders.

Order a Print Copy

Format
Paperback
Page count
112 pages
List Price
$29.50
Buy link
Add to Cart

Topics

Document Details

  • Copyright: RAND Corporation
  • Availability: Available
  • Year: 2016
  • Print Format: Paperback
  • Paperback Pages: 112
  • Paperback Price: $29.50
  • Paperback ISBN/EAN: 978-0-8330-9255-7
  • DOI: https://doi.org/10.7249/RR1386
  • Document Number: RR-1386-BVIU

Citation

RAND Style Manual
Phillips, Andrea, Kun Yuan, and Shannah Tharp-Gilliam, Evaluation of the Regional Choice Initiative, RAND Corporation, RR-1386-BVIU, 2016. As of September 23, 2024: https://www.rand.org/pubs/research_reports/RR1386.html
Chicago Manual of Style
Phillips, Andrea, Kun Yuan, and Shannah Tharp-Gilliam, Evaluation of the Regional Choice Initiative. Santa Monica, CA: RAND Corporation, 2016. https://www.rand.org/pubs/research_reports/RR1386.html. Also available in print form.
BibTeX RIS

The research described in this report was sponsored by the U.S. Department of Education and conducted within RAND Education, a research unit of the RAND Corporation.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.