Takeover Overtaken
Public Management of Philadelphia Schools Leaves Private Management Behind in Math
By Brian Gill
Brian Gill is a RAND social scientist. His research addresses a variety of topics in K–12 education policy, such as parental choice, districtwide institutional reform, and high-stakes testing.
Since 2002, Philadelphia has been the site of the nation’s largest experiment in the state takeover and private management of public schools. As such, the city serves as a test case for some of the most aggressive interventions sanctioned by the federal No Child Left Behind Act of 2001 and offers lessons for schools and districts nationwide.
Soon after Pennsylvania took over the Philadelphia school district, the district adopted a “diverse provider” model, handing over management of 45 of its lowest-achieving schools to for-profit organizations, nonprofit organizations, and local universities. The aim was to capitalize on the know-how of the private sector to improve the performance of public schools. The district also gave the private managers extra funding per student.
The aim was to capitalize on the know-how of the private sector. |
Four years later, student achievement across Philadelphia has risen substantially. On average, the schools under private management have matched but not exceeded the districtwide trends. In contrast, a set of district-managed schools that were given both additional funding and a “restructuring” intervention showed consistently and significantly larger achievement gains in math.
Philadelphia’s diverse provider model is of national importance for several reasons. First, the district outsourced the core functions of public schools: the design and delivery of education programs. Second, the scale of private management — 45 schools — is far larger than in any other district in the nation to date. Third, Philadelphia offers a preview of what could happen in other low-achieving schools across the country in response to the accountability directives of the No Child Left Behind Act, which is the most ambitious federal incursion into primary and secondary education in the nation’s history.
The No Child Left Behind Act requires states to create high-stakes testing systems that trigger steadily stronger interventions in chronically low-achieving schools and districts. Three of the strongest interventions advised by the law are state takeover, school restructuring, and private management. Few schools and districts have yet reached this phase, but many soon will. In Philadelphia, the state imposed the interventions well in advance of the federal requirement, giving an early demonstration of its potential effects.
The Philadelphia model diverged from theoretical models of competition in important ways. There was little competition among providers, no parental choice of schools, and continued district involvement in the privately managed schools. Thus, this case should not be viewed as a definitive test of private management under competitive conditions.
|
|
Diverse Providers, Similar Results
Fed up with years of low achievement and budget crises in the Philadelphia schools, Pennsylvania seized control of the district in 2002 and replaced its school board with an appointed School Reform Commission. The commission hired Paul Vallas, former chief executive officer of the Chicago public schools, to take the helm in Philadelphia.
Vallas instituted sweeping changes. He modernized efforts in hiring and retraining qualified teachers, launched an ambitious program of school construction and renovation, pushed vigorously for an upgraded curriculum, and balanced the budget. He mandated frequent assessments, a zero-tolerance discipline policy, assistance for low-performing schools, extended-day and summer school for poorly performing students, and the dismantling of middle schools in favor of K–8 schools. These districtwide initiatives applied equally to all schools, including those operated by private providers. The most controversial change of all was the diverse provider model: turning over management of 45 of the district’s elementary and middle schools to seven private managers.
Figure 1 —Student Proficiency Increased Throughout the School District of Philadelphia from 2002 to 2006 |
SOURCE: State Takeover, School Restructuring, Private Management, and Student Achievement in Philadelphia, 2007. |
The diverse provider model also included 21 low-achieving schools that were “restructured,” or given extra funding per student and intensive professional development for principals and leadership teams, while remaining under district management. (Restructuring in Philadelphia contrasts with the restructuring envisioned by No Child Left Behind. The latter anticipates “significant changes in the school’s staffing and governance” that might involve replacing principles and teachers, converting the school to a charter school, or allowing a private manager to take over.)
As a baseline for comparison, our RAND team looked at the districtwide achievement trends through spring 2006. The proportion of elementary and middle-school students achieving proficiency in reading and math increased substantially throughout Philadelphia since the 2002 state takeover. From the 2001–2002 school year to the 2005–2006 school year, an additional 11 percent and 23 percent of fifth graders reached proficiency in reading and math, respectively. Likewise, an additional 20 percent and 19 percent of eighth-graders reached proficiency in reading and math, respectively (see Figure 1). Philadelphia’s achievement gains were in most cases approximately equivalent to the gains of similarly low-achieving schools elsewhere in the state.
We then compared the trends in the privately managed and restructured schools with the trends of other Philadelphia students. Our major findings are as follows:
- Privately managed schools: There were no statistically significant effects, positive or negative, in reading or math in any of the four years after takeover.
- Restructured schools: There were significantly positive effects in math in all three years of implementation and in reading in the first year. In the fourth year, after the additional resources for these schools had ceased, they maintained a substantially positive effect in math (although the effect was just marginally statistically significant).
In short, after four years of intervention, the achievement gains in Philadelphia’s privately managed schools were, on average, no different from Philadelphia’s districtwide gains. However, the restructured schools remaining under district management outgained the rest of the district in math in all three years of restructuring, with evidence of the gain persisting a year afterward.
We found no statistically significant effects, positive or negative, among the three types of private providers: for-profits, nonprofits, or universities. We found few statistically significant differences among individual providers. And we found no clear indications of notable differences in effects on particular at-risk subgroups of students, such as special education students or those with limited English skills.
Figure 2 portrays the most salient empirical findings in a way that only a statistician could love. As shown in the figure, the only achievement gains that were statistically significant in comparison with the districtwide gains occurred in the restructured schools. At those schools, the gains in math in all three years of implementation and in reading in the first year were statistically significant at a level of 95-percent confidence. In the fourth year, after the restructuring ended, the enduring gains in math were statistically significant at a level of just 90-percent confidence. The results for each year represent the cumulative effects of each treatment from the start of the intervention through that year.
Figure 2 —Under the Diverse Provider Model, the Only Gains That Were Statistically Significant in Comparison with the Districtwide Gains Occurred in the Restructured Schools |
SOURCE: State Takeover, School Restructuring, Private Management, and Student Achievement in Philadelphia, 2007.*Statistically significant at the 95-percent confidence level.**Statistically significant at the 90-percent confidence level. |
All results are shown in terms of what are known as “z-scores,” which are admittedly incomprehensible to just about anyone who is not a statistician. Rather than comparing raw numerical gains in test scores, educational statisticians convert everything to z-scores so that all results from varied interventions across time and place can be put on a single scale. In this case, the “effect sizes” for each intervention are calculated as fractions of a standard deviation from the districtwide norm. The key insight here is that the effect sizes for math at the restructured schools, ranging from 0.16 to 0.22 standard deviations, are considered moderate to large, relative to those seen in other educational interventions.
Where Credit Is Due
Different interpretations of the results may lead to different judgments about whether Philadelphia’s experiment in the private management of public schools has succeeded and whether it should be continued. On the negative side of the ledger, privately operated schools did not produce average increases in student achievement that were any larger than those in the rest of the district. Meanwhile, the district-managed restructured schools outpaced the rest of the district in math.
Still, it is impossible to know definitively how well the privately managed schools would have performed had they remained under conventional district management or been restructured. Whether the district could have replicated the gains of the restructured schools in three times as many schools — not just 21 but an additional 45 — is unclear. The private managers were given some of the lowest-achieving schools in the district. They improved alongside the rest of the district at a time when achievement levels districtwide were increasing substantially. We cannot rule out the possibility that the assistance from private providers was an important part of the total reform effort in Philadelphia.
On the other hand, we found little reason to believe that competition from private providers spurred the districtwide improvement. We found no evidence to support the view that adding private managers increased the districtwide capacity for improvement or that the privately managed schools would have fared worse otherwise. If they had remained under district management, it seems likely that the district could have replicated the gains of the other schools that received no special treatment, obtaining results similar to those actually achieved by the private managers.
Advocates of privatization might contend that our focus on the relative gains of different groups of schools within the district obscures the larger story of citywide gains, and that those citywide gains are attributable to competitive pressure induced by the introduction of private managers. If healthy competition caused all schools in the district to improve, it is possible that the comparison of schools within Philadelphia underestimates the true benefit of introducing private managers.
There is no way to definitively address this hypothesis. First of all, Vallas and the School Reform Commission introduced so many districtwide reform efforts that it is impossible to know which of them were primarily responsible for the districtwide gains. Secondly, the diverse provider model in Philadelphia was not set up in a way that would maximize competition: Private managers’ freedom of action was constrained in ways that minimized differences among them, and, more important, the district did not promote parental choice among schools in a way that would allow the schools to compete for students. Moreover, other schools in Pennsylvania saw gains similar to those in Philadelphia, without introducing private management. In short, although there is no way to disprove the competition hypothesis, we do not find evidence supporting it. It appears at least as likely that other aspects of the district’s initiatives were primarily responsible for the citywide achievement gains.
The Philadelphia experience provides no evidence to support private management as an especially effective method of promoting student achievement — at least not under the model implemented. |
While the private managers’ contracts are now coming up for renewal, we do not find evidence that would support providing the private managers with additional funding beyond that available to district-managed schools. Nonetheless, average results obscure considerable variation across schools and managers, and we hope that the school district carefully considers the success of each school and each provider as it considers contract renewals. Moreover, our study was limited to an examination of achievement results in reading and math. Private providers may be producing other benefits that are not captured in test results.
The larger implications of these findings for the most aggressive sanctions of No Child Left Behind are unclear. With respect to state takeover, the Philadelphia results are ambiguous: Subsequent to the state takeover of the district, student proficiency increased districtwide, but the total increase over four years was not substantially greater than the increase for other low-achieving schools across the state. The Philadelphia experience provides no evidence to support private management as an especially effective method of promoting student achievement — at least not under the model implemented, with constrained competition and limited provider autonomy. Whether private management involving more autonomy for managers, parental choice, and competition for students would produce better results remains an open question. 
RAND® is a registered trademark. Copyright © 1994-2013 RAND Corporation.