A Feb. 1 report by our organizations examined student achievement in privately managed and district-managed public schools in Philadelphia. The report sparked a flurry of charges and countercharges. Proponents and opponents of the city's five-year-old experiment in the private management of public schools interpreted the report very differently.
As coauthors of the report, we'd like to set the record straight on what the report says, what it means, and what key questions Philadelphians need to answer about the future of their schools.
Our report examined academic achievement measured by reading and math test scores among students attending Philadelphia public schools. Some news accounts suggested — inaccurately — that our report indicated privately managed schools had fallen behind districtwide achievement trends. In fact, students at schools managed by private operators kept pace with — but did not exceed — the gains of students in the rest of the district in the past four years, when achievement levels districtwide rose substantially.
Defenders of the private managers suggested — inaccurately — that our report was unduly negative about them because we had not accounted for the fact that the private managers were given many of the lowest-achieving schools in Philadelphia to operate. In fact, our analysis explicitly accounts for baseline achievement levels in different schools, creating a fair comparison between privately managed schools and other schools in the district.
Some private managers have also complained that we overstated the true costs of their schools, because they have less-experienced, lower-salaried teachers. True, differences in total per-pupil costs for privately managed schools and other schools in Philadelphia are affected by differences in teacher experience. But the extra funding given to the private managers included money designed to compensate for those differences.
Our report was never intended to be a final judgment on the merits of the private managers. Instead, it was designed to contribute to a public discussion on how Philadelphia can learn from the experience of having some schools operated by the school district and others run by private operators as a model of school reform. In the interest of promoting that discussion, we offer the following questions for policy-makers.
Are the privately managed schools (or the other schools that received additional resources) producing benefits that are not measured by reading and math scores?
The reading and math results found in the RAND/Research for Action report represent only one aspect of the schools' performance. The district should also consider outcomes beyond the scope of our report, such as performance in other curriculum areas such as science, social studies, and the arts, along with student safety and parental satisfaction.
Can Philadelphia make the most of its experiment in private management by selectively retaining private managers in schools where the managers are working well, while moving in a different direction where they are not working well?
Anecdotal evidence suggests that the private managers' effectiveness varied substantially across schools. Our report did not conduct a school-by-school analysis of performance, but such an analysis could be conducted using the methods we employed.
What can be learned from the success of the 21 public schools given intensive intervention and extra resources while remaining under district management?
These 21 "restructured" schools outpaced the school district's average gains in mathematics. Those gains appear to have been sustained a year after the intervention ended, providing reason for optimism that the district succeeded in creating the capacity for sustained improvement. Whether and how that success can be replicated district-wide is a key question for the future.
How can the district continue the pace of achievement gains seen in the first three years the district was run by the School Reform Commission and CEO Paul Vallas?
Does the presence of private managers create a healthy competitive effect partly responsible for districtwide achievement gains? That question can't be answered from an analysis of these data. So many different reforms have been implemented in Philadelphia since 2002 that it is impossible to clearly identify which ones may be responsible for the achievement gains of the district as a whole.
Our own conclusion is that competition from private managers is unlikely to be a major factor in student achievement gains, because private management was not implemented in a way designed to promote competitive effects. In particular, private management did not include a regime of open parental choice of schools. Additionally, schools elsewhere in Pennsylvania improved at similar rates despite the absence of private managers. But reasonable observers may disagree.
We hope our study will contribute to a healthy discussion not only about whether and how the private provider relationships should be continued, but also about how the district and the city can build on past successes, continuing the hard work of raising the achievement of all students in Philadelphia.
Brian Gill is a social scientist in the Pittsburgh office of the RAND Corp.
Jolley Bruce Christman is a senior research associate at Research for Action, a Philadelphia-based nonprofit organization working in educational research and reform.
