Sep 29, 2005
In its comprehensive evaluation of Edison Schools, the nation's largest for-profit manager of public schools, RAND found that the resources and accountability systems that constitute Edison's design represent a coherent, comprehensive, and ambitious strategy to promote student achievement. The best-functioning Edison schools demonstrate the promise inherent in Edison's model, but the performance of Edison schools varies. Average gains of Edison schools during the first three years of Edison management do not exceed the gains of comparison schools, but many Edison schools outgain comparison schools after four or five years.
New forms of governing and managing public schools have proliferated in recent years, spawning the establishment and growth of companies that operate public schools under contract. Among these education management organizations, or EMOs, the largest and most visible is Edison Schools, Inc. In 2004–2005, Edison managed 103 public schools in 18 states and the District of Columbia, serving approximately 65,000 students, most of whom are nonwhite children from low-income families in urban centers. Some of Edison's schools are public schools that preexisted and were contracted to Edison by districts, whereas others are charter schools that Edison helped to start up. The RAND Corporation designed an evaluation of Edison to address the following research questions:
To address these questions, RAND researchers interviewed Edison leaders, reviewed documents, visited 23 Edison schools to interview teachers and principals, and analyzed school-level achievement trends in Edison schools and in conventional public schools with similar student populations and initial performance levels. The results of the research are documented in Inspiration, Perspiration, and Time: Operations and Achievement in Edison Schools.
Edison distinguishes itself from most other school improvement strategies (e.g., school choice, high-stakes testing, comprehensive school reform, class-size reduction, teacher development) by simultaneously addressing resources and accountability systems, rather than focusing on one or the other. Together, the resources and accountability systems that constitute Edison's design represent a coherent, comprehensive, and ambitious strategy to address key elements relevant to providing high-quality education, including capacities, motivation, and opportunities for school staff.
The resources Edison provides to its schools include curriculum, technology, and professional development. Edison aims to raise student achievement as measured by state high-stakes tests in reading and math, while also offering a broad and rich curriculum that includes fine arts and a foreign language for students in all grades. The company has developed a proprietary system of monthly, online assessments known as the Edison Benchmarks, designed to provide instant information to teachers and principals about students' academic strengths and weaknesses. Its professional development program includes not only summer training for teachers and principals but also school-based professional development that relies on teacher collaboration and common planning time to promote a strong professional environment.
Edison's accountability systems are distinguished from those of conventional public schools primarily by their greater focus on outcomes-based accountability—measured mostly by student achievement and school budget management—and reduced emphasis on bureaucratic and political accountability. Edison aims to provide financial rewards to staff in high-performing schools, while seeking to insulate its schools from some of the buffeting of local political winds that may result from turnover in superintendents or school boards.
In sum, Edison's well-developed information systems and its focus on achievement-based accountability should make it especially well suited to the high-stakes testing environment of the federal No Child Left Behind (NCLB) Act.
The comprehensive ambitions implicit in Edison's model suggest that successful implementation requires wholehearted commitment—and hard work—from Edison, its clients, and the staff of its schools. In fact, the best-functioning Edison schools demonstrate the promise inherent in Edison's model. These schools have strong instructional leadership, motivated teachers, effective use of achievement data, high-fidelity implementation of the Edison curricula, and high levels of professional collaboration. Nearly all the Edison schools we visited across the country showed enough consistency of implementation to be clearly recognizable as Edison schools, but we observed considerable variation in the extent to which the schools realized the Edison ideal.
Across Edison schools, the reading and math curricula appeared to be implemented more faithfully than were other subjects that are not regulated by state testing systems. In some schools, notable elements of the Edison program, such as common planning time and a longer school day, were sacrificed as a result of resource limitations or contractual constraints. Edison's ability to insulate its schools from local politics appears to be greater in its charter schools than in its district schools, as suggested by the longer endurance of its management contracts with charter schools.
Among the Edison schools we visited, several factors appear to be important in explaining some of the variation in implementation of the Edison model. In particular:
These findings bear out the importance of the sustained commitment of clients and school staff in promoting effective implementation of the Edison model.
We attempted to assess Edison's effect on student achievement by seeking evidence about what schools might have achieved if they had not been managed by Edison. This assessment involved two key components. First, we examined changes over time to help control for prior achievement levels of students served by Edison schools. Second, we compared achievement trajectories in Edison schools (using proficiency rates on state tests in reading and math) with achievement trajectories in a matched set of conventional public schools serving similar populations of students with similar baseline test scores.
In absolute terms, Edison schools are making gains: Average rates of reading and math proficiency in Edison schools have improved over time. But relative to matched comparison schools, the picture becomes more complicated. Average gains of Edison schools during the first three years of Edison operation have not exceeded the gains of matched comparison schools. Edison results relative to comparison schools improve in years four and five, but whether those improvements ultimately yield net positive effects is the key question. The answer depends on the specific analysis used.
The analysis with the most comprehensive set of Edison schools suggests that most Edison schools are exceeding the gains of their matched comparison schools by the fourth and fifth years after the initiation of Edison management, particularly in math. However, this analysis uses the spring of Edison's first year of management as the starting point and therefore excludes any effects that took place during the first year. Lacking a pre-Edison baseline, the analysis is comprehensive in its inclusion of Edison schools but not comprehensive in its coverage of the time period of Edison management.
Results from pre-Edison baselines are available only for a subset of Edison's schools, because a substantial number of Edison's schools (typically charter schools) were new start-ups that did not exist prior to Edison's management. In consequence, this analysis includes only conversion schools (i.e., schools that existed prior to being managed by Edison), and in the first three years it is dominated by a large number of schools from only three districts (Chester, Pennsylvania; Philadelphia; and Dallas) and therefore does not accurately represent the range of Edison's clients.
On average, Edison conversion schools have fallen behind comparison schools in the first year of Edison operation. As a result, Edison conversion schools on average show smaller gains than comparison schools over the first three years of operation and gains after four or five years that may match but not exceed those of comparison schools.
The evidence of a first-year decline for conversion schools, combined with a lack of first-year information for start-up schools, leaves us with uncertainty about whether gains of Edison schools after four or five years are comparable to or superior to those of matched comparison schools. Given this uncertainty, an examination of differences in achievement among Edison schools, and the factors that might explain those differences, is particularly important.
Estimates of average performance across all Edison schools obscure variation in the achievement trajectories of individual Edison schools. This variation is extensive, with some schools exceeding the performance of comparison schools and others falling behind. Some understanding of the factors that explain this variation among Edison schools is necessary for predicting how any particular Edison school (or prospective Edison school) might perform in the future.
We examined several factors that might be related to the variation in performance among Edison schools. First, we assessed whether Edison is producing better results systemwide in recent years than it did in its early years of operating schools. Schools that opened in 2000 or later showed slightly larger relative gains in mathematics than those that opened earlier, but no difference in relative gains between newer and older Edison schools was observed for reading.
We also used findings from the case study schools to shed light on differences in achievement trajectories among Edison schools. Case study findings are not definitive, because the sample is small and relationships can be measured only in simple, correlational terms, but they are nevertheless suggestive:
Together, our achievement results and our case study observations suggest some actions that Edison and its clients can take to improve the likelihood of successful implementation. We present two sets of recommendations: one for Edison and another for district staff and other policymakers considering hiring Edison.
As the largest private manager of public schools, Edison's experiences provide a model to help policymakers and members of the public understand the benefits and limitations of nontraditional forms of school management. Interest in alternative management may increase as the charter school sector continues to grow and as schools and districts that fail to meet their annual targets under NCLB face some of the more severe sanctions of the law. Our analyses provide evidence regarding what has occurred in schools that have been turned over to Edison management, in terms of both how the program is implemented in schools and what happens to student achievement over time.
Unfortunately, data limitations—particularly the lack of information about the first year in start-up schools and the small numbers of schools that have been managed by Edison for at least four years—render the historical evidence on Edison's net long-term effects equivocal. Predicting future long-term effects is especially challenging because it involves assumptions about the stability of Edison's management approach as well as the kinds of schools and districts with which Edison is likely to work in the future. Moreover, there was extensive variability in the results achieved by individual Edison schools. We therefore cannot make strong predictions for prospective clients about whether they will achieve better long-term results with Edison or with an alternative approach. Nevertheless, Edison's improving trends are encouraging, and some schools have thrived under Edison management, indicating that Edison is capable of producing favorable results. Edison and its current and future clients must work to establish the conditions in its schools to consistently promote such results.