Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief

Abstract

In its comprehensive evaluation of Edison Schools, the nation's largest for-profit manager of public schools, RAND found that the resources and accountability systems that constitute Edison's design represent a coherent, comprehensive, and ambitious strategy to promote student achievement. The best-functioning Edison schools demonstrate the promise inherent in Edison's model, but the performance of Edison schools varies. Average gains of Edison schools during the first three years of Edison management do not exceed the gains of comparison schools, but many Edison schools outgain comparison schools after four or five years.

New forms of governing and managing public schools have proliferated in recent years, spawning the establishment and growth of companies that operate public schools under contract. Among these education management organizations, or EMOs, the largest and most visible is Edison Schools, Inc. In 2004–2005, Edison managed 103 public schools in 18 states and the District of Columbia, serving approximately 65,000 students, most of whom are nonwhite children from low-income families in urban centers. Some of Edison's schools are public schools that preexisted and were contracted to Edison by districts, whereas others are charter schools that Edison helped to start up. The RAND Corporation designed an evaluation of Edison to address the following research questions:

  • What are Edison's strategies for promoting student achievement in the schools it manages?
  • How are Edison's strategies implemented in the schools it manages?
  • How does Edison's management of schools affect student achievement?
  • What factors explain differences in achievement trends among Edison schools?

To address these questions, RAND researchers interviewed Edison leaders, reviewed documents, visited 23 Edison schools to interview teachers and principals, and analyzed school-level achievement trends in Edison schools and in conventional public schools with similar student populations and initial performance levels. The results of the research are documented in Inspiration, Perspiration, and Time: Operations and Achievement in Edison Schools.

Inspiration: Edison's Strategies for Promoting Student Achievement

Edison distinguishes itself from most other school improvement strategies (e.g., school choice, high-stakes testing, comprehensive school reform, class-size reduction, teacher development) by simultaneously addressing resources and accountability systems, rather than focusing on one or the other. Together, the resources and accountability systems that constitute Edison's design represent a coherent, comprehensive, and ambitious strategy to address key elements relevant to providing high-quality education, including capacities, motivation, and opportunities for school staff.

The resources Edison provides to its schools include curriculum, technology, and professional development. Edison aims to raise student achievement as measured by state high-stakes tests in reading and math, while also offering a broad and rich curriculum that includes fine arts and a foreign language for students in all grades. The company has developed a proprietary system of monthly, online assessments known as the Edison Benchmarks, designed to provide instant information to teachers and principals about students' academic strengths and weaknesses. Its professional development program includes not only summer training for teachers and principals but also school-based professional development that relies on teacher collaboration and common planning time to promote a strong professional environment.

Edison's accountability systems are distinguished from those of conventional public schools primarily by their greater focus on outcomes-based accountability—measured mostly by student achievement and school budget management—and reduced emphasis on bureaucratic and political accountability. Edison aims to provide financial rewards to staff in high-performing schools, while seeking to insulate its schools from some of the buffeting of local political winds that may result from turnover in superintendents or school boards.

In sum, Edison's well-developed information systems and its focus on achievement-based accountability should make it especially well suited to the high-stakes testing environment of the federal No Child Left Behind (NCLB) Act.

Perspiration: Implementation of Edison's Strategies in Schools

The comprehensive ambitions implicit in Edison's model suggest that successful implementation requires wholehearted commitment—and hard work—from Edison, its clients, and the staff of its schools. In fact, the best-functioning Edison schools demonstrate the promise inherent in Edison's model. These schools have strong instructional leadership, motivated teachers, effective use of achievement data, high-fidelity implementation of the Edison curricula, and high levels of professional collaboration. Nearly all the Edison schools we visited across the country showed enough consistency of implementation to be clearly recognizable as Edison schools, but we observed considerable variation in the extent to which the schools realized the Edison ideal.

Across Edison schools, the reading and math curricula appeared to be implemented more faithfully than were other subjects that are not regulated by state testing systems. In some schools, notable elements of the Edison program, such as common planning time and a longer school day, were sacrificed as a result of resource limitations or contractual constraints. Edison's ability to insulate its schools from local politics appears to be greater in its charter schools than in its district schools, as suggested by the longer endurance of its management contracts with charter schools.

Among the Edison schools we visited, several factors appear to be important in explaining some of the variation in implementation of the Edison model. In particular:

  • Full implementation of the Edison design takes time—as might be expected in the implementation of a comprehensive, ambitious reform. Schools in their first year of operation encountered frequent challenges in implementing various elements of the design.
  • Strong instructional leadership by the principal is associated with stronger implementation of the curriculum, not only in subjects that are the focus of states' high-stakes testing programs (reading and math) but also in other areas of the curriculum (science, social studies, foreign language, and the arts).
  • Among the case study schools, strong instructional leadership by principals appeared to be somewhat more prevalent in charter schools than in district contract schools. But charter status did not appear to be directly related to curriculum implementation.
  • Local constraints—for example, those resulting from compromises required by local contracts—sometimes undermine the implementation of Edison's preferred professional environment.

These findings bear out the importance of the sustained commitment of clients and school staff in promoting effective implementation of the Edison model.

Time: Effects of Edison Management on Student Achievement

We attempted to assess Edison's effect on student achievement by seeking evidence about what schools might have achieved if they had not been managed by Edison. This assessment involved two key components. First, we examined changes over time to help control for prior achievement levels of students served by Edison schools. Second, we compared achievement trajectories in Edison schools (using proficiency rates on state tests in reading and math) with achievement trajectories in a matched set of conventional public schools serving similar populations of students with similar baseline test scores.

In absolute terms, Edison schools are making gains: Average rates of reading and math proficiency in Edison schools have improved over time. But relative to matched comparison schools, the picture becomes more complicated. Average gains of Edison schools during the first three years of Edison operation have not exceeded the gains of matched comparison schools. Edison results relative to comparison schools improve in years four and five, but whether those improvements ultimately yield net positive effects is the key question. The answer depends on the specific analysis used.

The analysis with the most comprehensive set of Edison schools suggests that most Edison schools are exceeding the gains of their matched comparison schools by the fourth and fifth years after the initiation of Edison management, particularly in math. However, this analysis uses the spring of Edison's first year of management as the starting point and therefore excludes any effects that took place during the first year. Lacking a pre-Edison baseline, the analysis is comprehensive in its inclusion of Edison schools but not comprehensive in its coverage of the time period of Edison management.

Results from pre-Edison baselines are available only for a subset of Edison's schools, because a substantial number of Edison's schools (typically charter schools) were new start-ups that did not exist prior to Edison's management. In consequence, this analysis includes only conversion schools (i.e., schools that existed prior to being managed by Edison), and in the first three years it is dominated by a large number of schools from only three districts (Chester, Pennsylvania; Philadelphia; and Dallas) and therefore does not accurately represent the range of Edison's clients.

On average, Edison conversion schools have fallen behind comparison schools in the first year of Edison operation. As a result, Edison conversion schools on average show smaller gains than comparison schools over the first three years of operation and gains after four or five years that may match but not exceed those of comparison schools.

The evidence of a first-year decline for conversion schools, combined with a lack of first-year information for start-up schools, leaves us with uncertainty about whether gains of Edison schools after four or five years are comparable to or superior to those of matched comparison schools. Given this uncertainty, an examination of differences in achievement among Edison schools, and the factors that might explain those differences, is particularly important.

Understanding Variation in Performance Among Edison Schools

Estimates of average performance across all Edison schools obscure variation in the achievement trajectories of individual Edison schools. This variation is extensive, with some schools exceeding the performance of comparison schools and others falling behind. Some understanding of the factors that explain this variation among Edison schools is necessary for predicting how any particular Edison school (or prospective Edison school) might perform in the future.

We examined several factors that might be related to the variation in performance among Edison schools. First, we assessed whether Edison is producing better results systemwide in recent years than it did in its early years of operating schools. Schools that opened in 2000 or later showed slightly larger relative gains in mathematics than those that opened earlier, but no difference in relative gains between newer and older Edison schools was observed for reading.

We also used findings from the case study schools to shed light on differences in achievement trajectories among Edison schools. Case study findings are not definitive, because the sample is small and relationships can be measured only in simple, correlational terms, but they are nevertheless suggestive:

  • Edison case study schools with stronger curriculum implementation had stronger achievement results in both reading and math.
  • Moreover, implementation of the Edison curriculum in subjects other than reading and math (science, social studies, foreign language, and the arts) was correlated with stronger achievement results in reading and math, suggesting that schools need not neglect the broader aspects of the curriculum to achieve gains in basic skills.
  • Case study schools that more fully implemented the Edison professional environment (including the on-site professional development, planning time, and house teams) had stronger achievement results in both reading and math.
  • Edison schools with stronger instructional leaders as principals had somewhat better achievement results than those that did not.
  • Finally, limited evidence suggests that Edison schools operating with fewer local constraints on the model, and where principals have full authority over hiring and firing teachers, may have better achievement trends.

Recommendations

Together, our achievement results and our case study observations suggest some actions that Edison and its clients can take to improve the likelihood of successful implementation. We present two sets of recommendations: one for Edison and another for district staff and other policymakers considering hiring Edison.

For Edison

  • Provide improved support and oversight during the first year.
  • Apply value-added assessment methods to identify which schools and teachers have been most successful at improving the performance of individual students and which schools and teachers appear to need more assistance in this regard.
  • Continue to promote a comprehensive vision of the curriculum.
  • Take further steps to ensure the development of principals' instructional leadership skills.
  • Avoid compromises to the design that may undermine the professional environment in the schools.

For Clients and Prospective Clients

  • Manage the transition with care.
  • Give Edison full authority to implement its design.
  • Ensure that teachers and principals support the model.
  • Do not expect instant improvement.
  • Develop data systems that facilitate assessment of individual students' achievement gains over time, to permit better understanding of the performance of each Edison school.
  • Carefully consider the incentives created by state and local accountability systems.

As the largest private manager of public schools, Edison's experiences provide a model to help policymakers and members of the public understand the benefits and limitations of nontraditional forms of school management. Interest in alternative management may increase as the charter school sector continues to grow and as schools and districts that fail to meet their annual targets under NCLB face some of the more severe sanctions of the law. Our analyses provide evidence regarding what has occurred in schools that have been turned over to Edison management, in terms of both how the program is implemented in schools and what happens to student achievement over time.

Unfortunately, data limitations—particularly the lack of information about the first year in start-up schools and the small numbers of schools that have been managed by Edison for at least four years—render the historical evidence on Edison's net long-term effects equivocal. Predicting future long-term effects is especially challenging because it involves assumptions about the stability of Edison's management approach as well as the kinds of schools and districts with which Edison is likely to work in the future. Moreover, there was extensive variability in the results achieved by individual Edison schools. We therefore cannot make strong predictions for prospective clients about whether they will achieve better long-term results with Edison or with an alternative approach. Nevertheless, Edison's improving trends are encouraging, and some schools have thrived under Edison management, indicating that Edison is capable of producing favorable results. Edison and its current and future clients must work to establish the conditions in its schools to consistently promote such results.

This report is part of the RAND Corporation research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.