Comprehensive School Reform

Under pressure to improve student achievement, schools throughout the nation have, over the past 20 years, turned to comprehensive school reform (CSR). CSR is based on the idea that a school should have a coherent educational strategy that addresses all aspects of its operations and aligns them in a well-functioning delivery system.

Hundreds of CSR models have been developed. More than 8,000 schools have adopted a CSR model, and more than $2 billion in federal funds have been used to implement CSR. CSR is one of a limited set of interventions that the No Child Left Behind Act explicitly allows for schools that need to improve their performance. Nevertheless, CSR's effectiveness continues to be hotly debated.

Does CSR work? Research results have been mixed. Some studies have measured a modest improvement in student achievement; others have found no effect.

A team of RAND researchers has approached the question of CSR's effectiveness by first focusing on an even more basic question: Has CSR been implemented? A shortcoming of nearly all previous studies is that they have assumed that schools have implemented their CSR model in its entirety.

But what if most schools have implemented CSR only partially or not at all? In such cases, improved student achievement cannot be expected—or, if it does occur, cannot be attributed to CSR. Until we can measure the level of implementation, we cannot determine whether CSR works, or whether one CSR model works better than another.

Measuring Implementation

RAND developed a unique five-step methodology, incorporating surveys and in-depth case studies, to quantitatively measure the level of CSR implementation. The study team then used the methodology to measure actual implementation of four different CSR models in a large number of schools. For comparison, the study also included a sample of schools that did not use any CSR model.

Four CSR models are included in this study: Accelerated Schools, Core Knowledge, Direct Instruction, and Success for All. These models were chosen because they have been widely selected by schools throughout the nation, and because they differ from each other significantly.

Each of the four CSR models is based on a different philosophy and a different set of prescribed practices. Each model, however, generally emphasizes and attempts to align six core areas of schooling: curriculum, methods of instruction, appropriate student grouping, governance (such as establishing a school steering committee and working groups), student assessments, and parent involvement. The importance and configuration of each area vary by model.

Most CSR models were developed by educational consultants. The study team used discussions with the model designers to develop indicators for each model's implementation. Surveys were sent to principals and teachers in a sample of 250 model schools in two states (Florida and Texas). Surveys were also sent to a set of comparison schools that had not implemented a CSR model. To measure any changes in implementation over time, surveys were sent in each of three consecutive years. Finally, the surveys were complemented with in-depth case studies in 12 schools to obtain more insight into the implementation process.

No School Had Fully Implemented Its CSR Model

No school in our study had fully implemented all core components of its chosen CSR model as envisioned by the designers. Some core components were implemented more widely than others. For example, schools were generally able to implement the prescribed curriculum. But they had more difficulty in following the prescribed instructional practices. In comparison with other core components, practices to increase parental involvement were consistently implemented at the lowest level. Year-to-year comparisons indicate that the level of implementation of each component, and the model as a whole, did not change over time.

CSR Implementation Support Falls Short of Recommended Levels

Model designers typically prescribe a high level of support to ensure that the model is implemented successfully. Such support includes external support (providing for principal and teacher consultation with the model designers, teacher training, and ongoing professional development) and internal support (appointing a school staff member to facilitate and coordinate implementation).

Most schools did not have the recommended implementation support. For example, on average, teachers reported receiving about half of the recommended initial training and about one-quarter of the recommended ongoing professional development. In general, teachers reported only a lukewarm commitment to implementing the school's model, and most teachers felt the training they had received did not fully prepare them to start using the model. However, when the level of support increased, so did the level of implementation.

Model and Non-Model Schools Are Very Similar

Survey results revealed a surprise: Most schools were engaged in many of the same types of activities regarding curriculum, methods of instruction, student groupings, governance, assessment of students, and parent involvement—regardless of whether they used one of the four models or not. And, on average, all schools engaged in these activities at the same frequency or level of intensity.

However, a handful of practices differed between types of model schools and between model schools and their matched non-model schools. The practices that were implemented at higher levels of intensity or frequency when specifically prescribed by a model included students working collaboratively, teachers' adherence to a word-for-word-script, placing students in groups by level of performance, assigning daily homework, and obtaining parent signoff on that homework. Also, parent involvement was higher in most model schools than in non-model schools.

At the Current Level of Implementation, CSR Can Be Expected to Have Little Effect

Research to date has found that the implementation of a CSR model has had modest, if any, effect on student achievement. However, our study indicates that very few schools using these four models have fully implemented them. Assuming that these circumstances also apply to other models and their implementation, it is not surprising that CSR has had little effect.

There is, however, room to increase the level of CSR implementation. For instance, schools could ensure that teachers are committed to implementing the adopted model and that they receive the necessary initial training. Teachers could be allowed to practice the prescribed changes before they implement them in the classroom. Ongoing professional development related to the CSR model could be increased initially and maintained as long as the school uses the model. The level of interaction with the internal facilitator and the model designer could also be increased.

If CSR is more fully implemented, a key area for future investigation will be to determine whether CSR is effective and, specifically, which CSR practices are the most effective. It will then be possible to determine if CSR is a useful tool in meeting the proficiency goals of the No Child Left Behind Act.

Topics

• Children, Families, and Communities
• Education and Literacy
• Education Reform