Lessons From New American Schools' Scale-Up Phase: Prospects for Bringing Designs to Multiple Schools
Jan 1, 1998
Traditional school reform efforts have focused on improving specific and isolated educational performance gaps. To address reading problems, for example, a primary school may adopt a whole language or phonics program. To raise math scores, the same school may introduce a new, technology-supported math curriculum. Often these reform initiatives are adopted by only one or two grade levels or by a few teachers, creating a patchwork of reforms within a school, or what critics have dubbed the "Christmas tree effect."
Growing dissatisfaction with the outcomes of piecemeal reform has led to the emergence of a new type of reform at the primary and secondary school levels. This new approach, comprehensive school reform, takes an integrated view of the reform process. It is based on the concept that the way to successfully improve school performance is to simultaneously change all elements of a school's operating environment so as to bring each element into alignment with a central, guiding vision.
There are a number of indications that comprehensive reform is taking hold in the nation's K–12 school system. A growing number of schools are shifting toward a comprehensive reform model, and district leaders in many school districts that have committed to comprehensive reform are looking to "school designers" for help in bringing this new type of reform to their schools. In addition, comprehensive school reform has become a critical piece of a bipartisan educational policy agenda, with the new federal Obey-Porter legislation promising financial support of up to $50,000 a year for three years to schools that adopt comprehensive school reform.
The success of comprehensive school reform depends on careful planning of school and district resource commitments. Unlike traditional reform efforts, comprehensive reform is not easily funded through a small increase in a school's operating budget. Because this type of reform touches on all aspects of a school's operating environment, it imposes significant funding requirements and demands a rethinking of the way resources are allocated.
There is currently little knowledge on the resource implications of comprehensive reform. The rush to embrace comprehensive reform has taken place with little analysis of central funding questions, including:
Drawing on data from 58 New American Schools currently implementing comprehensive reform models, this paper provides answers to each of these questions. The analysis and conclusions offered here will help educators and educational policymakers better evaluate the resource impacts of comprehensive reform.
The growing appeal of the comprehensive reform approach is rooted in the many advantages it has over traditional reform efforts. The first advantage is that its adoption encourages the termination of single-focus reforms and thus prevents the fragmentation associated with these traditional reform efforts (Glennan, 1998). At the core of comprehensive school reform is a unifying vision, or mission, for a school, one that offers an integrated approach across all grade levels, all students, and all elements of school practice (Bodilly, 1996, 1998).
The second advantage of the comprehensive reform approach is that it provides schools with access to external assistance and expertise. A number of school designers have developed comprehensive reform models for K–12 schools. These models include but are not limited to the Coalition of Essential Schools, High Schools That Work, Direct Instruction, all eight New American Schools designs, Paideia, and the School Development Program. School designers offer assistance to schools, usually for a fee, to help them in transforming themselves. They also help cultivate a network of schools implementing similar designs that can serve as informal resources to one another.
The third advantage of this type of reform approach is that it introduces quality control mechanisms often lacking from previous reform efforts. Comprehensive designs bring with them a clear blueprint for changing a school's educational standards, curriculum, and instructional practice. These blueprints not only give schools a clear path to reform but also make it easier for educational researchers to evaluate the effects of comprehensive reform efforts on educational outcomes (Bodilly, 1996; Fashola and Slavin, 1998; Ross et al., 1997; Slavin, 1995; Stringfield, Millsap, and Herman, 1997).
Recognizing all of these possible benefits of comprehensive reform, the U.S. Congress recently passed legislation intended to promote comprehensive reform efforts (Riley, 1995; Smith and Scoll, 1995; Smith, Scoll, and Link, 1996). Known as the Obey-Porter legislation, after the two sponsoring congressmen, the law established a new federal program to provide individual schools with $50,000 a year for three years to help in their efforts to adopt comprehensive school reform. In November 1997, $150 million was committed to help a first round of approximately 2,500 schools adopt comprehensive reform models.
While the Obey-Porter legislation gives an important boost to comprehensive reform, it is not clear that the level of funding it provides will be sufficient. To use Obey-Porter funds to plan effectively for implementation of comprehensive reforms, schools and school districts need access to more systematic information about the resources required by this type of reform. In particular, they need to know what the total resource requirements will be, the portion of these resource requirements that can be covered by Obey-Porter funds, and, if necessary, the sources available for supplemental funding.
RAND's ongoing evaluation of the New American Schools provides some preliminary information on these questions. New American Schools is not the only organization working to implement comprehensive school reform, but it is the single biggest. The eight design teams working under the New American Schools umbrella are currently in 1,000 schools. One part of RAND's evaluation has focused on tracking the resources dedicated to comprehensive reform and the sources of funding used to meet these resource requirements.
This paper reports on findings from the 1996–1997 academic year, which are based on data collected from a sample of 58 schools in six of the eight New American Schools designs. To develop a sense of the resource requirements created by comprehensive school reform, we collected data at each school on usage in four different resource categories: teacher time, personnel, design services, and materials and conferences.
Teacher Time. As is true for other reform initiatives, teachers in schools implementing comprehensive reform need time to learn new curricula, new teaching practices, and new management techniques, and they need time to collaborate (Purnell and Hill, 1992). To develop a sense of the time commitment being required of schools by comprehensive reform, we collected information at each of the schools in our sample on participation in planning and training activities. We counted the total number of hours teachers spent in common planning time, teacher teams, and management teams, and the total number of days teachers spent in on- and off-site design training.
Personnel. A second type of resource required by comprehensive reform is school personnel whose function is to support design implementation (Slavin, 1995; Bodilly, 1996). Many comprehensive reform designs use on-site resource experts that work with teachers to provide continual troubleshooting and coaching. These experts may include site facilitators, curriculum coordinators, and technology coordinators. Some comprehensive designs also use experts that work directly with students to provide instruction or with families and other community members to build bonds to the local community. These experts might include reading tutors, instructional assistants, and family outreach personnel. At each of our schools, we collected data on the total number of full-time equivalents (FTEs) dedicated to comprehensive reform for each of these categories of personnel.
Design Services. A third type of resource often required is school designers, or design teams, to help schools work through implementation. Design teams (such as those that have developed the New American Schools models) bring considerable expertise on new curricula, standards, assessments, instructional practices, and governance techniques. They also act as an external change agent and help build the momentum for change that schools find difficult to generate on their own. At each of the schools in our sample, we collected information on the costs associated with providing design services. In particular, we collected information on the costs of consulting days, airplane flights, hotel nights, per diems, and participation fees.
Materials and Conferences. A fourth category of resources required by comprehensive reform is materials and conferences. Some comprehensive designs require new instructional materials; others require student field trips. Most comprehensive designs involve teacher participation in design conferences and visits to model schools, both of which present opportunities for teachers to network with and learn from other teachers implementing the same design. To gather data on materials costs, we collected information on the costs of the teacher books and student notebooks used to support instruction and on the entrance fees and bus rental costs for student trips. To gather data on conference costs, we collected information on the costs of airplane trips, hotel nights, and teacher per diems.
Once we had developed a sense of the total resource requirements associated with comprehensive reform, we used a costing methodology (see Appendix for details) to assign a total dollar value. We found that the average resource use for comprehensive reform across all 58 schools in our sample was $162,000 per school in the first year of implementation (see Figure 1). The average school in our sample had 40 teachers and 740 students. To support design implementation, this average school used 1.8 hours of planning time per teacher per week, 6.5 days of training per teacher per year, 1.7 FTE dedicated school personnel, $25,000 for design services, and $12,000 for materials and conferences.
As the figure shows, 76% of the resource usage stemmed from the combination of teacher time for planning and training and the use of design-dedicated school personnel. The average combined value for these two resources was $125,000—$66,000 for teacher time and $59,000 for personnel. Design services averaged $25,000, or about 16% of total implementation costs, and materials and conferences averaged $12,000, or the remaining 8%.
Several important qualifiers need to be made about our estimate of average resource requirements, however. First, resource requirements are not the same as out-of-pocket costs. Our average school did not actually pay $162,000 to implement a comprehensive reform design. Some portion of the resource requirements was "paid" by schools by reallocating existing funding. (The role of reallocated funding is discussed in the next section.)
Second, the $162,000 figure is our calculation for the academic year 1996–1997. At the time of our analysis, several design teams were still figuring out their true costs of providing consulting services and were in effect subsidizing schools by not charging full cost. These teams have now raised the price of their consulting. We suspect that the total average resource requirements currently are closer to $180,000.
Third, the average resource figure masks variation in the range of resource use at schools in our sample. The main reason for variation in resource requirements is the type of comprehensive reform design a school adopts. Two of the six designs in our sample had average resource requirements of about $100,000, three clustered between $150,000 and $170,000, and the final design, which was particularly personnel intensive, required approximately $300,000. School size is also a contributor to variation in resource requirements, though much less so than design type. When implementing the same design, smaller schools use somewhat fewer resources and larger schools use somewhat more.
The second part of our analysis focused on identifying funding sources that schools draw on to meet the resource requirements associated with comprehensive reform. We collected data on two basic types of funding strategies: reallocation of a school's existing resources and the use of funds outside a school's existing budgets.
Under resource reallocation, we included the use of funds from a school's normal district allocation for personnel, materials, staff development, and discretionary budgets. For example, the use of in-service days or substitute dollars from a school's budget to cover teacher training time was considered to be reallocation of funds. So too was the shifting of personnel already on a school's payroll to new personnel duties required by the design, as well as the use of discretionary budgets to pay for travel, materials, and conferences.
With regard to the use of funds outside a school's existing budget, we collected data on the use of federal Title I funds, district contributions above the normal budget allocation to schools, grants from other outside sources, and volunteer contributions (i.e., teacher contributions of unpaid time for planning and training, and parent contributions to the costs of student trips). Title I funds, normally controlled at the school level, were tracked separate from the rest of a school's budget because many schools in our sample did not have access to these dollars.
Our analysis revealed that about $62,000, or 38%, of the $162,000 average total resource burden for implementing comprehensive reform was met through reallocation—of a school's existing personnel, substitute dollars, and materials budgets (Figure 2). The other $100,000 came from sources outside a school's normal operating budget—$53,000 from Title I, $30,000 from district budgets, $11,000 from outside grants, and $6,000 from volunteer sources.
In looking at the sources of funding within our resource categories, we found that reallocated funds went disproportionately to paying for teacher time (Figure 3). Most schools effectively reallocated their existing planning time and in-service days: 78% of the funding for teacher time came through reallocation. Of the funding for school personnel, very little came from reallocation, while a total of 90% came from outside sources—76% from Title I money alone.
For design services, district budgets and outside grants were the most important sources of funding, providing a total of 84% of funds—58% from district budgets and 26% from grants. Materials and conference costs were covered by all five funding sources, with approximately 84% coming from sources other than existing school budgets.
Our conversations with school principals and district officials in the course of collecting our resource data suggest that school districts play a particularly important role in helping schools find funding for comprehensive reform. With many competing resource priorities and fragmented reform efforts, schools may not be able to effectively reallocate their existing resources without district leadership. Indeed, our analysis indicated that reallocation of resources to support comprehensive reform was more likely to occur if district officials clearly signaled that such reform was a key district priority and that reallocation was not only permissible but encouraged (Bodilly, 1998).
In addition to encouraging schools to reallocate resources to support comprehensive reform, districts played a central role in providing schools with further funding. A district's own general funds were an important source of money for design services and for materials and conferences. Moreover, districts took the lead in finding external grant money that schools then used to implement comprehensive reform.
Our analysis of the resource issues at the 58 schools in our sample led us to four conclusions about funding comprehensive reform:
Resource reallocation is key to funding comprehensive reform. Nearly 40% of the funds for comprehensive reform at the schools in our sample came from reallocated resources. Failure to reallocate resources can raise the costs of implementing these comprehensive designs by tens of thousands of dollars, which may make implementation prohibitively costly.
Access to Obey-Porter funds should allow most Title I schools to fully fund comprehensive reform. Obey-Porter funds are intended to supplement, not replace, the Title I allotment. Many schools receiving Obey-Porter funds will also be Title I schools. Assuming effective resource reallocation, the addition of $50,000 in Obey-Porter funds will allow most Title I schools to implement comprehensive reform without needing additional resources. Our analysis suggests that an "average" school can generate $115,000 from its own operating and Title I budgets before receiving Obey- Porter funds.
Access to Obey-Porter funds will not be sufficient for schools without Title I funds. The combination of $50,000 in Obey-Porter funds and $62,000 in reallocated resources will in most cases leave a non-Title I school considerably short of what it requires to implement comprehensive reform. The needed funds will have to come in the form of additional district funding and/or outside grants.
District leadership is crucial in funding comprehensive reform. School districts are important to schools not only because they provide funds directly from their own general funds or by securing external grants, but because they encourage schools to reallocate school budgets and available Title I funds for comprehensive reform. District leadership plays an essential role in keeping schools from viewing comprehensive reform as an add-on program.
Bodilly, Susan J. (1998). Lessons from New American Schools' Scale-up Phase: Prospects for Bringing Designs to Multiple Schools. Santa Monica, CA: RAND, MR-942-NAS.
Bodilly, Susan J. (1996). Lessons from New American Schools Development Corporations' Demonstration Phase. Santa Monica, CA: RAND, MR-729-NASDC.
Fashola, Olatokunbo S., and Robert E. Slavin (1998). "Schoolwide Reform Models: What Works?" Phi Delta Kappan 79(5): 370–379.
Glennan, Thomas K., Jr. (1998). New American Schools after Six Years. Santa Monica, CA: RAND, MR-945-NASDC.
Purnell, S., and P. Hill (1992). Time for Reform. Santa Monica, CA: RAND, R-4234-EMC.
Riley, R. W. (1995). "Reflections on Goals 2000." Teachers College Record 96(3): 380–388.
Ross, S., A. Troutman, D. Horgan, S. Maxwell, R. Laitinen, and D. Lowther (1997). "The Success of Schools in Implementing Eight Restructuring Designs: A Synthesis of First Year Evaluation Outcomes." School Effectiveness and School Improvement 8(1), March.
Slavin, R. (1995). "Bricks, Sand, and Seeds: School Change Strategies and Readiness for School Reform." Mimeo, Center for Research on Education of Students Placed at Risk, Johns Hopkins University, June.
Smith, M. S., and B. W. Scoll (1995). "The Clinton Human Capital Agenda." Teachers College Record 96(3): 389–404.
Smith, M. S., B. W. Scoll, and J. Link (1996). "Research-Based School Reform: The Clinton Administration's Agenda." In Improving America's Schools: The Role of Incentives, edited by E. A. Hanushek and D. W. Jorgenson. Washington, DC: National Research Council, 9–28.
Stringfield, S., M. A. Millsap, and R. Herman (1997). Special Strategies for Educating Disadvantaged Children: Findings and Policy Implications of a Longitudinal Study. Washington, DC: U.S. Department of Education.
The resource analysis described in this paper was based on a survey of 58 schools in five districts that are implementing New American Schools designs. The sample included 10 schools in Cincinnati, 15 schools in Dade, 17 schools in Memphis, eight schools in Philadelphia, and eight schools in San Antonio. We attempted to make the sample representative by selecting out the "highest" and "lowest" implementers in each district so as to focus on the typical, or average, school. We also left out both partial implementers and schools having extraordinary resources available to them.
We conducted a 45-minute phone survey with the school principal or the principal's designated representative at each school. The survey focused on the school's levels of resource use and funding strategies. The school-level data collection was complemented with interviews of district officials and design team representatives. District interviews were used to gather more information about available funding sources; design team interviews were used to confirm a school's estimates of resource use and the costs of design services.
The estimates of resource use and funding strategies presented in this paper are based on a methodology that involves four analytical steps. First, we observed the major types, or "units," of resource use in each of our four main resource categories: teacher time, personnel, design services, and materials and conferences. Second, we carefully tracked through the school survey units of resource use in each category. Third, to develop cost estimates, we assigned a standard cost figure to every resource unit. Fourth, we developed estimates of the relative contributions of different funding streams to supporting resource investments by collecting data on the sources of funding for each individual unit of resource use.
Our resource units for teacher time included hours of planning and days in training. We counted the total number of hours teachers spent per week in common planning time, teacher teams, and management teams, and the total number of days teachers spent in on- and off-site design training. We figured the total amount of teacher time devoted to each design by calculating a day equivalent for all teacher time spent in planning and then adding this day equivalent to the number of days spent in training. For this calculation, we assumed that a teacher day was 7.25 hours long. To arrive at a cost for teacher time, we assigned a standard cost figure of $110 to a teacher day. We arrived at the $110 figure by assuming an even split in use of substitute days (which averaged $90 across all districts) and stipend days (which averaged $130 across all districts).
Under the personnel, or school expert, category, our only resource unit was full-time equivalents (FTEs) of school-level personnel working to coordinate or implement a design. To arrive at a total FTE count, we included the percentage of time devoted to the design by all site facilitators, curriculum coordinators, technology coordinators, reading tutors, and family outreach personnel. To calculate the cost of personnel, we assigned a standard annual cost figure of $45,000 to a single FTE. This was the average of salary and benefits across the five districts in our sample.
For design services, our resource units were consulting days, airplane flights, hotel nights, and days of per diem for design consultants, and fees for teacher participation in design-sponsored training. To develop cost estimates for consulting days and participation fees, we used cost data from the design teams. For airplane flights, hotel nights, and per diems, we collected actual costs where possible. Where no actual cost data were available, airplane flights were assigned a standard cost of $450, and a hotel night and day of per diem were assigned a combined standard cost of $200.
For materials, our resource units were the teacher books and student notebooks used to support instruction and the entrance fees and bus rental costs for student trips. Cost figures for all resource units coming under materials came from the schools' own records. For conferences, we used airplane trips, hotel nights, and teacher per diems as our resource units. We figured a total number of resource units for each category by counting the number of teacher participants in the designs' national conferences, the designs' principals' conferences, and site visits to demonstration schools implementing a design. To develop cost estimates, we used school records on the costs of airplane tickets and hotels and on dollars budgeted for stipends. Where this information was not available, we used standard cost figures of $450 for airplane tickets, $100 for hotel nights, and $30 for per diems.
It is worth noting that we collected but did not systematically analyze information on investments in educational technology, a possible fifth resource category. Technology is important to both the Co-Nect and the Modern Red Schoolhouse design, but our early research findings suggested that it is an extraordinary type of resource. Through effective reallocation of resources and infusions of additional resources, schools can reasonably be expected to eventually pay for the teacher time, personnel, design services, and materials and conferences associated with the New American Schools designs. The same is not true for technology. Running an effective technology program can cost up to several hundred thousand dollars annually for a single school. In our view, creating an appropriate technological infrastructure is related to but separate from the New American Schools implementation. The more technologically intensive of the New American Schools designs will succeed only when introduced into a district with a well-developed strategic technology plan.