Advancing Systemwide Instructional Reform

Lessons from Three Urban Districts Partnered with the Institute for Learning

Research Brief

Abstract

 

Improving school systems is critical to bridging the achievement gap and achieving federal accountability goals. Research in three urban districts partnered with a university-based intermediary organization sheds light on promising instructional reform strategies and challenges to bringing about systemwide change. Analyses of district efforts to promote the instructional leadership of principals, support teacher learning through school-based coaches, specify curriculum, and promote data-based decisionmaking identify common factors constraining and enabling instructional improvement. The research also shows that third-party organizations can help facilitate policy alignment and build the capacity of district staff to lead instructional change.

Urban school systems face immense challenges in trying to meet the goals of the No Child Left Behind Act (NCLB) and in trying to erase the achievement gap between minority and nonminority students. A new RAND study, funded by the Hewlett Foundation, offers insights and lessons learned from the efforts being made by three urban districts to improve the instructional quality and performance of their schools. These districts had partnered with an intermediary organization, the University of Pittsburgh’s Institute for Learning (IFL), as part of their reform strategy.

The study sought to answer four broad questions:

  • What strategies did districts use to promote instructional improvement? How did these strategies work?
  • What constrained or enabled district instructional improvement efforts?
  • What was the impact of the IFL? What constrained or enabled the district-IFL partnerships?
  • What are the implications for district instructional improvement and district-intermediary partnerships?

The study sample consisted of three urban school districts that had made districtwide instructional improvement a high priority, had worked with the IFL for at least three years, and were willing to participate in the study. The districts varied in district size, union environment, and state. Data were collected through extensive field interviews and focus groups conducted over a two-year period; RAND-developed surveys of elementary, middle, and high school principals and teachers; district and IFL documents; and demographic and student achievement databases.

Findings
 

District Instructional Improvement Strategies

In the three districts, instructional reform efforts centered on four common areas of focus: building the instructional leadership skills of principals; supporting the professional learning of teachers, with a particular focus on school-based coaching; providing greater specification and support for standards-aligned curriculum; and promoting the use of data to guide instructional decisions. Although all three districts pursued strategies within each of these areas, each tended to focus on two of the four key areas of reform to leverage change throughout the system.

Instructional leadership. All three districts attempted to increase principals’ instructional leadership capacity (e.g., knowledge about teaching and learning, ability to observe instruction and provide feedback to teachers, skills for setting improvement goals and reviewing data). All three sites provided principals with professional development and expected principals’ supervisors (who typically had titles such as area or assistant superintendent) to focus their school visits and meetings with principals on matters related to improving instruction.

Despite a relatively consistent focus on instructional leadership among districts and over time, principals varied greatly in the extent to which they were reported to act as instructional leaders. Our data suggest that several factors help explain this variation: quality of professional development for principals and supportive supervisors, lack of time for principals to carry out instructional leadership tasks given the managerial and disciplinary demands on their time, and lack of perceived legitimacy of some principals as instructional leaders.

School-based coaching. Two of the districts invested in school-based instructional coaches as a means of providing ongoing, job-embedded professional development for teachers, but they implemented different models.

Teachers tended to prefer a more flexible, school-centered approach to coaching rather than the relatively standardized curriculum-centered one, although both models were intended to build the instructional capacity of schools and to support implementation of district initiatives and were generally viewed favorably by teachers. The perceived value and effectiveness of interactions with coaches was greater when coaches tailored their work to school and/or teacher needs and advised teachers about instruction (rather than simply delivering messages from the district). Time to meet with teachers and clearly defined roles also facilitated coaches’ effectiveness.

Curriculum specification. All three districts developed and implemented curriculum guidance documents that were intended to improve alignment of instruction with state standards and assessments, and to increase consistency of instruction across classrooms and schools by specifying districtwide guidelines for the scope, content, and pacing of curriculum. Leaders in two districts viewed these curriculum guides as a major focus of their improvement efforts, investing significant resources in developing and monitoring teachers’ use of the documents.

District and school staff generally viewed the curriculum guides as useful for planning, promoting consistency of instruction, and helping principals observe and monitor teachers; but teachers reported a limited impact on pedagogy. That is, the guides influenced “when” and “what” teachers taught, but they did not make major shifts in “how” they taught the curriculum. Teachers were apt to value and use the guides when (a) they perceived them to be aligned with state tests, (b) the curriculum materials were delivered to them in a timely manner, and (c) the development process solicited and incorporated teacher input. However, many teachers in all districts described the pacing and content of the guides as conflicting with their need to tailor instruction to individual students (e.g., the pacing did not allow enough time for teachers to reteach concepts to students who did not master them the first time).

Data use. Although all three districts invested to varying degrees in multiple strategies to promote the use of data to guide instructional decisions — such as providing professional development on how to interpret test results and encouraging structured reviews of student work — two districts focused much more on the use of data. One district emphasized the school improvement planning (SIP) process. The district required schools to use a comprehensive, district-designed SIP template to examine student data, identify areas needing improvement, and develop strategies to address those needs. Unlike the SIP process in the other two districts, this district provided additional ongoing school-based support. The second district focused on interim assessments designed to provide an “early warning system on progress being made” toward meeting state standards. The assessments were aligned with the curriculum guides and linked to a sophisticated data management system intended to provide school staff with quick access to results.

Teachers and principals in both districts generally found the various sources of data useful and reported regularly using them to identify areas of weakness and to guide instructional decisions. Principals and teachers in the district that focused on the SIP process, however, described the process as overly labor-intensive. Furthermore, teachers in the district that focused on interim assessments were less enthusiastic about these assessments than principals, preferring more-timely, regular classroom assessment data. Variation in data use among and within districts was affected by the degree to which teachers viewed the assessment results as valid measures of students’ knowledge and ability, as well as the degree to which school staff received training and support for analyzing and interpreting data.

Constraints and Enablers of Instructional Improvement

A number of common factors affected districts’ success in achieving their intermediate goals for instructional reform.

Comprehensive focus on a small number of initiatives. Although it seems counterintuitive, a combination of comprehensiveness and focus facilitated instructional reform in all three districts. Comprehensiveness encompasses a systemic approach (e.g., non-piecemeal strategies targeting an array of principal and teacher professional development and supervision, accountability and data), strategies addressing all dimensions of instruction (e.g., school-based coaching addressing pedagogy and curriculum guides targeting content), and a dual focus on infrastructure and direct instructional support (e.g., enhancing data systems or revising supervisory structures while also providing direct teacher training). Focusing on two key areas of reform complemented these efforts by sending a clear and consistent message about district priorities and by channeling limited district and school resources to a finite number of areas. However, districts also faced significant trade-offs by deciding to focus on certain areas of reform (e.g., limiting their ability to respond to new areas of need).

District and school capacity. All three districts nonetheless faced significant capacity gaps that hindered instructional improvement efforts. According to district and school staff across the sites, these included: insufficient time (e.g., for planning, to act as instructional leaders); lack and/or instability of fiscal or physical resources (e.g., instructional materials, funding); and limited capacity of central office staff (e.g., inadequate numbers, lack of expertise). At times, however, the districts adeptly capitalized on capacity strengths, such as assistance from knowledgeable district staff and on-site instructional specialists, which enabled district work.

Broader policy context. At the local level, some union policies hindered reform in two districts, most notably by limiting the time available for teachers and coaches to work together. In addition, state and federal accountability policies shaped much of the districts’ work on curriculum and data use. These accountability policies created strong incentives for staff to examine student achievement data (more so in the two districts with a long history of high-stakes, state accountability systems), to specify linkages between state standards and curriculum, and to follow curriculum pacing plans to cover state standards adequately before state testing.

Key attributes of the district policies. District progress at achieving intermediate instructional improvement goals hinged in large part on the degree to which strategies:

  • Were aligned and mutually supportive. Ensuring both horizontal alignment (within levels of a system) and vertical alignment (across levels of a system) required considering perspectives of multiple stakeholders, who often differed in their perceptions on alignment. As we found in several districts, perceived misalignment frequently frustrated some stakeholders and negatively affected buy-in.
  • Enabled multiple stakeholders to engage in reform. Although leaders in all three districts tried to design reform strategies and policy tools that simultaneously addressed the needs of multiple stakeholders and involved them in the reform effort, most found it difficult to achieve this goal. They often attained greater success engaging one group than another.
  • Balanced standardization and flexibility. Perceived lack of flexibility frequently decreased buy-in for district reforms. In many cases, however, district leaders intended a greater degree of flexibility than school staff believed there was — and the district faltered in conveying that message to schools.
  • Used local accountability policies to provide incentives for meaningful change. In some instances, district efforts to hold teachers and principals accountable for implementing certain initiatives promoted implementation. For example, the use of classroom visits to monitor the use of curriculum guides in two districts communicated clear expectations that teachers would follow the pacing plans in these documents. In other instances, however, accountability tools that either were superficial (e.g., checklist-driven principal evaluation instrument) or created incentives for “gaming” (e.g., interim assessments used to rank schools) did not motivate staff to change their practice meaningfully.

Impact of IFL on District Instructional Reform

Partnerships with the IFL also contributed to district reform in all three districts. In exchange for a fee, the IFL provided districts with a variety of resources, including on-site technical assistance from IFL resident fellows, opportunities to attend national meetings with other partner districts, advice from IFL leaders, and access to research, materials, and other tools. Thus, the IFL did not present an intervention or model for districts to implement but instead acted as a “reform coach,” assisting each district with various aspects of instructional improvement appropriate to its local context.

Overall, district and school leaders reported that the IFL affected the organizational culture, norms, and beliefs about instruction. District leaders reported shifts in beliefs and norms around a set of ideas emphasized in IFL materials, professional development, and technical assistance. These included notions of effort-based intelligence, two-way accountability, organizational focus on instruction and learning, the idea that everyone is a learner, and instruction as a public endeavor.

The IFL was credited with helping develop the knowledge and skills of central office and school administrators. The majority of principals in all three districts reported that professional development opportunities organized by the IFL and the districts improved their skills as instructional leaders, deepened their knowledge about learning, and provided them with a common language facilitating dialogue. Similarly, central office leaders and staff reported that IFL staff pushed them to focus on instruction and system-level structures and policies that enabled high-quality instruction. They also reported that the IFL helped them become more knowledgeable about instruction and more skilled at supervising and supporting principals. In particular, two of the IFL’s most practical, well-developed, and emphasized resources — the Learning Walk and the Principles of Learning (POLs) — were found to be especially influential in the districts. Unlike other IFL resources, district staff embraced Learning Walks and POLs because they addressed local, day-to-day needs and came with specific tools and ongoing professional development that spelled out their purpose and how to use them. Nevertheless, the perceived value of these two resources varied within and across districts. For example, administrators tended to find Learning Walks more useful than teachers did, and — while district and school leaders consistently reported that POLs had become a “common language” for all staff — some also reported needing additional support and time to translate these ideas into practice.

According to district and IFL leaders, the IFL’s contribution to other areas of reform — data use, coaching, and curriculum specification — was not as strong or consistent across sites.

Several common factors appeared to influence IFL partnerships with and impact on the districts. The impact of the IFL was particularly strong when there was support from district and school leaders; when IFL staff were viewed as trustworthy, credible, and having expertise that matched a particular district need; and when the IFL offered practical tools to support implementation of theoretical ideas. On the other hand, the IFL’s influence was constrained, in some cases, by the perception that it was a vendor brought in to provide particular services; by its limited capacity to support districts in all areas of reform; and by turnover within the districts and the IFL.

Lessons Learned

The experiences of these three urban districts and their partnerships with the IFL provide evidence of promising results from systemwide instructional improvement efforts, yet they also raise warnings for districts and intermediary organizations about several important challenges they might face when attempting similar reforms. The findings show that urban districts can facilitate wide-scale changes — in particular, greater uniformity of curriculum and use of data to inform instructional decisions — and that an intermediary organization can help districts address persistent constraints to reform by building the capacity of district staff to engage in instructional change and by facilitating policy alignment. While the results are not definitive and cannot be generalized to other districts, the experiences of these three urban districts and their partnerships with the Institute for Learning provide some important lessons regarding systemwide instructional improvement efforts and partnering with third-party organizations.

Lessons for Instructional Improvement

  • Investing in the professional development of central office staff can enhance the capacity to lead instructional reform.
  • Instituting local accountability policies that create incentives for meaningful change can promote effective implementation.
  • Aligning and developing a comprehensive set of strategies can reinforce overarching instructional improvement goals.

Lessons for District-Intermediary Partnerships

While the specific characteristics of the IFL set it apart from some other types of third-party organizations, the experiences of the IFL in these three districts nevertheless offer potentially useful insights for other intermediary organizations, as well as for districts considering partnerships with such organizations. Partnership success depends on:

  • Buy-in and support from top-level leaders
  • Capacity of the intermediary organization and its alignment with district needs and existing reform efforts
  • Ability to provide practical tools tailored to the district context.

New organizations seeking to partner with districts might consider both top-down and bottom-up approaches to influencing instruction. Teachers would then have direct exposure to ideas and strategies, and district leaders would understand these ideas and strategies well enough to create supportive structures and policies to help teachers use them in the classroom.


View the print-friendly version: PDF (0.2 MB)
 
Request a printed copy: Order Now

This product is part of the RAND Corporation research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

This research brief describes work done for RAND Education documented in The Role of Districts in Fostering Instructional Improvement: Lessons from Three Urban Districts Partnered with the Institute for Learning, by Julie A. Marsh, Kerri A. Kerr, Gina Ikemoto, Hilary Darilek, Marika Suttorp, Ron W. Zimmer, and Heather Barney, MG-361-EDU, 2005, 228 pages, ISBN: 0-8330-3853-2 (Full Document). MG-361-EDU is also available from RAND Distribution Services (phone: 310-451-7002; toll free: 877-584-8642; or email: order@rand.org).

Copyright © 2005 RAND Corporation

The RAND Corporation is a nonprofit research organization providing objective analysis and effective solutions that address the challenges facing the public and private sectors around the world. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.

RB-9142-EDU (2005)