The Challenges of Building Local Collaboratives for Sustaining Educational Improvement

In attempting to create an educational system that promotes and ensures high standards for all students, policymakers have reached for both old and new policy levers (such as standards, incentives, and choice options) to create and sustain organizational improvement. One newer lever is the creation of community-based collaborations to sustain education reforms across the chaotic conditions often associated with switches in district superintendents. A RAND Corporation study reported in Challenges and Potential of a Collaborative Approach to Education Reform looks at one such attempt and draws out lessons that can benefit further efforts to use collaboration as a tool for improving a community's schools.

The Collaborating for Education Reform Initiative

In 1998, the Ford Foundation launched a new initiative, Collaborating for Education Reform Initiative (CERI), to help community-based organizations and central district offices in several urban centers build collaborative partnerships among these organizations whose purpose would be to promote and sustain educational improvement in the public schools. The urban centers included Cataño, Puerto Rico; Charlotte-Mecklenburg, North Carolina; Denver, Colorado; the District of Columbia (DC); Jackson, Mississippi; Miami-Dade, Florida; San Antonio, Texas; and Santa Ana, California.

The foundation asked the collaboratives to build interorganizational linkages, such as joint planning, joint implementation of reform activities, or pooling of resources to more effectively achieve reform goals, and to work jointly with a cluster of schools (a high school and the elementary and middle schools whose students feed into it) to increase the effectiveness of the teacher professional-development offerings; to align the standards, curriculum, and instruction across grade levels; and to improve community involvement in the schools. These activities were expected to result in changes in classroom-level teaching and learning, eventually improving student outcomes. In addition, the foundation asked that the collaboratives attempt to make changes in the policy structure within their districts to ensure that these cluster-level activities were sustained and to eventually become financially independent of the foundation.

The composition and focus of the eight collaboratives differed substantially across the sites. At the time the initial grants were awarded, the number of collaborative members ranged from five in one site to 19 in another and included local colleges and universities; community-based organizations, such as local foundations and advocacy groups for school reform; educators; parents; and concerned citizens. By design, the award amounts of $300,000 per year were not intended to fund a districtwide reform effort. Rather, the funds were intended for use by the collaboratives to help produce a stronger, more consistent focus on reform. Collaboratives used the funding in different ways: for example, for meetings and development of collaborative plans, for helping schools develop improvement plans, for professional development in the cluster schools, for training parents as organizers and increasing community involvement, and for newsletters and research.

What RAND Evaluated

In fall 1999, RAND began an evaluation of the effort. The evaluation had three goals: to provide feedback to sites to improve their efforts; to provide information to the Ford Foundation that would inform its decisions about support and funding provided to sites; and to document for the public the challenges and possible successes of this approach to improvement. A new report by Bodilly and colleagues, Challenges and Potential of a Collaborative Approach to Education Reform, presents the results of this evaluation four years into the effort and offers some important lessons regarding collaboration as a strategy for sustaining reform.

Findings

The researchers examined the progress made by the sites along five dimensions: development of interorganizational linkages, development and implementation of cluster-level activities, development and implementation of plans for changes in policy to sustain the cluster activities, achievement of independence by the collaborative, and achievement of changes in student outcomes.

• Development of interorganizational linkages. All of the grantees developed networks to share information and cooperated with each other in developing activities. Grantees in four of the eight sites—Cataño, DC, Jackson, and Miami—made comparatively strong progress toward the formation of deep interorganizational linkages, defined in the literature as collaboration that includes developing consensus-building decisionmaking structures, agreeing to jointly held goals, pooling resources, and jointly implementing activities. The other grantees had difficulty creating and implementing joint activities or agreeing to jointly held goals.
• Development and implementation of plans for achieving high-quality teaching and learning in the cluster. Although never fully implementing all of their proposed activities concerning professional development, alignment, and community involvement, grantees in Cataño, DC, Jackson, and Miami contributed to the development and implementation of high-quality teaching and teaching supports, such as professional development activities; teachers' visits to innovative, high-performing schools; support for master's degree programs; and development of more-effective induction programs for new teachers. Grantees in the other sites with less-developed linkages made weaker progress toward implementation of their specific activities, at least in part because members did not agree on the vision or did not pool resources to implement the vision if they did agree on one.
• Development and implementation of plans for changes in policy. The grantees in Cataño, DC, and Jackson made progress toward policy changes and/or toward expanding policy influence, whereas the grantees in Charlotte, Denver, San Antonio, and Santa Ana were still at the planning stages of policy development. The collaborative in Miami-Dade did not see its work in terms of systemic change and did not move deliberately toward ensuring districtwide support of changes to policies.
• Achievement of financial stability by the collaborative. Cataño and DC grantees took significant steps toward sustaining the collaborative and its CERI focus through a combination of deliberate planning and taking advantage of opportunities that presented themselves. The Santa Ana grantee also brought in significant funding, such as U.S. Department of Education grants, but such grants were directed toward non-CERI initiatives that the collaborative members were pursuing. Grantees in other sites were in the planning stages for this activity or had drawn in very minor funding support aside from that of the foundation.
• Achievement of improvement in student outcomes as a result of collaborative actions. Very little improvement in student achievement that could be attributed directly to the grantees was evident across sites. Test scores in Jackson, Miami, and Santa Ana improved in grades and in schools that had undertaken some of the collaborative activities, which could be presumed to possibly have had an effect. Without further data and analysis (outside the scope of this study), a stronger relationship between collaborative activities and test-score gains could not be made. Likewise, collaborative activities in Cataño might be associated with an increasing percentage of children staying in school through the middle-school years. Test scores there, however, did not paint a consistent picture of progress.

Important Factors in Developing Collaboratives

Much of the difference in progress could be traced to the difficulties of creating collaboratives themselves. The following are factors that proved to be important in these eight sites:

• Inclusion of stakeholders integral to the local context and able to contribute to the collaborative's goals. For example, inclusion of teachers and principals in collaborative planning proved to be a useful strategy to gain school-level buy-in to plans for improving professional development and other activities.
• The perceived legitimacy and authority of the lead organization. For example, collaboratives with the central office acting as the lead lacked legitimacy in the eyes of many stakeholders.
• How collaborative members worked together. Those collaboratives that employed a top-down or non-inclusive style of decisionmaking made less progress toward joint goals and joint activities.
• The characteristics of and action by the collaborative leadership. Actions by the collaborative lead, especially those encouraging open discussion and inclusion, played a crucial role in several sites in creating joint commitment.
• The fostering of the collaborative's legitimacy and reputation over time. A strong lead with recognized legitimacy was not enough. Eventually, the collaborative was judged on its own record. Those collaboratives that relied solely on the lead for authority stumbled; those that paid attention to developing collaborative recognition in their own right made progress toward influence in the community.
• The matching of goals to the local context. Collaboratives that took the time to understand the needs in the community and the assets and programs already available produced value-added activities, as opposed to redundant or unneeded programs.
• The adept use of data to inform theories of action and activities and the habit of continuously reflecting on work and of using data to alter strategies as necessary. Several collaboratives did not match activities very directly to the outcomes they desired, nor did they track data to understand whether they were having an impact. Others made a concerted effort to do so and, as a result, could reflect on data and recommend improvements to programs.
• Early attention to a plan for institutionalizing systemic change, including strategies for sustaining the collaborative as well as sustaining and scaling-up the reform agenda. Some never really addressed this function, assuming it would take care of itself. It did not. Others took this function quite seriously and made it an important part of the work from the very beginning of the effort.

Collaboratives Are an Uncertain Approach to Sustaining Education Reform

Although, in some sites, progress was made and collaboratives developed, none of the collaboratives achieved the improved student-performance outcomes that the foundation desired. The researchers conclude that collaboration-building is an uncertain process, but one with at least some significant promise for improving our schools.

The likelihood of greater effectiveness of this approach might be improved by the following: careful choice of leads; strong planning, coordination, and communication among the parties in the early stages of formation; greater technical assistance to the collaboratives in the form of help with data gathering, strategic needs assessment, and planning; and more-routine convenings of all the sites together to improve the valuable exchange of practices and approaches.