Improving School Leadership Through the Pittsburgh Principal Incentive Program

In the past decade, policymakers have looked increasingly to performance incentives as a potential way to improve education. In 2007, the Pittsburgh Public Schools (PPS) received funding from the U.S. Department of Education to implement reforms designed to improve school leadership in the district. A major component of these reforms was the Pittsburgh Principal Incentive Program (PPIP). PPIP provided principals with support, such as professional development focused on leadership, feedback and coaching from their supervisors, and directed professional growth projects focusing on a topic of the principal's choice. The program also provided two types of monetary compensation: (1) an annual opportunity for a permanent salary increase of up to $2,000 based primarily on a rubric measuring practices in several areas and (2) an annual bonus of up to $10,000 based primarily on student achievement growth. The bonuses included premiums for growth by previously low-performing students and at schools with high concentrations of disadvantaged students, which were intended to reduce the achievement gap between disadvantaged and other students.

RAND researchers evaluated the implementation and outcomes of PPIP from 2007–2008 through 2010–2011. They used surveys, conducted focus groups, interviewed key district and school staff, reviewed program documentation, and analyzed administrative data from PPS. They found that most principals believed that the program contributed to their professional growth, and the principals reported changing their practices during the program. Student achievement growth increased during the program as well. However, because PPS was undertaking multiple, well-integrated reforms at the same time, it is impossible to attribute any changes directly to PPIP. In light of the findings, the researchers provide recommendations for school districts to consider as they design principal evaluation, support, and compensation policies.

Principals Viewed the Capacity-Building Positively

Principals responded positively to most of the capacity-building components of PPIP. A majority of principals agreed that the program's leadership training helped them improve their leadership skills, especially in the areas of monitoring instruction and providing feedback to teachers. Principals also viewed the rubric that was used to evaluate their practices as useful, and they were increasingly accepting of the idea that unfavorable school conditions should not prevent them from engaging in the practices specified by the rubric.

At the same time, a majority of principals expressed concerns about fairness, including perceptions that the criteria for assigning ratings varied among those who were assigning them, that the rubric did not distinguish effective from ineffective principals, and that the rubric might not be fair given variations among schools.

Principals' opinions about the bonus were mixed. Most did not report negative effects on morale or collaboration. However, a majority of principals expressed concerns about the fairness and validity of the measures used to award bonuses in the program. More than two-thirds reported that the prospect of earning a bonus did not affect their practices, suggesting that the bonus alone was not a motivation for most principals.

Principals Reported Improving Their Practices During the Program

Principals reported spending more time observing teachers and providing feedback on instruction during the program. Curriculum coaches corroborated those reports, and they reported that teachers found the principals' feedback useful. Coaches also reported that principals were effective in providing professional development opportunities, giving feedback on instruction, and helping teachers use data.

In addition, a majority of teachers gave their principals high ratings as instructional leaders. Similarly, principals themselves indicated through survey responses that they believed that their skills had grown most in the areas of observing teachers' classrooms, providing feedback, and evaluating teachers. In later years, the district took steps to align PPIP with a newly adopted approach to improving teaching effectiveness, and principals viewed this alignment as beneficial for the implementation of both sets of reforms.

Principals' Scores on Performance Measures Remained Steady over Time

Average principal performance on the rubric remained steady over time, with almost all principals being assigned the highest two out of four categories (proficient or accomplished) on almost all standards. Performance on the bonus measure was also relatively constant regardless of student characteristics in the school. Although some principals were concerned that the bonus might favor one type of school—such as those serving high-income families—over another, the researchers did not find evidence to support that concern.

The researchers did find some evidence that the skills and practices measured by the rubric are associated with improved student achievement. In the final year of the evaluation, the level of math achievement growth was significantly and positively related to three standards on the rubric and the total rubric score. In earlier years, there was also evidence of a positive relationship between the growth in rubric scores and subsequent achievement growth. These findings suggest that the rubric indeed measured practices and skills associated with the desired outcomes.

Student Achievement Increased

In the last year of the evaluation, student achievement growth in grades 4–8 for both math and reading reached its highest levels since the beginning of the evaluation, and growth exceeded that of the rest of the state in three of the four years of PPIP. In addition, achievement growth among the lowest-scoring students and at the most-disadvantaged schools was beginning to accelerate, consistent with the presence of bonus premiums for high performance by these students and schools.

Recommendations

Because these findings are not definitive evidence of the effects of this type of program, the researchers could not comment on whether districts or states should adopt such policies. Instead, they provide recommendations for districts that might be developing or revising principal evaluation, support, and compensation policies.

Recommendations for Principal Evaluation Systems and Performance Measures

• Consider incorporating a range of evaluation measures, including those that reflect input from various stakeholders.
• Gather evidence of validity, reliability, and fairness of the evaluation system throughout its implementation, and communicate this information to principals.
• Take steps to ensure consistency in the application of rubrics across evaluators.
• Involve all stakeholders in any reviews or redesigns of measures or incentives used in evaluation systems.

Recommendations for Program Implementation

• Monitor the achievement of disadvantaged students using student-level growth measures.
• Align the elements of a performance-based compensation system with the district's approach to improving teaching and learning.
• Devise a communication strategy that provides clear, timely, and ongoing information to help principals understand the evaluation measures and the steps the district took to ensure their validity.
• Provide principals with concrete tools for accomplishing the instructional leadership tasks encouraged by the compensation system.

Topics

• Education Policy
• Education Reform
• Pittsburgh
• Principals
• Teacher Incentives