This report examines Pittsburgh Public Schools' implementation and outcomes of the Pittsburgh Principal Incentive Program from school years 2007–2008 through 2010–2011, how principals and other school staff have responded to the reforms, and what outcomes accompanied program implementation.

Read Online

Improving School Leadership Through Support, Evaluation, and Incentives

The Pittsburgh Principal Incentive Program

by Laura S. Hamilton, John Engberg, Elizabeth D. Steiner, Catherine Awsumb Nelson, Kun Yuan

Download

Full Document

Full Document

FormatFile SizeNotes
PDF file 0.7 MB Best for desktop computers.

Use Adobe Acrobat Reader version 10 or higher for the best experience.
mobi file 2 MB Best for Kindle 1-3.

On desktop computers and some mobile devices, you may need to download an eBook reader to view mobi files. Amazon Kindle is the most popular reader for mobi files.
ePub file 8.6 MB Best for mobile devices.

On desktop computers and some mobile devices, you may need to download an eBook reader to view ePub files. Calibre is an example of a free and open source e-book library management application.

Summary Only

FormatFile SizeNotes
PDF file 0.2 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Appendix Materials

FormatFile SizeNotes
PDF file 0.5 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Purchase

Purchase Print Copy

 FormatList Price Price
Add to Cart Paperback152 pages $22.95 $18.36 20% Web Discount

Research Questions

  1. How were the Pittsburgh Principal Incentive Program capacity-building interventions implemented, and how have principals responded to them?
  2. To what extent have principals' skills and practices changed over the course of the Pittsburgh Principal Incentive Program?
  3. What conditions have changed at the school and classroom levels over the course of the Pittsburgh Principal Incentive Program?
  4. How did principals perform on the rubric and bonus measures, and how was performance related to principal mobility?
  5. How did student achievement change during the course of the Pittsburgh Principal Incentive Program, and how did racial and socioeconomic gaps change?

In 2007, the Pittsburgh Public Schools (PPS) received funding from the U.S. Department of Education's Teacher Incentive Fund (TIF) program to implement the Pittsburgh Urban Leadership System for Excellence (PULSE), a set of reforms designed to improve the quality of school leadership throughout the district. A major component of PULSE is the Pittsburgh Principal Incentive Program (PPIP), a system of support, performance-based evaluation, and compensation with two major components: (1) an annual opportunity for a permanent salary increase of up to $2,000 based primarily on principals' performance on a rubric that is administered by assistant superintendents and that measures practices in several areas and (2) an annual bonus of up to $10,000 based primarily on student achievement growth. The district also offered bonuses to principals who took positions in high-need schools. PPIP provided principals with several forms of support. This report examines implementation and outcomes from school years 2007–2008 through 2010–2011, with a focus on understanding how principals and other school staff have responded to the reforms, and on documenting the student achievement outcomes that accompanied program implementation.

Key Findings

The Pittsburgh Principal Incentive Program Relied on a Combination of Capacity-Building Interventions and Financial Incentives to Improve Principals' Instructional Leadership

  • Principals gave high ratings to the capacity-building interventions, including professional development and feedback from their supervisors structured around the program's leadership rubric.
  • Principals expressed concerns about the financial incentives, particularly those linked to student achievement, although resistance lessened somewhat over the course of implementation.

Principals' Behavior Changed in Ways Aligned with Program Goals

  • Principals reported spending increasing amounts of time observing teachers and providing feedback on instruction.
  • Large majorities of teachers rated their principals highly as instructional leaders.

Average Principal Performance on the Rubric Remained Steady over Time

  • Most principals received high scores on the rubric
  • The scores on the rubric standards and components were correlated, and the rubric appeared to measure a single construct related to principal leadership.

In the Last Year of the Evaluation, Student Achievement Growth in Grades 4-8 in Mathematics and Reading Reached Their Highest Levels Since the Beginning of the Evaluation

  • Growth exceeded that of the rest of the state in three out of four years since the beginning of the program.

In the Final Years of the Evaluation, There Were Increases in Achievement Growth by the Low-Achieving Students

  • This finding suggests that program design features focusing on these students are having a positive impact.
  • However, low-income and minority students continued to experience lower achievement growth than their peers when growth was measured using scale scores on the state achievement test.

Recommendations

  • Consider incorporating a range of measures, including those that reflect input from a variety of stakeholders, into the evaluation system.
  • Gather evidence of validity, reliability, and fairness of the evaluation system throughout its implementation.
  • Take steps to ensure consistency in application of rubrics across evaluators.
  • Develop a scale that differentiates performance at all points along the distribution.
  • Involve all stakeholders in any reviews and redesigns of measures and incentives used in evaluation systems.
  • Monitor racial and socioeconomic achievement gaps using student-level growth throughout the achievement distribution.
  • Align the elements of a performance-based compensation system with the district's approach to improving teaching and learning.
  • Devise a communication strategy that provides clear, timely, and ongoing information to help principals understand the evaluation measures and the steps the district took to ensure their validity.
  • Provide principals with concrete tools for accomplishing the instructional leadership tasks encouraged by the compensation system.
  • Help principals find the time needed to engage in the practices promoted by the initiative.
  • Assess the extent to which principal mobility leads to improved access to effective principals at high-need schools and to higher levels of principal effectiveness overall.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Data Sources and Analytic Approach

  • Chapter Three

    District Context and Pittsburgh Principal Incentive Program Theory of Action

  • Chapter Four

    Capacity-Building Interventions

  • Chapter Five

    Principals' Leadership Practices, Principals' Skills, and School- and Classroom-Level Responses to the Pittsburgh Principal Incentive Program

  • Chapter Six

    Principals' Performance on Rubric and Achievement Measures

  • Chapter Seven

    Student Achievement Trends

  • Chapter Eight

    Key Findings and Recommendations

The research in this report was produced within RAND Education, a unit of the RAND Corporation. The research was sponsored by the Pittsburgh Public Schools.

This report is part of the RAND Corporation Monograph series. RAND monographs present major research findings that address the challenges facing the public and private sectors. All RAND monographs undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.