In 2013, the NYC Leadership Academy received funding from the U.S. Department of Education through the Investing in Innovation program to develop the Targeted Intensive School Support (TISS) program, a team-based leadership intervention that aimed to support schools in need of turnaround. This report describes the program's findings and effects according to the authors' independent evaluation of the program.
A Team-Based Leadership Intervention in New York City Schools
An Evaluation of the Targeted Intensive School Support Program
Download eBook for Free
| Format | File Size | Notes |
|---|---|---|
| PDF file | 0.6 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Research Questions
- To what degree was the TISS program implemented as planned?
- Were adaptations made to the program?
- What were the barriers to and facilitators of implementation of TISS?
- What were the effects of TISS on schools with participating leaders relative to schools with new residency-trained principals who were not supported by TISS in terms of student performance, student attendance, school climate, and principal retention?
In 2013, the NYC Leadership Academy (NYCLA) developed a leadership intervention—the Targeted Intensive School Support (TISS) program—in collaboration with the New York City Department of Education (NYC DOE) to support schools that were facing particular challenges. NYCLA asked the RAND Corporation to provide an independent evaluation of the program's implementation and effects, and those findings are detailed in this report.
The TISS program consisted of five key components: (1) teaming and collaborative training in aligned preservice preparation programs for a principal and assistant principal (AP); (2) coplacement of a principal and AP into an NYC DOE school; (3) team-based coaching to support the principal and AP; (4) 328 hours of extended coaching over the first three years after placement; and (5) use of a diagnostic process to guide goal setting and coaching according to school needs.
Implementation findings suggest that only two of the five key components were implemented with fidelity. A propensity weighting approach was used to compare schools with TISS principals to other schools with new principals trained through residency-based preservice programs who did not participate in TISS. Findings suggest no statistically significant differences between TISS schools and comparison schools on measures of student achievement, school culture, and principal retention. TISS schools underperformed relative to comparison schools on the measure of chronic student absenteeism.
Key Findings
- The diagnostic component was implemented with fidelity, with 82 percent of TISS schools completing a diagnostic and using findings from the process to inform goals for school improvement and coaching priorities.
- Coplacement of a TISS principal and AP who were paired and cotrained occurred in only 50 percent of all TISS schools, with rates of coplacement decreasing over time.
- The team-based coaching and extended coaching components were not implemented with fidelity. There was wide variation in the amount of coaching received and the degree to which this coaching was team-based across TISS schools.
- The program benefited from buy-in among coaches and some participants, support from program staff, and the flexibility of NYCLA in adapting the program to face challenges.
- Flexibility and willingness to adapt interventions were not sufficient to overcome the barriers to implementation and generate positive effects for students.
- The evolution of TISS to focus on team-building among existing staff may have helped to build transferrable skills around teaming.
- Substantial barriers hindered NYCLA's ability to implement TISS with fidelity, including a lack of buy-in from district leadership and some participants, limited understanding of the program among some stakeholders, major challenges in coplacing principals and APs, and limited time among some participants to engage in coaching hours.
- There were no statistically significant differences in standardized achievement scores when comparing TISS schools with the comparison schools.
- There were no statistically significant differences relative to comparison schools in the areas of school culture and principal retention.
- TISS schools underperformed relative to the comparison schools in terms of chronic absenteeism among students.
Recommendations
- When leadership interventions are offered by external organizations, ensure close coordination with and strong buy-in from districts. However, remember that this coordination and support may not be sufficient to overcome shifting priorities.
- Interventions that hinge on coplacement should be considered with caution because very specific conditions, including strong buy-in from the district, are needed for coplacement to succeed, and those conditions may change with turnover in leadership and evolving priorities. Note that, when coplacement does not occur, investments made in matching and cotraining teams of leaders prior to placement are lost.
- Although TISS is unlikely to continue as a program, integrating some valuable aspects of the intervention into other leadership development and support interventions should be considered. Among these aspects is the diagnostic process, which was reported to have helped TISS participants and coaches engage in more strategic coaching and leadership activities that align with priorities for school improvement.
Table of Contents
Chapter One
Introduction
Chapter Two
The Context for Leadership Support in New York City
Chapter Three
The Targeted Intensive School Support Program
Chapter Four
Study Approach
Chapter Five
Findings on Implementation of the TISS Program
Chapter Six
Findings on Effects of the TISS Program
Chapter Seven
Discussion
Appendix A
Methods for the Report
Appendix B
Additional Findings and Robustness Checks
Appendix C
Interview Protocols
Research conducted by
This research was sponsored by the NYC Leadership Academy and conducted by RAND Education and Labor.
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.