- To what degree was the TISS program implemented as planned?
- Were adaptations made to the program?
- What were the barriers to and facilitators of implementation of TISS?
- What were the effects of TISS on schools with participating leaders relative to schools with new residency-trained principals who were not supported by TISS in terms of student performance, student attendance, school climate, and principal retention?
In 2013, the NYC Leadership Academy (NYCLA) developed a leadership intervention—the Targeted Intensive School Support (TISS) program—in collaboration with the New York City Department of Education (NYC DOE) to support schools that were facing particular challenges. NYCLA asked the RAND Corporation to provide an independent evaluation of the program's implementation and effects, and those findings are detailed in this report.
The TISS program consisted of five key components: (1) teaming and collaborative training in aligned preservice preparation programs for a principal and assistant principal (AP); (2) coplacement of a principal and AP into an NYC DOE school; (3) team-based coaching to support the principal and AP; (4) 328 hours of extended coaching over the first three years after placement; and (5) use of a diagnostic process to guide goal setting and coaching according to school needs.
Implementation findings suggest that only two of the five key components were implemented with fidelity. A propensity weighting approach was used to compare schools with TISS principals to other schools with new principals trained through residency-based preservice programs who did not participate in TISS. Findings suggest no statistically significant differences between TISS schools and comparison schools on measures of student achievement, school culture, and principal retention. TISS schools underperformed relative to comparison schools on the measure of chronic student absenteeism.
- The diagnostic component was implemented with fidelity, with 82 percent of TISS schools completing a diagnostic and using findings from the process to inform goals for school improvement and coaching priorities.
- Coplacement of a TISS principal and AP who were paired and cotrained occurred in only 50 percent of all TISS schools, with rates of coplacement decreasing over time.
- The team-based coaching and extended coaching components were not implemented with fidelity. There was wide variation in the amount of coaching received and the degree to which this coaching was team-based across TISS schools.
- The program benefited from buy-in among coaches and some participants, support from program staff, and the flexibility of NYCLA in adapting the program to face challenges.
- Flexibility and willingness to adapt interventions were not sufficient to overcome the barriers to implementation and generate positive effects for students.
- The evolution of TISS to focus on team-building among existing staff may have helped to build transferrable skills around teaming.
- Substantial barriers hindered NYCLA's ability to implement TISS with fidelity, including a lack of buy-in from district leadership and some participants, limited understanding of the program among some stakeholders, major challenges in coplacing principals and APs, and limited time among some participants to engage in coaching hours.
- There were no statistically significant differences in standardized achievement scores when comparing TISS schools with the comparison schools.
- There were no statistically significant differences relative to comparison schools in the areas of school culture and principal retention.
- TISS schools underperformed relative to the comparison schools in terms of chronic absenteeism among students.
- When leadership interventions are offered by external organizations, ensure close coordination with and strong buy-in from districts. However, remember that this coordination and support may not be sufficient to overcome shifting priorities.
- Interventions that hinge on coplacement should be considered with caution because very specific conditions, including strong buy-in from the district, are needed for coplacement to succeed, and those conditions may change with turnover in leadership and evolving priorities. Note that, when coplacement does not occur, investments made in matching and cotraining teams of leaders prior to placement are lost.
- Although TISS is unlikely to continue as a program, integrating some valuable aspects of the intervention into other leadership development and support interventions should be considered. Among these aspects is the diagnostic process, which was reported to have helped TISS participants and coaches engage in more strategic coaching and leadership activities that align with priorities for school improvement.
Table of Contents
The Context for Leadership Support in New York City
The Targeted Intensive School Support Program
Findings on Implementation of the TISS Program
Findings on Effects of the TISS Program
Methods for the Report
Additional Findings and Robustness Checks