The Effects of Providing Intensive Coaching and Professional Development to School Principals

School principals are critical to students' academic achievement, second only to teachers.^[1] They are the primary agents of change in their schools, fostering supportive learning environments that enhance student learning and teacher effectiveness. Moreover, they often design and oversee the initiatives and hiring to sustain the positive changes that they help bring about.

There is a growing body of knowledge focused on the professional development of principals to help them be more effective in their key functions, such as coaching teachers and developing teachers' leadership skills. But there are few proven models for effective professional development of principals that improves student achievement. Because of this, developers, providers, and funders of such programs are short on evidence for how to align principals' professional learning and support with practices proven to help students achieve.

To help address this gap, the U.S. Department of Education awarded two grants — an Investing in Innovation (i3) grant awarded in 2014 and a Supporting Effective Educator Development (SEED) grant awarded in 2015 — for the National Institute for School Leadership (NISL) to deliver its Executive Development Program (EDP) to approximately 900 school principals across five U.S. states. In addition to providing the EDP, NISL provided one-on-one in-person coaching. The RAND Corporation was the independent evaluator for both grants. RAND's two evaluations, the largest of their kind to date, resulted in several findings that can inform principal training and coaching programs going forward.

Study Findings

This section describes the main findings from the two separate evaluations, referring to the name of the grant (either i3 or SEED) when needed to distinguish them.

Positive Impacts on Student Achievement Were Contained to Only One Study

In the SEED study, the researchers found small positive effects of NISL coaching on students' ELA achievement after three years when coaching was offered to principals who had previously completed the EDP. The researchers estimated that the average effect of full participation in coaching was equivalent to raising student test scores by between 1.7 and 2.4 percentile points (i.e., moving the median student from the 50th percentile in ELA performance to around the 52nd percentile in performance). The team also found small positive effects on ELA test scores in schools where principals received both the EDP and coaching concurrently. Here, the researchers estimated that the effect of full participation in both activities was equivalent to raising student test scores by between 2.1 and 2.7 percentile points. They did not find effects on math scores.

In the i3 study, the researchers found no significant effects on student achievement in ELA or math from offering the EDP and coaching within three years of the start of the EDP. It is not clear why results were different between the two studies. However, implementation may have been a factor. In the i3 study, coaching was usually provided by out-of-state NISL staff; in the SEED study, the coaches were mostly current principals from the same state as the coachee. In addition, compared with the number of principals in the SEED study, fewer principals in the i3 study fully participated in coaching. Failure to fully participate may have diluted effects and could also indicate less interest or engagement among districts and principals participating in the i3 study.

Coaching Had the Biggest Impact in Disadvantaged Schools

In the SEED study, the effects of coaching were as high as an increase of 5 to 7 percentile points in students' ELA test scores. The schools that were affected the most by coaching were those with higher poverty rates, lower baseline achievement levels, and more students of color. Thus, although the overall average effects of coaching on student achievement were limited, there appear to be some contexts in which coaching was particularly effective.

Coaching May Have Been an Accelerant or Essential Component for There to Be Achievement Effects

The results suggest that affecting student achievement at scale may require sustained, personalized engagement to help school leaders improve their schools. Offering principals the EDP alone did not have any significant effects on student achievement in the SEED study; effects were apparent only when the EDP was paired with coaching. In our interviews of principals from either the i3 or SEED study, principals most strongly endorsed coaching, although they also highly praised the EDP.

The 41 SEED principals who were interviewed provided almost universally positive feedback about coaching; for instance, many said that their coach helped them think more strategically and intentionally. However, coaching was time-intensive and therefore a costly component of the intervention; coaches spent almost as much time in transit as they did in the face-to-face coaching sessions.

About Nine out of Ten Principals Who Participated in the EDP, Coaching, or Both Endorsed the Programs

In the final survey of principals participating in each study, about nine out of ten principals agreed either "somewhat" or "to a great extent" with the statements "The NISL EDP courses helped me to lead my school better" and "I would recommend NISL EDP to a friend who is a principal." The strength of the principals' agreement was strongest for coaching, in which eight in ten i3 principals and seven in ten SEED principals agreed "to a great extent" with the statement "My coach has helped me to improve my school." In phone interviews of 74 principals, their top three takeaways from the EDP were learning to be more strategic, learning to distribute leadership more, and learning how to integrate into their work the research about how adults and children learn. The benefits that principals most valued about their coach were having a thought partner, learning to distribute leadership more, and learning to be more strategic.

In Both Studies, the EDP Influenced Principals' or Teachers' Perceptions of Aspects of Their Schools' Climate and Instructional Practices

Although the researchers did not find that taking the EDP alone affected student achievement within three years, they did find that it influenced school practices on dimensions emphasized by the program. By the third year of the SEED study, teachers in schools where principals participated in the EDP reported receiving classroom observations with greater frequency, receiving more actionable feedback than they had received before, and using more formative assessment practices as part of instruction. Furthermore, teachers reported higher ratings of the effectiveness of their principals' leadership and the overall quality of academics at their schools. In addition, principals in the SEED study who participated in the EDP were more likely to report that instruction in their school was personalized to students, and they reported prioritizing several activities aligned with research on how people learn.

In the i3 study, the researchers did not identify any significant differences in teachers' reported practices as a result of the intervention. However, principals who participated in both the EDP and coaching were more likely than were principals who participated in neither to describe their school as having a strategic plan and to report that instruction at their school was personalized for students.

These findings suggest that, even in contexts where the researchers did not detect an impact on student achievement, providing professional development to principals did have some influence on schoolwide practices. What is not known is whether these changes in practices were necessary but not sufficient to influence student achievement, had no impact on student achievement, or simply required more time to influence student achievement.

Implementing EDP-Aligned Changes to Schoolwide Practice Was a Multi-Year, Progressive Process

The results from nine in-depth case studies indicated that principals who engaged actively in the EDP shifted practice gradually in their schools over two to three years. This progression typically started with getting staff buy-in for change, led to the alignment of grade-level teachers' instruction and a connected ladder of instruction from one grade level to the next, and developed teacher leaders along the way. The nine principals focused on school improvement in different ways, including introducing small-group reading instruction, using data more effectively, and implementing more-effective formative assessments. However, all nine principals tended to follow a similar sequence for their work, which took place across several years. That sequence was as follows:

1. Establish a positive school culture that built staff trust for change.
2. Refine the use of teacher collaboration time to focus more on instruction and learning.
3. Group a coalition of the willing into a leadership team.
4. Establish more structures for teacher collaboration and learning with the leadership team; such structures may include new professional development opportunities, new professional learning communities, and more planning time.
5. Push teachers to focus on instructional improvement and increased rigor.
6. Increase transparency of instruction — for example, by encouraging teachers to visit peers' classrooms; bringing problems of practice, instructional artifacts, and student work to professional learning communities for discussion; and establishing student assessment rules.
7. Address inconsistencies in teacher practice when they arise, and adopt more-coherent instruction, such as common assessments or curricula.
8. Distribute leadership further by identifying teacher-experts to model their work for others in the building.

Study	Number of Principals Offered the EDP, Coaching, or Both	Percentage of Principals Who Fully Participated in Each Program
i3	166 principals offered the EDP concurrent with coaching	EDP: 57% Coaching: 37%
SEED	301 principals offered the EDP 180 principals offered coaching (some concurrently, some as former EDP graduates)	EDP: 66% Coaching: 58%

NOTE: Principals were considered to have fully participated in the EDP if they attended at least ten of the 12 EDP units; principals were considered to have fully participated in coaching if NISL coach logs showed that they received 60 or more hours of coaching.

Principal Turnover Weakened the Implementation of the EDP and Coaching

As shown in the table, many principals did not fully participate in the EDP and coaching, when offered. The most common reason that a principal did not fully participate was because he or she transitioned out of the original study school. The second most common reason was that the principal or district opted not to participate in one or both programs. Although the skills developed by the EDP and coaching could be useful to principals even after they change schools, those skills were intended to provide support for principals over an extended period to implement reforms in the original school.

Principal turnover limits the reach of school-based interventions that rely on stable school leadership over a period of years. Nationally, 17 percent of principals were not working in the same school building in the 2016–2017 school year as they were in the 2015–2016 school year.^[2] If this national attrition rate holds steady, then only 57 of an original 100 principals would remain in the same school building for four years in a row.

District and Principal Buy-In May Be an Important Contributor to the Effectiveness of Principal Professional Development and Coaching

RAND's i3 and SEED studies were both randomized controlled trials that yielded fewer student achievement effects than two prior studies of the EDP by John Nunnery and colleagues.^[3] Differences between how those studies were designed and how the RAND studies were designed could be one reason for the differences in results. Another is that, in the two Nunnery studies, the school districts and sometimes the state department of education funded the EDP for school principals. This is unlike the i3 and SEED studies, in which federal grant funds paid for the full cost of the EDP and coaching in participating districts. Also, the districts in the two Nunnery studies sought volunteer principals to participate in the EDP, whereas in the i3 and SEED studies, the superintendent assigned a portion of principals to participate in a staggered rollout of the EDP. Having the intervention paid for by federal funds, rather than local or state funds, and assigning principals to take the EDP may have lowered both district and principal investment in the i3 and SEED studies. Interviewees from case studies and interviews with NISL state coordinators indicate that district leaders' support for principals to take the time to leave their buildings to attend the EDP was an important signal to principals to invest their own time.

Recommendations for Funders, Technical Assistance Providers, and Developers of Principal Professional Development Programs

First, secure strong buy-in from school districts and principal participants. The RAND studies of the EDP suggest that developers, technical assistance providers, and funders should first get principal and district buy-in for principal professional development and coaching. Both can take a substantial investment of principal time, which makes principals' and their supervisors' endorsement of the professional development and attendant reforms to school practice all the more important.

Learn more about how best to scale up intensive leadership coaching. Principals in the two studies strongly endorsed coaching, and the researchers found that coaching was particularly effective for principals leading disadvantaged schools. But face-to-face coaching was also costly, partly because coaches had to spend large amounts of time traveling. Exploring approaches to delivering more of the coaching virtually and identifying the principals and schools most likely to benefit from coaching may be key.

Remember that positive systemic change in schools takes time. The case studies suggest that changes implemented by school leaders took years to take effect. Taking a longer, multi-year view of sustained, consistent school reform could be key to improved instruction and student outcomes.

Notes

• [1] Karen Seashore Louis, Kenneth Leithwood, Kyla. L. Wahlstrom, Stephen E. Anderson, Michael Michlin, Blair Mascall, Molly Gordon, Tiiu Strauss, Emanda Thomas, and Shawn Moore, Learning for Leadership: Investigating the Links to Improved Student Learning, New York: The Wallace Foundation, 2010.
• [2] Rebecca Goldring and Soheyla Taie, Principal Attrition and Mobility: Results from the 2016–17 Principal Follow-Up Survey First Look, Washington, D.C.: U.S. Department of Education, National Center for Education Statistics, NCES 2018-066, 2018.
• [3] John A. Nunnery, Steven M. Ross, Shanan Chappell, Shana Pribesh, and Elizabeth Hoag-Carhart, The Impact of the NISL Executive Development Program on School Performance in Massachusetts: Cohort 2 Results, Norfolk, Va.: Old Dominion University, Center for Educational Partnerships, July 2011; and John A. Nunnery, Cherng-Jyh Yen, and Steven M. Ross, Effects of the National Institute for School Leadership's Executive Development Program on School Performance in Pennsylvania: 2006–2010 Pilot Cohort Results, Norfolk, Va.: Old Dominion University, Center for Educational Partnerships, March 2011.

Related Content

• 

  Report

  Putting Professional Learning to Work: What Principals Do with Their Executive Development Program Learning

  Oct 7, 2019

• 

  Report

  Effects of the Executive Development Program and Aligned Coaching for School Principals in Three U.S. States: Investing in Innovation Study Final Report

  Jul 1, 2020

• 

  Journal Article

  Developing School Leaders: Findings From a Randomized Control Trial Study of the Executive Development Program and Paired Coaching

  Jan 12, 2022

Topics

• Educational Program Evaluation
• Principals
• Randomized Controlled Trial
• Secondary Education