For many administrators and educators, it's no surprise that evidence shows principals can have a positive impact on student achievement. With new Every Student Succeeds Act (ESSA) rules allowing funds to be spent on strengthening school leadership, states and school districts have more opportunities to establish and support such programs. Both the law and common sense require states and districts to direct resources toward interventions that have shown evidence of effectiveness. ESSA establishes four tiers of evidence that are used to determine the effectiveness of an intervention. As we discovered during a recent review of ESSA requirements and the existing body of evidence for school leadership interventions, how evidence requirements are interpreted could have significant implications for gaining access to funds.
The first three evidence tiers are relatively straightforward. Tier I (strong evidence) requires at least one well-designed and well-implemented experimental study (a randomized controlled trial). Tier II (moderate evidence) requires at least one well-designed and well-implemented quasi-experimental study (such as a study that creates an equivalent comparison group either with nonrandomly assigned participants or by analyzing existing data and comparing those exposed to the intervention with those who were not). Tier III (promising evidence) requires at least one well-designed and well-implemented correlational study (such as a study that uses existing data to examine the relationship between the intervention and one or more relevant outcomes) that controls for selection bias. Our review finds that there is promising to strong evidence that interventions focused on leadership evaluation systems, principal preparation, professional learning, school autonomy, and school reform are linked to improvements in student outcomes.
Interpretation of the evidence definition becomes a little less certain in tier IV, which requires that “an activity, strategy, or intervention demonstrate a rationale based on high-quality research or a positive evaluation that suggests it is likely to improve student or other relevant outcomes.” In addition, tier IV activities require ongoing efforts to evaluate the effects of the activity, strategy, or intervention. The Department of Education released a non-regulatory guidance in September that provides a bit more detail on what, exactly, constitutes a “rationale.” Specifically, it calls for a theory of action (i.e., a logic model, showing how the intervention should affect the outcomes) that provides a well-specified conceptual framework identifying key components (i.e., “active ingredients”) of the proposed intervention and “describes the relationships among the key components and outcomes, theoretically and operationally.”
Consistent with ESSA and the department's new guidance, we developed an approach to identify tier IV-supported school leadership interventions, and used this new approach to update our April report, School Leadership Interventions Under the Every Student Succeeds Act: Volume I—A Review of the Evidence Base, Initial Findings.
What might administrators and educators expect could be covered by ESSA evidence tiers I through IV? Key findings from our revised report may provide some answers:
- Two tools that evaluate school leaders—the Vanderbilt Assessment of Leadership in Education and Marzano School Leadership Evaluation Model—are built on well-specified, research-based logic models that indicate the tools may have positive effects on school leaders.
- Two principal preparation programs—New Leaders and the Texas Principal Excellence Program—have tier II evidence, indicating that they may improve principal leadership behaviors and/or student achievement. In addition, based on tier IV-relevant logic models, a number of other principal-preparation programs that have been studied would likely have positive impacts.
- Professional learning interventions such as McREL's Balanced Leadership Program and the National Institute for School Leadership Executive Development Program have tier I or tier II support, and there is some evidence that additional professional learning programs are supported by tier-IV research-based logic models.
- Several tier II studies support school-based decisionmaking, a component of principal working-condition interventions.
- Tier IV support exists for districts using incentives and helping principals manage their time.
- Two school-leadership focused school-improvement models—the Knowledge is Power Program and the University of Virginia's School Turnaround Specialist Program—are supported by tier I and/or tier II evidence.
While our review highlights specific interventions with effects that have been documented in the published literature, states and districts interested in pursuing leadership interventions should not feel constrained by this list. At its heart, ESSA gives states and districts the authority to set their own trajectories, guided by both evidence and the determination of what would work in context. For example, states that want to adapt an existing, branded intervention may call on the evidence for that intervention to justify their own planned approach. To support such efforts, our report provides brief descriptions of each intervention to help administrators understand key components they may seek to replicate.
Given the new opportunities ESSA provides to invest in school leadership, our review can be used as a starting point by states and districts seeking to implement locally relevant solutions—and to build the evidence base for what works.
Rebecca Herman is a senior policy researcher and distinguished chair in education policy at RAND and Susan M. Gates is a senior economist and director of the Office of Research Quality Assurance at RAND.
Commentary gives RAND researchers a platform to convey insights based on their professional expertise and often on their peer-reviewed research and analysis.
