Expanding Measures of School Performance Under the Elementary and Secondary Education Act

When the Elementary and Secondary Education Act (ESEA) was reauthorized in 2001 as the No Child Left Behind Act (NCLB), it established a system of school accountability based primarily on student performance on tests of mathematics and English Language Arts (ELA). Critics have lamented that NCLB's accountability system has prioritized these subjects at the expense of other important goals, such as preparing students for college and improving social and behavioral outcomes, including self-discipline, the ability to work in teams, civic-mindedness, and maintaining a healthy lifestyle. The upcoming reauthorization of ESEA will provide an opportunity to reconsider how school performance reporting systems might incorporate these other educational goals. This study focuses on how ESEA can encourage states to expand their measures of school performance to address goals beyond mathematics and ELA.

In discussions with the Sandler Foundation, key federal policymakers involved in reframing ESEA legislation expressed uncertainty about the research basis for expanding measures of school performance. In response, the foundation asked RAND to examine measures that states and districts use beyond those required under NLCB and to outline a set of principles to inform public education agencies about trade-offs associated with the new measures. To do this, RAND researchers convened a panel of experts on school accountability policies, scanned published research, conducted interviews with staff from local and state education agencies and research institutions, and reviewed the measures employed in each state that publishes its own school ratings in addition to those required under NCLB. They identified the most commonly used measures, examined the trade-offs in developing school indicator systems, and developed a set of promising options for how the new federal legislation can promote improvement in school performance measurement. They recommend that the reauthorized ESEA require states to expand their measures of school performance without dictating the specific measures states must use. They also recommend leveraging other federal grant programs to encourage states to develop and evaluate new measures.

States Measure Many Things Beyond the NCLB Requirements

RAND found 20 states that published ratings of schools in 2008–2009 or 2009–2010 based on measures beyond those required by NCLB. These measures fall into four categories:

• student test performance in additional subjects (most often history or social studies)
• growth in student performance over time
• indices of student achievement along the entire spectrum of performance, from low to high, instead of the NCLB focus on proficient or above for accountability decisions
• college readiness, such as ACT and other test scores or Advanced Placement course-taking.

In addition, the authors identified three other types of measures that are rapidly becoming more common among states:

• indicators of a safe and supportive school environment
• indicators of risk for students not graduating on time
• results of "interim" academic assessments.

Expanded Measures Could Yield Many Benefits

There is little research about how adopting additional measures such as those described above affects school performance. However, the potential benefits of an expanded set of measures are that they could

• allow for more accurate assessment of school outcomes that are widely valued but currently overlooked
• promote more valid inferences about school performance
• provide more balanced incentives to teachers and principals to improve performance in multiple areas.

But Expanding Measures Poses Risks and Requires Trade-Offs

When designing student indicator systems, decisionmakers must consider the characteristics of each measure and how it will be used in an accountability context. The authors identified several trade-offs. An important one is the balance between breadth—representing more of the outcomes that matter—and focus—highlighting a few areas where educators should concentrate their efforts. Another involves balancing complexity—for example, statistical measures that might be difficult for educators and the general public to understand—against transparency—simpler measures that are easier to interpret but less useful for decisionmaking. Policymakers must also balance comprehensiveness—the ability to measure many facets of school performance—against affordability. And, they must weigh uniformity, which would allow easy comparisons across states, against flexibility, which would permit innovation and promote local relevance. Decisions about such trade-offs must be informed by the local context and the goals the system is intended to promote.

How ESEA Can Promote Improved Measurement of School Performance

There is insufficient research to recommend specific measures, but research on high-stakes testing indicates that educators tend to focus on what is measured. A federal mandate requiring states to broaden their school outcome measures might allow stakeholders to draw more valid inferences on school performance that better reflect the multiple goals of schooling and could create incentives for educators to focus on processes or outcomes that were underemphasized in the past. The researchers recommend the following:

• In the ESEA reauthorization, broaden the range of performance measures beyond those currently mandated under NCLB. The following five domains could serve as starting points:
  1. Expand the measures of achievement and attainment, to include both current scores and growth and to capture outcomes in subjects besides mathematics and ELA, as well as in advanced course-taking.
  2. Promote a positive school culture, including such indicators as student and teacher satisfaction, academic challenge, engagement, safety, and orderliness.
  3. Adopt leading indicators, such as measures to track high school graduation rates, to gain information on students' progress toward college and career readiness.
  4. Promote positive behavioral, emotional, and physical health outcomes for students, as reflected by indicators of suspensions, expulsion, and physical health.
  5. Augment unadjusted performance indicators with indicators that adjust for discrepancies in the resources that children and, by extension, schools have available.
• Avoid immediately mandating specific new measures. The new ESEA should encourage states to expand their measures and evaluate their success. Additional measures can be gradually integrated into the system to measure school performance over the next 4–5 years. States should be required to assess the technical quality and the effects on student outcomes and school resource allocation when they add new measures to their accountability framework.
• Leverage existing federal grant programs to encourage the development and evaluation of additional school performance measures. Recognizing that states differ in their capacity to develop and test new measures, federal grants could be offered to support states in developing new measures and encourage them to coordinate their efforts, as through interstate consortia.

The reauthorization of ESEA should be informed by lessons learned from NCLB and other efforts to promote school-level measurement and accountability. This study describes promising directions for expanding the set of measures schools have at their disposal.

Topics

• Education Policy
• Educational Program Evaluation
• Elementary and Secondary Education Act of 1965
• No Child Left Behind Act of 2001