RAND Review
Passing or Failing?
A Midterm Report Card for “No Child Left Behind”
Performance Measurements
One of the most widespread criticisms of the No Child Left Behind Act is its dependence on measures of achievement that represent the grade-level scores of students at a single point in time. Individual growth models are generally preferred because they follow the same students over time, defining the performance target in terms of improvement rather than a fixed level of attainment.
Fortunately, one recommendation that is being widely discussed in the law’s reauthorization debate is the adoption of growth-based measurements of student achievement. Some states are now experimenting with growth-based measurements as part of a U.S. Department of Education pilot program. However, this program requires the existing grade-level measurements to continue to be used, and it still requires 100-percent grade-level proficiency by 2014, a goal that many educators view as unrealistic.
Teachers and principals alike doubt the validity of the existing measurements. Teachers are especially concerned that school scores are being influenced by student background characteristics and other factors beyond a school’s control. Adopting alternative performance indicators that mitigate the influence of external factors might increase the likelihood that teachers will view the performance metrics as reflecting their own efforts. And given the widespread skepticism that grade-level targets can be reached over the next several years, a growth-based measure might increase the likelihood that educators will view their targets as attainable, perhaps raising their motivation.
Improved performance measurements could also redress the excessive focus on bubble kids. The current focus on proficiency rates creates incentives to move students from just below to just above proficient but fails to reward teaching that does not push students over this threshold (whether because students’ prior performance is far below it or already above it). A growth-based measurement that gives credit for movement all along the achievement scale could still reflect state or national priorities without ignoring certain types of achievement gains. For example, such a measurement tool could incorporate weights that create extra incentives for movement at the lower end of the scale.
Another serious issue is the treatment of students with disabilities and students with limited English proficiency. Under the law, these students are included in school and district scores and, when numbers are sufficient, must meet subgroup targets as well. Educators at all levels in California, Georgia, and Pennsylvania doubted the appropriateness of this policy. More than 80 percent of superintendents believed that special education students should not be included in the proficiency tabulations. More than 80 percent of principals agreed that the system of accountability does not allow sufficient flexibility for meeting the needs of students in special education programs and students with limited English proficiency (see Figure 6).
Figure 6 —Superintendents and Principals Agreed That the Accountability Systems Were Too Inflexible to Meet the Needs of Students with Disabilities and of English-Language Learners | |
SOURCE: Standards-Based Accountability Under No Child Left Behind, 2007.NOTES: Response options included strongly agree, agree, disagree, and strongly disagree. Each percentage represents the sum of responses either agreeing or strongly agreeing. |
Among schools that miss adequate yearly progress for just one subgroup, the subgroup that fails to make the cut most often is students with disabilities. School personnel seem acutely aware of this. Teachers often expressed frustration that these students were being blamed for the school’s failure to meet the state standard and that this was hurting their self-esteem. As one teacher lamented, “Every finger is pointing at special education.”
Education policymakers should address the concerns expressed to us by teachers. Their beliefs about subject matter, students, and accountability-related reforms are likely to be important predictors of implementation. The extent to which teachers believe they can achieve the goals of the reforms will also affect their behavior. Because any effects of the No Child Left Behind Act are ultimately going to occur as a result of what teachers do in the classroom, it is critical to ensure that they respond to the requirements of the law in educationally productive ways. For their part, teachers are in a unique position to observe the effects of accountability policies on teaching and learning. Policymakers, therefore, should listen carefully to teachers when considering revisions to the policies.
On the Bubble
We found some reasons for optimism and several reasons for concern regarding the state accountability systems, parental options, teacher quality provisions, classroom practices, school practices, and performance measurements associated with the No Child Left Behind Act of 2001. The challenge now facing Congress, as it debates the reauthorization of the law, is to maximize its desirable effects and minimize the undesirable ones. The challenge is made tougher by the fact that the data are still coming in. Nonetheless, we offer some tentative recommendations based on the discussion above.
First, if the goals are to foster achievement for all students and accountability for all schools, Congress should require similar yardsticks for all states. Because each state now defines both student proficiency and teacher quality differently under No Child Left Behind, this already leaves many children behind their peers in other states.
Second, Congress should make the option of tutoring and other supplemental educational services available to eligible parents in the first year of school improvement rather than waiting for the second year, because this option has shown positive effects on student achievement. Congress should also stress the need for more timely and effective communication to parents regarding the school transfer option so that the long-term data will be more reliable.
Third, Congress should adopt the principle that schools should be rewarded for improving performance across the distribution of achievement, not just at the proficient level. High-paying jobs in our society will require higher-order skills, and so schools should be encouraged to promote advanced learning for all students. One way to do this would be to design an accountability index that recognizes movement from each level to the next higher level (for example, from the proficient to the advanced level). Such an index is not likely to lead to higher-quality instruction, though, unless the index is tied to assessments that measure the desired outcomes.
Congress should adopt the principle that schools should be rewarded for improving performance across the distribution of achievement, not just at the proficient level. |
Fourth, states should therefore be encouraged to develop assessments that measure higher-order thinking and problem-solving skills. Currently, states tend to rely on multiple-choice tests because they are relatively inexpensive, but many of these tests emphasize only facts and routine tasks. States are likely to need financial and technical assistance to improve their assessments, but the investment may be worthwhile if it promotes better instruction and a broader range of student skills and knowledge.
Fifth, Congress should allow for greater flexibility in the treatment of students with disabilities and those with limited English proficiency. Growth-based performance measurements could still be used to hold schools and districts accountable for improving the achievement of these students, thus allowing them to be included fairly in calculations for the schools.
Sixth, Congress should look beyond math, reading, and science and explore ways to incorporate other important outcomes of schooling into the state accountability systems — including, for example, achievement in other subjects, healthy physical behaviors, and progress through the grades. At the same time, Congress should move cautiously in this direction so that efforts to increase the breadth of the accountability system do not overburden schools and students.
Seventh, Congress should monitor which of the sanctions are working and which are not. Those schools that repeatedly fail to meet state standards are supposed to face severe sanctions, such as the replacement of all staff or the imposition of new public or private leadership. In reality, only about 15 percent of the schools now in restructuring have experienced these harsh sanctions. Most states and districts are opting for lesser sanctions, such as the adoption of a new curriculum. In any reauthorization bill, Congress should include provisions to sort out, from the variety of actions being taken by schools and districts, those that are most effective from those that are least effective.
Continued monitoring will be needed to grade the effects of the No Child Left Behind Act at all levels of the educational system and to ensure that any reauthorized or extended law reflects best practices as currently understood. The studies summarized here, along with others still in progress, should help policymakers identify those areas of the law and its implementation that warrant continuation or expansion and those areas of the law itself that require improvement, corrective action, or restructuring. 
Top