RAND Review

A Midterm Report Card for “No Child Left Behind”

Passing or Failing?

A Midterm Report Card for “No Child Left Behind”

1 | 2 | 3

Parental Options

A key aim of the No Child Left Behind Act is to give new options to parents whose kids attend Title I schools that have not made adequate yearly progress toward meeting state testing goals. The two key options are school transfers and supplemental services. The supplemental services include tutoring, remediation, or other academic instruction given by a state-approved provider, in addition to instruction given during the school day.

We gathered data on the effects of the two parental options in nine large urban school districts: Baltimore, Chicago, Denver, Long Beach, Los Angeles, Palm Beach, Philadelphia, San Diego, and Washington, D.C. We excluded three districts from the school choice study and two districts from the supplemental services study, because in those districts there were fewer than 100 participating students with test score data that we could track over time.

Participation rates were very low for the school choice option (see Figure 1). Just 0.5 percent of eligible students across the six applicable districts transferred to new schools. Because of the small number of participants in each district, the statistical power of our conclusion is limited. But among the students who transferred across all six of the districts, we found no statistically significant effect on achievement, even though the students generally transferred to higher-achieving schools.

Figure 1 —

Participation Rates Were Lower for the School Transfer Option Than for the Tutoring Option

Participation Rates Were Lower for the School Transfer Option Than for the Tutoring Option
SOURCE: State and Local Implementation of the No Child Left Behind Act: Volume I, 2007.

In contrast, 12 percent of the students eligible for supplemental services availed themselves of this opportunity, and the results were much more favorable. In five of the seven applicable districts, students scored better in both reading and math in the first year of participating in the services and even better in the second and subsequent years. Students participating for multiple years enjoyed gains twice as large as those of students participating for just one year. African-Americans, Latinos, and students with disabilities all recorded positive achievement results. All these gains were statistically significant.

Moreover, the students who enrolled in tutoring or other supplemental services had been especially disadvantaged from the outset. On average, they had been lower achievers than other eligible students who did not enroll. (In contrast, the school choice participants had prior achievement scores similar to those of other eligible students who did not transfer.)

Compared with other racial or ethnic groups, eligible African- American students had the highest participation rate in supplemental services (with 16.9 percent enrolling in tutoring) and an above-average participation rate in school choice (with 0.9 percent opting to transfer), as shown in Figure 1. Eligible white students had an above-average participation rate of 1.1 percent in school transfers. Eligible Latinos, English-language learners, and students with disabilities had average or nearly average participation rates in tutoring.

Because our findings are based on a small number of districts that are not nationally representative, the results do not represent the national effects of school choice and supplemental services. Nonetheless, the results are important because they are based on data from districts that include a range of the underperforming schools and disadvantaged populations that the No Child Left Behind Act is designed to serve.

And one important factor does hold true nationally: Across America, 99 percent of parents of eligible children did not exercise the option to transfer their children from low-performing schools to high-performing ones. Three reasons contributed to this. First, in many districts, there were no high-performing schools to which students could transfer. Second, nearly 20 percent of districts that did have such schools available did not notify eligible families about the transfer option in a clear and timely manner. Third, even when parents were informed of viable transfer options, most parents simply did not want to send their children to a school outside their neighborhood.

Teacher Quality

Another central goal of the No Child Left Behind Act is to ensure that every child is taught by a highly qualified teacher. By June 2006, in fact, all states were supposed to have ensured that 100 percent of their teachers of core academic subjects — English, reading or language arts, math, science, foreign languages, civics and government, economics, arts, history, and geography — were designated “highly qualified.”

The law requires the states to set standards for teachers to be considered highly qualified and requires districts to notify parents of students in Title I schools if their child’s teacher does not meet the standards. It provides money that states can use to improve teacher certification systems, to recruit and retain highly qualified teachers, and to offer professional development for all teachers. The study, again conducted in collaboration with the American Institutes for Research, analyzed the progress of states, districts, and schools nationwide in implementing these provisions. We found several areas of concern.

The first problem is that state standards concerning who is “highly qualified” vary greatly, both in the passing scores that new teachers must attain to demonstrate content knowledge and in the extent to which state policies give existing teachers credit for years of experience. (These requirements go above and beyond the ordinary teacher requirements of having a bachelor’s degree and full certification.)

For new teachers, many states use different tests to assess teacher knowledge, and even states that use the same test set dramatically different passing scores. For example, out of a maximum score of 200 on the widely used Praxis II test for subject-specific knowledge and teaching skills, the minimum score required for passage ranges from 135 in Mississippi to 168 in Pennsylvania. For existing teachers, some state policies are likewise more lenient than others, with some states weighing experience more heavily than more-direct indicators of subject knowledge.

Among all teachers nationwide, 74 percent said they were considered highly qualified in 2004–05 under the provisions of the No Child Left Behind Act. Nearly a quarter did not know their status, and just 4 percent said they were not considered highly qualified.

In comparison, 15 percent of special education teachers said they were not considered highly qualified, making them almost four times as likely to fall into this category as general education teachers. Just 52 percent of special education teachers said they were considered highly qualified, while 29 percent did not know their status. Another 4 percent said they were exempt from the requirements of No Child Left Behind because of their particular assignments.

The disparities in teacher qualifications across types of teachers and schools highlight enduring inequities in access to highly qualified teachers.

Teachers of English-language learners were more likely than others to report that they were not considered highly qualified (6 percent compared with 4 percent). Teachers who were not considered highly qualified were three times more likely to be teaching in high-minority schools than in low-minority schools (7 percent compared with 2 percent) and in high-poverty schools than in low-poverty schools (6 percent compared with 2 percent).

A majority of districts had difficulty attracting highly qualified applicants in special education, math, and science. School administrators in high-minority, high-poverty, and urban districts were also more likely to cite both competition with other districts and financial obstacles as barriers to recruiting highly qualified teachers.

Nearly all teachers said they participated in professional development pertaining to instructional strategies, but few participated for an extended period of time. During the 2003–04 school year, 90 percent of elementary school teachers participated in at least an hour of professional development for teaching reading, but only 20 percent did so for more than 24 hours. Only 9 percent received extended professional development for teaching math.

If the goal is to improve the teaching workforce and thus to serve students better, several issues demand attention. First, the variation across state policies regarding highly qualified teachers raises questions about whether some states have set standards high enough. Second, the disparities in teacher qualifications across types of teachers and schools highlight enduring inequities in access to highly qualified teachers.

Third, many teachers might not have taken steps to become highly qualified because they were not aware or notified of their status. And fourth, the low proportion of teachers participating in professional development over an extended period of time suggests that more could be done in this area. The potential for the No Child Left Behind Act to effect positive change in the teaching workforce depends, in part, on addressing these issues.

Classroom Practices

We observed how educators have responded to the new accountability systems in three states — California, Georgia, and Pennsylvania — that represent a range of regions, student populations, and testing approaches. Based on surveys, interviews, and visits to schools in 89 districts across these states, we determined that the No Child Left Behind Act has affected the work of educators both positively and negatively. Our subsequent national study of states, districts, and schools has reaffirmed many of these conclusions.

Majorities of school and district administrators in most of the 50 states have engaged in similar improvement strategies: aligning curriculum with state standards and tests, using data for decisionmaking, and providing extra support to low-performing students. Many superintendents have also provided technical assistance to schools and offered a variety of professional development opportunities for principals and teachers.

Nationwide, 90 percent of school principals reported placing a major focus on at least one improvement strategy during 2004–05, and most principals reported placing a major focus on multiple strategies. Almost all schools were using test results for instructional planning and professional development. Two-thirds of schools implemented interim tests to monitor student performance during the school year.

Our in-depth surveys in California, Georgia, and Pennsylvania revealed that principals were more positive about the overall impact of the accountability systems than were teachers, and Georgia educators were more positive than those in the other two states. More than half of principals in all three states reported that their state accountability systems had benefited students in their schools. In contrast, just one-third of teachers in California and Pennsylvania and just over half in Georgia agreed (see Figure 2).

Figure 2 —

Teachers Were Less Likely Than Principals to Agree That State Accountability Systems Had Benefited Students

Teachers Were Less Likely Than Principals to Agree That State Accountability Systems Had Benefited Students
SOURCE: Standards-Based Accountability Under No Child Left Behind, 2007.
NOTE: Response options included strongly disagree, disagree, agree, and strongly agree. Each percentage represents the sum of responses either agreeing or strongly agreeing that the state accountability systems had benefited students.
  

Figure 3 —

Math Teachers Reported Instructional Changes That Might Increase Test Scores but Not Student Understanding

Math Teachers Reported Instructional Changes That Might Increase Test Scores but Not	Student Understanding
SOURCE: Standards-Based Accountability Under No Child Left Behind, 2007.
NOTES: Response options included not at all, a small amount, a moderate amount, and a great deal. Shown are the percentages of teachers reporting that they engaged in each practice either a moderate amount or a great deal as a result of the state tests.

Teachers noted a variety of ways in which the law has influenced their instruction. Some of the changes, such as efforts to align instruction with standards and to improve teaching practices, seem beneficial. But teachers also described responses that seem less desirable: Majorities of teachers in all three states reported focusing on tested topics, emphasizing test styles and problem formats, and spending more time on test-taking strategies; and about a third focused more on students near the proficiency cutoff score (see Figure 3).

The instructional behaviors listed in Figure 3 may cause test scores to rise without a concomitant increase in student understanding of the subject matter. Similarly, the reallocation of effort or resources toward certain topics, activities, or students and away from others could improve school scores while not really improving learning for all students or in all subjects.

Most principals in all three states reported employing various strategies to help teachers prepare students for state tests. As shown in Figure 4, the most common strategies were helping teachers identify content covered on the state test (so that the content could be covered in class) and discussing at staff meetings how to prepare students for the test. Roughly half the principals in all three states encouraged or required teachers to spend more time on tested subjects and less time on other subjects.

Figure 4 —

Principals Reported Using Numerous Strategies to Help Teachers Prepare Students for State Tests

Principals Reported Using Numerous Strategies to Help Teachers Prepare Students for State Tests
Color key
SOURCE: Standards-Based Accountability Under No Child Left Behind, 2007.
NOTE: Response options included yes or no.

Among the schools we visited, teachers and principals described a wide range of efforts to capture more time for reading and math instruction: eliminating an instrumental music program, decreasing the number of physical education classes offered each week from five to two, eliminating chorus and assemblies, and refocusing summer school from enrichment opportunities to academic instruction in tested subjects. Several teachers described the resulting dilemma: Activities that teachers believed kept students in school and engaged in learning were exactly those activities that schools cut because of time constraints due to increased pressure to focus on tested subjects.

Activities that teachers believed kept students in school and engaged in learning were exactly those activities that schools cut because of time constraints due to increased pressure to focus on tested subjects.

Parents, too, were concerned about the loss of activities that made schools fun and engaging, such as field trips, parties, and arts instruction. A few teachers reported a need to broaden the curriculum to attract back to the public school system students who had left because of dissatisfaction over test-related curriculum narrowing.

Beyond narrowing the curriculum, some teachers also reported focusing on “bubble kids” — those students near the proficiency cutoff score — and expressed concern that students who are not performing near this level might be shortchanged. As one elementary school teacher told us, “The high-basic child that’s almost proficient . . . that’s what we call our target group. Every teacher got a printout of their target group. We went over strategies on how to make sure you get them involved. They’re the kids that we need to push up to proficient. So, that’s our . . . strategy.”

Figure 5 —

About Half of Teachers Agreed That High-Achieving Students Were Not Being Appropriately Challenged

About Half of Teachers Agreed That 	High-Achieving Students Were Not Being Appropriately Challenged
SOURCE: Standards-Based Accountability Under No Child Left Behind, 2007.
NOTE: Response options included strongly disagree, disagree, agree, and strongly agree. Each percentage represents the sum of responses either agreeing or strongly agreeing with this statement: “As a result of the state’s accountability system, high-achieving students are not receiving appropriately challenging curriculum or instruction.”

Because the No Child Left Behind Act puts so much emphasis on having everyone score at the proficient level, there is no incentive to help high-achieving students move beyond the proficient to the advanced level. About half the teachers in all three states agreed that, “as a result of the state’s accountability system, high-achieving students are not receiving appropriately challenging curriculum and instruction” (see Figure 5).

An ironic result of No Child Left Behind, according to reports from teachers and parents, is that many high-achieving students are not being pushed ahead. As one perturbed Pennsylvania parent put it: “What’s the teacher spending her time on? Getting everyone to the middle.”

The focus on students near the proficiency cutoff score also raises concerns about the learning opportunities given to low-performing students who are not viewed as likely to achieve proficiency in just one year. In the absence of incentives to improve the performance of these students even if their scores do not approach the proficient level, there is a risk that resources will be diverted away from these students as well.


Next section: Performance Measurements
1 | 2 | 3