Trends in Access to Effective Teaching

A student works on an assignment

Photo by Christopher Futcher

In school year 2009–2010, the Bill & Melinda Gates Foundation launched the Intensive Partnerships for Effective Teaching initiative, a $290 million project aimed at improving student achievement through more-effective management and support of the teacher workforce.

The foundation identified seven Intensive Partnership sites—three school districts and four charter management organizations (CMOs)—to implement reforms covering teacher evaluation, staffing, professional development, and compensation and career ladders over a six-year period.^[1] These reforms are intended, among other things, to improve teachers' overall effectiveness and to ensure that low-income minority (LIM) students have access to highly effective teachers. A detailed description of the initiative is available in a RAND report, Implementation: The Intensive Partnerships for Effective Teaching Through 2013–2014.^[2].

As part of an evaluation of the Intensive Partnerships initiative, RAND Corporation researchers investigated the relationship between teachers' effectiveness in raising student achievement in mathematics and reading (the teachers' value added) and the demographic characteristics of the students the teachers serve. We measured whether, on average, LIM students were taught by more or less effective teachers than non-LIM students were. We examined the issue among teachers and students in grades 4 through 8 in four sites (Hillsborough County Public Schools in Florida; Pittsburgh Public Schools in Pennsylvania; Memphis City Schools in Tennessee; and Aspire Public Schools, a California CMO) and investigated whether site policies designed to improve LIM students' access to effective teachers have worked. We also explored differences in student access to effective teachers between schools and differences in access between classes within schools. We focused the analyses on the three years prior to the initiative's implementation and the four years following implementation.

How has access to effective teachers changed over time for LIM students at the initiative sites?

Low-Income Minority Students' Access to Effective Teachers Has Not Changed Much Since Implementation of the Initiative Began

Using student achievement data from school years 2007–2008 through 2013–2014, we constructed a value-added measure of teacher effectiveness, which we used to examine how teacher effectiveness was distributed between LIM and non-LIM students each year. In the school years prior to the initiative's implementation, teachers with more LIM students were more effective, on average, than teachers with fewer LIM students. This pattern has remained fairly consistent from 2007–2008 (several years before the intervention began) to 2013–2014, although districts differ in terms of access (see next page).

Assignment of Effective Teachers to Particular Schools (Between-School Assignment) Was More Favorable for Low-Income Minority Students Than the Assignment of Effective Teachers to Classes Within Schools

A teacher helps a schoolboy.

Photo by Monkey Business Images

During both the preintervention and intervention periods, effective teachers were generally more likely to be assigned to schools with higher proportions of LIM students than other schools, but, within a school, effective teachers were generally less likely to be assigned to classes with higher proportions of LIM students than to other classes. That is, the sites have been somewhat successful at placing the most-effective teachers in schools with high percentages of LIM students; however, they have been less successful in placing the most-effective teachers within each school in high-LIM classrooms.

One possible explanation for the within-school pattern could be the division of classes into advanced and regular tracks that occurs in many middle schools. However, that explanation does not appear to be true in the Intensive Partnership sites. The assignment of effective teachers to classes within elementary schools did not appear to be more favorable for LIM students than the assignment within middle schools in either reading or math in any of the districts. These findings suggest that efforts to improve LIM students' access to effective teachers should consider improving within-school access to effective teachers at all grade levels.

There Is No Strong Evidence That Site Policies Have Yet Affected the Distribution of Effective Teachers to Low-Income Minority Students

A schoolgirl works on an assignment.

Once sites had access to teacher effectiveness ratings, the initiative could have increased LIM students' access to effective teachers in any of three ways: by reassigning LIM students to more-effective teachers, by replacing ineffective teachers with effective teachers so LIM students have better teachers, or by improving the effectiveness of teachers who teach many LIM students. We did not find evidence that sites had accomplished any of these changes on a broad scale by 2013–2014, the first year they all had opportunities to use effectiveness ratings in teacher assignments. A few sites improved LIM students' access to effective teachers in either math or reading in one or another year, but, in most cases, improvements in one area were offset by declines in another.

When We Measure Teacher Effectiveness Using Sites’ Composite Measures, Low-Income Minority Students Appear to Have Less Access to Effective Teachers

A teacher helps a student.

Photo by Christopher Futcher/iStock

In addition to measuring LIM students' access based on our value-added measure of effectiveness, we also assessed access using the sites' own achievement growth and composite measures of teacher effectiveness. Each site calculates a composite measure that is a weighted combination of classroom-observation ratings, site-generated value-added estimates or student growth percentiles, and, in some sites, parent and student surveys. LIM students had less access to effective teachers according to the sites' composite measures of effectiveness than they did using our achievement growth measure. Furthermore, the sites' composite measures of effectiveness also showed less access for LIM students than the sites' own achievement growth measures, although the patterns varied by site and subject. However, the sites' measures showed the same pattern as our measures for changes in access to effective teachers over time and the between-school-versus-within-school split in access to effective teaching.

A possible explanation for the discrepancies between the sorting estimates based on our effectiveness measures and those based on the sites' composite measures is that the classroom-observation scores included in the composite measures do not systematically take student background into account. Although the value-added measures attempt to isolate teachers' effects by controlling for student background characteristics, the observation ratings do not. If the sites' composite measures of effectiveness are more accurate than our value-added measures, LIM students have less access to effective teaching than reported above. If, however, value added is a less biased measure, use of sites' composite measures could overstate the problem.

Conclusion

Given that LIM students' access to high-performing teachers has not increased significantly in most sites from preintervention levels, sites might need to expand their efforts to improve the distribution of effective teachers. They can do this through replacement, improvement, or reassignment of teachers in ways that favor LIM students. Also, the fact that the assignment of effective teachers to schools appears to be more favorable for LIM students than the assignment of effective teachers to classes within schools might suggest that the Intensive Partnership sites should pay particular attention to within-school scheduling, counseling, and curricular factors that could restrict LIM students' access to the top teachers in their schools. These sorts of changes could take longer to implement than some of the other components of the reform. Future RAND/American Institutes for Research evaluations will describe the continued implementation of the reforms through 2015–2016 and monitor changes in LIM students' access to effective teaching as the sites have more time to use effectiveness ratings in this manner.

Key Findings

1. Prior to Intensive Partnership implementation, low-income minority (LIM) students had teachers who were somewhat more effective, on average, in improving student achievement than non-LIM students had. This pattern has remained relatively consistent since implementation of the intervention.
2. The assignment of effective teachers to particular schools within a district or school system (that is, the betweenschool assignment) has generally been more favorable to LIM students than the assignment of effective teachers to classes within schools.
3. There is no strong evidence that sites are making changes that increase the proportion of LIM students who have access to effective teachers.

Notes

• [1] We use the word site to describe the three school districts and the four CMOs that received funding from the foundation to implement the Intensive Partnerships initiative.
• [2] Brian M. Stecher, Michael S. Garet, Laura S. Hamilton, Elizabeth D. Steiner, Abby Robyn, Jeffrey Poirier, Deborah Holtzman, Eleanor S. Fulbeck, Jay Chambers, and Iliana Brodziak de los Reyes, Implementation: The Intensive Partnerships for Effective Teaching Through 2013–2014, Santa Monica, Calif.: RAND Corporation, RR-1295-BMGF, 2016.