Lessons Learned

Education Priorities for the Obama Administration

By Lynn A. Karoly, Gail L. Zellman, Brian M. Stecher, Georges Vernez, Laura S. Hamilton, Susan J. Bodilly, and Jennifer Li

Economist Lynn Karoly, psychologist Gail Zellman, social scientist Brian Stecher, and behavioral scientists Georges Vernez and Laura Hamilton are senior RAND researchers. Susan Bodilly is the director of RAND Education. Jennifer Li is a RAND communications analyst.

Education research offers guidelines for the Obama administration as it pursues key priorities, including high-quality preschool, accountability, pay for performance, and charter schools. Below, we outline the lessons learned from the pertinent research conducted at RAND and elsewhere:

  • With respect to preschool, many states are pursuing initiatives to address the shortfalls in access and quality, but there is a potential role for federal policy to complement state efforts. Federal dollars could be used, for example, to increase the number of children served in Head Start, to earmark additional Title I funds for state preschool programs, to supplement the state programs directly, or to do a combination of these.
  • As the U.S. Congress is expected to decide by December 2010 whether to amend and to extend the No Child Left Behind Act (NCLB), some observers have called for staying the course, some for major changes, and some for abandoning the accountability systems altogether. If accountability is going to be retained when the law is reauthorized, then policymakers should adopt a broader set of accountability measures, set consistent expectations for student performance, choose appropriate improvement targets, and redesign the system of consequences for struggling schools.
  • As for the NCLB component of school choice, policymakers should recognize that its power to induce educational improvement is limited at this time. Improved parent notifications might help, but the focus should be on reforms that improve performance across all schools while still offering school choice.
  • With regard to pay for performance for educators, knowledge about designing effective programs is limited, and so funding for further piloting and evaluation will be important. Pay for performance should also be combined with other reforms, such as assistance to educators to promote continuous improvement in educational practices.
  • Although charter schools fall under the purview of state and local policymakers, federal leaders can help states improve their charter schools by identifying and disseminating best practices, providing guidelines for performance reviews, and examining the reasons behind the success of charter high schools in raising student attainment, as identified in research on Chicago and Florida charters.
Actress Jennifer Garner reads to children at a Head Start program in Washington, D.C.
AP IMAGES/JACQUELYN MARTIN
Actress Jennifer Garner reads to children at a Head Start program in Washington, D.C., on May 5, 2009. Garner was at the school with the group Save the Children to promote early childhood education.

High-Quality Preschool

Recent studies find that academic achievement gaps among students of different ethnic, racial, and socioeconomic backgrounds are rooted in readiness gaps that are present at the starting gate, when children first enter school. A rigorous body of research also shows that effective preschool programs can narrow readiness gaps and subsequent achievement gaps (measured by test scores) through at least third grade. Several decades-long evaluations of preschool programs serving at-risk children document a lifetime of individual and societal benefits from such programs, including higher rates of high school graduation and employment, increased earnings, and reduced rates of crime and delinquency.

But access and quality vary. While federal and state programs are designed primarily to serve disadvantaged children, most programs do not receive enough money to serve all those who are eligible. As a result, disadvantaged children are less likely to attend preschool than their more advantaged peers. And for many who do attend subsidized preschool, program quality is below the level required to produce strong positive outcomes.

These findings point to several potential federal policy actions. Because Head Start lacks the funding to serve all eligible children, policymakers should explore options to combine federal Head Start funds with state funds for preschool programs. Increased Title I funding, which is set aside for schools and districts with high percentages of students from low-income families, could be earmarked for preschools. Federal matching funds could also directly supplement state efforts to serve more disadvantaged children in high-quality programs.

The federal Child Care and Development Block Grant (CCDBG) program should be leveraged to improve program quality. Currently, states are required to set aside 4 percent of their CCDBG funds for quality improvement in child care settings, whereas the National Association of Child Care Resource and Referral Agencies suggests a set-aside of 10 percent. Quality might be improved by increasing the size of the set-aside and requiring all paid providers caring for children on a regular basis to have a state license or permit and to meet rigorous training requirements.

States are experimenting with a number of initiatives to spur quality improvement, including quality rating and improvement systems, early learning standards that align with standards for kindergarten and beyond, and systems that link data from preschool to postsecondary levels. The federal government can contribute to these efforts by promoting the sharing of knowledge about best practices across the states.

The variation across states in early care and education policy provides a laboratory that can inform decisionmakers about effective ways to expand access and to raise quality. The federal government should promote rigorous evaluation of state initiatives and act as a clearinghouse for the growing base of knowledge about what does and does not promote high-quality preschool programs.

Accountability

Under NCLB, states must set standards for proficiency in reading and math, collect information about student achievement in these subjects, and implement strong corrective actions for schools and districts that do not meet the goal of moving all students toward proficiency by 2014. Both critics and supporters agree, though, that NCLB accountability has not led to enough improvement in student achievement to reach this goal.

The accountability system should set improvement targets in terms of individual growth rather than by requiring each student to reach a single, absolute level of performance.

Under the current accountability system, educators have focused attention on reading and math, and state test scores have improved, with gaps between population subgroups narrowing in some states. However, the average test-score gains may not be all they seem because of (a) widespread “teaching to the test,” (b) a focus on students near the proficiency cutoff point at the expense of others, and (c) the fact that proficiency means different things in different states. Also, the reading and math tests have drawn attention away from other subjects. Yet almost eight years after NCLB was signed into law in December 2001, the proportion of schools slated for corrective action or restructuring is increasing, the existing interventions do not seem to be working to improve performance in the most needy schools, and teachers and administrators report frustration with NCLB.

These shortcomings suggest several policy improvements. First, if the accountability system is to be retained, it should rely not merely on multiple-choice tests but also on writing samples or other open-ended answer formats (including written responses to math questions) that provide richer assessments of content knowledge and application. Second, a new policy could hold schools accountable for history, social studies, science, music, art, health, and physical education as well as reading and math, not necessarily by testing in all subjects but possibly by using other methods, such as monitoring lessons or course enrollment.

Third, if proficiency standards continue to be used for reporting purposes, policymakers should consider a combination of voluntary and mandatory approaches to make proficiency standards more comparable across states. There is currently agreement that the variation across states is problematic but disagreement about how to make the standards more consistent. Fourth, the accountability system should set improvement targets in terms of individual growth rather than by requiring each student to reach a single, absolute level of performance.

Fifth, policymakers should redesign the system of consequences so that it addresses the most serious problems more effectively. Under the current system, rigid and seemingly mechanical rules govern the number of schools receiving interventions and the type of interventions. A better system would allow states to identify the schools most in need and to design consequences for the particular needs. A two-stage process might work best. In the first stage, states would gather information about local deficiencies; in the second, states would craft interventions tailored to the shortcomings.

School Choice

One of the main innovations of NCLB is to offer students in low-performing schools the opportunity to switch schools. However, in 2006–2007, the fifth year of implementation of NCLB, only about 1 percent of students in schools identified for improvement took the opportunity to transfer to better-performing schools. This low transfer rate is due to a combination of operational issues, parental lack of knowledge, and parental consideration of other factors, such as convenience and child preference.

Recent research found that, operationally, just one in five parents of students in schools identified for improvement knew the schools were so identified. Nearly half of all districts nationwide did not notify parents of their school choice options in time for parents to make an informed decision. Most states have difficulty compiling test results in time to notify districts of the status of their schools before July of each year, and fewer than half the districts were able to notify parents of their eligibility for school choice before the beginning of the school year.

Policymakers must recognize the limited benefits of school choice and should focus efforts on reforms that improve performance in all schools.

The research further revealed that when districts did notify parents, the parents often did not receive the notification or remained unaware that they had been notified. Parents commonly reported that the notifications failed to convey key information, such as how and where to apply or whom to contact with questions. The majority of parents who did not transfer their children said they were satisfied with the quality of teaching at their current school, that transferring would be inconvenient, or that their children did not want to change schools.

These findings point to several policy actions. Test results from the previous year could be used to give parents adequate time to make informed decisions. Schools, in addition to districts, could be required to inform parents of their school choice options, because parents are more likely to pay attention to notifications from their children’s schools than those from a district office. States could be required to develop a standardized letter for districts to use in notifying parents about school choice; the states could consult with parent advocate groups in helping to draft these notifications. Federal and state officials could also consider further ways to educate parents about their children’s schools.

However, many factors besides school performance influence parents’ decisions. These factors include school location, after-school activities, and student preferences to remain in the same school with friends. Policymakers must recognize the limited benefits of school choice and should focus efforts on reforms that improve performance in all schools.

Pay for Performance

One proposal to improve teacher effectiveness is a shift from a uniform salary schedule to a pay-for-performance system, which would attach teachers’ pay to student achievement on standardized tests, possibly in combination with other data, such as graduation rates or measures of educators’ practices. Pilot programs are under way, but evidence about their effectiveness is limited.

Teacher Lisa Clemens works with students at Waadookodaading, a charter school where all subjects except English are taught in Ojibwe, in Hayward, Wisconsin.
AP IMAGES/WISCONSIN STATE JOURNAL, JOSEPH W. JACKSON III
Teacher Lisa Clemens works with students at Waadookodaading, a charter school where all subjects except English are taught in Ojibwe, in Hayward, Wisconsin. With few remaining speakers of the state’s native languages, some Wisconsin tribes are turning to such schools as the surest, and perhaps only, way to raise a new generation of bilingual students.

Studies in Israel, Kenya, and India have found that linking pay to performance has led to higher student test scores, but some of these studies have also found negative outcomes, from manipulation of test scores to “gaming” the system. There is insufficient evidence to support claims that pay for performance will raise achievement in U.S. schools.

Pay for performance raises many concerns. As noted in the section on accountability, educators tend to respond to high-stakes tests by focusing more on tested material and less on untested material. To estimate the added value of any teacher, consecutive-grade testing is necessary, which raises the costs of testing and may increase the risk of undesirable narrowing of curriculum and instruction. Even then, the contribution of a single teacher is difficult to measure accurately and fairly, because students interact with multiple teachers and other staff and because they are also influenced by factors outside school. There are questions about the role of principals, the effects of these programs on staff morale, whether to judge individual teachers or an entire school, and how to deal with English-language learners, students with disabilities, and others for whom existing tests may not provide valid information.

There are several ways to strengthen pay-for-performance programs. First, achievement measures should assess the full range of knowledge and skills that students are supposed to learn, and the measures must resist manipulation. Tests should be unpredictable from one year to the next, and, as suggested in the section on accountability, they should not rely exclusively on multiple-choice questions.

Second, test scores should be supplemented with other measures of student outcomes or instruction to reduce the likelihood that teachers or principals will focus narrowly on the content of tests. Other measures could include the percentage of students enrolled in (and possibly successfully completing) college-preparatory or Advanced Placement courses, perhaps along with assessments of teacher knowledge or practices.

Third, districts and schools using pay for performance should provide their principals and teachers with high-quality curriculum materials and professional development so that the educators can build on their strengths and address their weaknesses. Fourth, states or districts should implement data systems that link student outcomes to the responsible teachers and follow students over time, even as they transfer across schools or districts.

Detroit Midtown Academy teacher Michille Few of Ferndale, Michigan, works with seniors Corbin Stephens and Betty Osburn
AP IMAGES/GARY MALERBA
Detroit Midtown Academy teacher Michille Few of Ferndale, Michigan, works with seniors Corbin Stephens and Betty Osburn. There has been “a virtual explosion of interest” in such charter schools in Detroit, according to the Michigan Association of Public School Academies.

Fifth, the U.S. Department of Education should fund a variety of these programs and evaluate them to compare the options. A national evaluation should gather data across sites for dissemination to policymakers, practitioners, and researchers. Policymakers should also fund a long-term evaluation to understand how redesigned pay policies affect the ability of districts to recruit and to retain high-quality teachers and administrators.

Charter Schools

Since their controversial inception in 1992, charter schools have become a widely used alternative to traditional public schools. Operating independently but with public funding, charter schools represent a variety of educational approaches — from schools stressing only core subjects to Montessori schools to virtual schools that operate through telecommunication networks. Proponents predict that these schools will produce important benefits, such as expanded educational options for students, increased innovation by educators, improved student achievement, and healthy competitive pressure for traditional public schools. Opponents predict that charter schools will cause serious negative effects, such as increasing racial and ethnic stratification and drawing the highest-achieving students away from traditional public schools.

The vast majority of studies have shown that charter schools are generally on par with traditional public schools in terms of raising test scores, but studies from California and Ohio suggest that students in virtual charter schools tend to have lower test scores than students in other public schools. Charter schools in the first year of operation also often show a negative effect on test scores. In contrast, a recent study found an association between attending a charter high school and substantially higher probabilities of graduating from high school and attending college. This association was noted in Florida and Chicago, the only two sites where enough data were available.

Further research is needed to confirm whether a multigrade approach of incorporating middle grades and perhaps elementary grades onto the same campus could benefit all public high schools.

Research on charter schools in Arizona, Boston, California, Chicago, Denver, Florida, Michigan, Milwaukee, New York, North Carolina, Ohio, Philadelphia, San Diego, and Texas shows that these schools do not skim the majority white students or the highest-achieving students from traditional public schools, as feared. Conversely, charter schools do not appear to produce positive competitive effects on achievement in traditional public schools, as hoped.

Federal policymakers can help state leaders understand how to improve their charter schools. To help new charter schools avoid the low test scores typically associated with the first year, federal policymakers could provide financial support for state efforts to gather lessons learned and best practices for charter school start-up operations. Likewise, federal policymakers could assist states in identifying which types of charter schools are more or less likely to be successful so that low-performing charter schools could be eliminated and the average performance of charter schools could be raised.

Federal policymakers should fund research to identify the charter school practices that lead to higher rates of high school graduation and college attendance. A better understanding of how and why charter schools outperform traditional public schools on these measures would help educators address the national concern over high school students at risk of failure. For example, there are some indications that higher graduation rates and college enrollment are especially evident in multigrade charter high schools (those that include middle-school grades), but further research is needed to confirm whether a multigrade approach of incorporating middle grades and perhaps elementary grades onto the same campus could benefit all public high schools, both charter and traditional. square

Related Policy Briefs

Designing Effective Pay-for-Performance in K–12 Education, RAND/RB-9425, 2009, 2 pp.
Improving Accountability in Public Education, RAND/RB-9426, 2009, 2 pp.
Increasing Participation in No Child Left Behind School Choice, RAND/RB-9424, 2009, 2 pp.
Promoting Effective Preschool Programs, RAND/RB-9427, 2009, 2 pp.
The Role of Charter Schools in Improving Education, RAND/RB-9428, 2009, 2 pp.

Other Related Reading

Achievement and Attainment in Chicago Charter Schools, Kevin Booker, Brian Gill, Ron Zimmer, Tim R. Sass, RAND/TR-585-1-BMG/JOY/SRF/STRF/WPF, 2009, 50 pp.
Charter School Operations and Performance: Evidence from California, Ron Zimmer, Richard Buddin, Derrick Chau, Glenn A. Daley, Brian Gill, Cassandra M. Guarino, Laura S. Hamilton, Cathy Krop, Daniel F. McCaffrey, Melinda Sandler, Dominic J. Brewer, RAND/MR-1700-EDU, 2003, 312 pp., ISBN 978-0-8330-3354-3.
Charter Schools in Eight States: Effects on Achievement, Attainment, Integration, and Competition, Ron Zimmer, Brian Gill, Kevin Booker, Stephane Lavertu, Tim R. Sass, John Witte, RAND/MG-869-BMG/JOY/WPF, 2009, 160 pp., ISBN 978-0-8330-4693-2.
Child-Care Quality Rating and Improvement Systems in Five Pioneer States: Implementation Issues and Lessons Learned, Gail L. Zellman, Michal Perlman, RAND/MG-795-AECF/SPF/UWA, 2008, 90 pp., ISBN 978-0-8330-4551-5.
Early Childhood Interventions: Proven Results, Future Promise, Lynn A. Karoly, M. Rebecca Kilburn, Jill S. Cannon, RAND/MG-341-PNC, 2005, 200 pp., ISBN 978-0-8330-3836-4.
Evaluating Value-Added Models for Teacher Accountability, Daniel F. McCaffrey, Daniel Koretz, J. R. Lockwood, Laura S. Hamilton, RAND/MG-158-EDU, 2004, 182 pp., ISBN 978-0-8330-3542-4.
Inspiration, Perspiration, and Time: Operations and Achievement in Edison Schools, Brian Gill, Laura S. Hamilton, J. R. Lockwood, Julie A. Marsh, Ron Zimmer, Deanna Hill, Shana Pribesh, RAND/MG-351-EDU, 2005, 290 pp., ISBN 978-0-8330-3824-1.
Making Sense of Test-Based Accountability in Education, Laura S. Hamilton, Brian M. Stecher, Stephen P. Klein, eds., RAND/MR-1554-EDU, 2002, 192 pp., ISBN 978-0-8330-3161-7.
Pain and Gain: Implementing No Child Left Behind in Three States, 2004–2006, Brian M. Stecher, Scott Epstein, Laura S. Hamilton, Julie A. Marsh, Abby Robyn, Jennifer Sloan McCombs, Jennifer Russell, Scott Naftel, RAND/MG-784-NSF, 2008, 168 pp., ISBN 978-0-8330-4610-9.
Prepared to Learn: The Nature and Quality of Early Care and Education for Preschool-Age Children in California, Lynn A. Karoly, Bonnie Ghosh-Dastidar, Gail L. Zellman, Michal Perlman, Lynda Fernyhough, RAND/TR-539-PF/WKKF/PEW/NIEER/WCJVSF/LAUSD, 2008, 256 pp.
Rhetoric Versus Reality: What We Know and What We Need to Know About Vouchers and Charter Schools, Brian Gill, P. Mike Timpane, Karen E. Ross, Dominic J. Brewer, Kevin Booker, RAND/MR-1118-1-EDU, 2007, 314 pp., IBSN 978-0-8330-2765-8.
Standards-Based Accountability Under No Child Left Behind: Experiences of Teachers and Administrators in Three States, Laura S. Hamilton, Brian M. Stecher, Julie A. Marsh, Jennifer Sloan McCombs, Abby Robyn, Jennifer Russell, Scott Naftel, Heather Barney, RAND/MG-589-NSF, 2007, 302 pp., ISBN 978-0-8330-4149-4.
“State and Local Implementation of the No Child Left Behind Act, Volume IV — Title I School Choice and Supplemental Educational Services: Interim Report,” Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy/Program Studies, 2008, 112 pp., Brian Gill, Jennifer Sloan McCombs, Scott Naftel, Karen E. Ross, Mengli Song, Jennifer Harmon, Georges Vernez, Beatrice Birman, Michael Garet, Jennifer O’Day. Also available as RAND/RP-1332.
“State and Local Implementation of the No Child Left Behind Act, Volume VII — Title I School Choice and Supplemental Educational Services: Final Report,” Washington, D.C.: U.S. Department of Education, Office of Planning, Evaluation and Policy Development, Policy/Program Studies, 2009, 132 pp., Georges Vernez, Scott Naftel, Karen E. Ross, Kerstin Carlson Le Floch, Christopher Beighley, Brian Gill, Beatrice Birman, Michael Garet, Jennifer O’Day. Also available as RAND/RP-1383.
Who Is Ahead and Who Is Behind? Gaps in School Readiness and Student Achievement in the Early Grades for California’s Children, Jill S. Cannon, Lynn A. Karoly, RAND/TR-537-PF/WKKF/PEW/NIEER/WCJVSF/LAUSD, 2007, 132 pp.