Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief

Abstract

The No Child Left Behind Act of 2001 increased the importance of accountability in education by mandating an accountability system that measures school performance through student test results. To assist those educators trying to meet the act's requirements, RAND researchers analyzed accountability approaches from manufacturing, job training, law, and health care and assessed their usefulness for education. They conclude that while education poses unique demands on its professionals, other sectors' accountability methods provide important insights for improving school performance.

In December 2001, accountability in education gained increased importance when the U.S. Congress approved the No Child Left Behind Act (NCLB), a reauthorization of the Elementary and Secondary Education Act of 1965. NCLB's goal, like that of the Improving America's Schools Act of 1994, is to improve education through a performance-based accountability system built around student test results. The emphasis on performance improvement and accountability over the last ten years represents an important change from past federal educational initiatives, which focused primarily on ensuring that services were provided. Based on examples of successful private-sector management practices, NCLB supporters argued that student achievement would improve only when educators were judged in terms of student performance and faced consequences for the results. RAND Corporation researchers analyzed accountability systems from other sectors to see how well they had worked and whether these processes might be applied in education. The systems include specific methods of encouraging performance excellence as well as broad, profession-wide accountability mechanisms. While the uniqueness of the educational setting sets it apart from the other sectors, the researchers conclude that each model provides useful insights for enhancing educational accountability and improving the operations and performance of schools and districts.

Quality Programs Can Encourage Performance Improvement Within Organizations

Malcolm Baldrige National Quality Award Encourages Self-Assessment

The Malcolm Baldrige National Quality Award program was established to recognize organizations for "achievements in quality and performance and to raise awareness about the importance of quality and performance excellence as a competitive edge."[1] It is widely regarded as the most prestigious quality award in the United States. The program is jointly sponsored by the U.S. government and the American Society for Quality, a nonprofit organization. The Baldrige program evaluates organizations according to strict criteria in seven categories: leadership, strategic planning, customer and market focus, information and analysis, human resource focus, process management, and business results.

The Baldrige Award program was extended to the educational sector in 1999, and so far, two school districts have received the award. RAND researchers conclude that the primary value of the Baldrige program is that it requires schools and districts to assess themselves in a systematic and coherent manner and gives them constructive feedback. The positive comments of Baldrige applicants who did not win emphasize the value of focused self-examination as a tool for institutional improvement. With its guidance on bringing together processes, resources, and data to promote strategic goals, the Baldrige program offers a promising model to schools and districts struggling to meet the demands of NCLB.

TPS (Lean Manufacturing) Looks to Workers to Improve the System

The Toyota Production System (TPS), commonly referred to as "lean manufacturing," is the unique manufacturing system pioneered at the Toyota Motor Company in Japan after World War II. Although it was created as a manufacturing system, it is now widely recognized for its revolutionary approach to doing business, which provides more choice to consumers, more decisionmaking involvement for workers, and enhanced productivity to companies. TPS centers on three principles. (1) Producers must understand the value stream—how every step in the process adds value to the end product—so they can design a process without waste or unnecessary steps. (2) The organization's culture must encourage continuous improvement and empower workers to make this happen through formal testing of alternative production procedures. (3) Production must be closely tied to demand, so products are built only when there is a demand for them and inputs are supplied at the appropriate time.

Applying the principles of TPS to accountability and organizational improvement in education means translating the concepts of the company and assembly line production into the context of schools and educators. While the analogy is imperfect, two elements can inform educators. First, a focus on the educational value stream would encourage teachers to think about how each step in the educational process contributes to the desired outcomes, which activities add value, and which tasks are wasteful. Second, a culture that makes teachers responsible for hypothesis-testing, experimentation, and continuous improvement could facilitate an increase in research on educational practices, better student outcomes, and a move toward the scientifically based teaching practices favored by NCLB.

Job Training Partnership Act and Workforce Investment Act Highlight the Need for Careful Goal-Setting

The Job Training Partnership Act (JTPA) of 1982 and the Workforce Investment Act (WIA) of 1998 were passed to provide job training and services to those facing serious barriers to employment. The JTPA was unique in its time because it linked explicit performance standards with performance incentives. JTPA operated through local training centers that were given a monopoly on providing JTPA-funded services. It provided the centers with performance standards and incentives for exceeding performance targets. JTPA was succeeded by the WIA in 1998, which was intended to provide greater accountability through the use of new performance measures.

JTPA and WIA are useful examples for educators concerned about NCLB because they focus on outcomes, just as NCLB does. The most important lesson they offer educators is that performance objectives can shape behavior in unanticipated and sometimes undesirable ways. While creating positive results for some individuals, JTPA and WIA both resulted in dysfunctional behaviors. Training centers found ways to meet performance targets by choosing program participants according to their likelihood of success, by manipulating reporting data, and by providing only the minimum mandated services and no more. The legislation focused on short-range goals, and the training centers followed suit by disregarding long-term goals and manipulating the system to make their short-term performance look as good as possible.

The results of JTPA and WIA highlight the need for educational policymakers to select goals carefully, to use multiple measures of performance, and to ensure that accountability pressures do not lead to undesirable educational practices. RAND suggests that educators pay attention to both long- and short-term goals and that they identify a strategy for translating long-term goals into short-term targets. Otherwise, they risk encouraging teaching practices that meet short-term objectives and disregard long-term needs.

Profession-Wide Measures Can Encourage Performance Improvement and Accountability

Law and medicine are two fields in which professional accountability mechanisms help to regulate practitioners. Their methods are relevant to educators because many researchers consider professional accountability the most promising method of advancing teaching quality. Teaching shares some important characteristics with the legal and medical professions: a public perception of professional authority and a sense of professional responsibility among practitioners in the field. There also appears to be growing interest in self-policing and oversight among members of the teaching profession.

Regulation within the legal profession begins with strict controls on entry. In most cases, an individual must hold a degree from an accredited law school. Then the person must pass a licensure exam and a test of moral character and fitness. After entry into the profession, lawyers must continue to develop their skills and knowledge by means of continuing education. The profession maintains quality and accountability through clearly articulated disciplinary procedures, which are developed through professional organizations and administered by state agencies. Professional organizations such as the American Bar Association play prominent roles in setting and enforcing standards. Market forces complement the profession's self-regulation because consumers can freely choose which lawyers to hire based on their perceptions of the lawyers' knowledge and skills.

Although teaching bears a number of similarities to the legal profession, the differences between the fields would make it difficult for educators to apply similar systems of professional accountability immediately. Education currently lacks both the infrastructure and culture to make this change. To institute self-regulation similar to that in the legal profession, educators would need to establish a credible disciplinary system for dealing with incompetent or unethical teachers, perhaps involving a state-level professional authority. To be effective, this system would require a cultural change within the profession. Teachers would need to recognize that the removal of incompetent or unethical teachers serves the interest of the profession, and they would have to support disciplinary action toward ineffectual members.

Medical Practice Guidelines Support Decisionmaking

While there is no formal, industry-wide accountability system for health care, the profession has a number of mechanisms that promote accountability. In addition to controls on entry and licensure exams similar to those of the legal profession, independent organizations within the health care industry offer best-practice guidelines to support medical clinicians in their decisions on patient care. The medical practice guidelines are developed by panels of experts and designed to cover the most common conditions, and they encourage quality by establishing standards of care based on research and expert opinion.

Despite the appeal of guidelines, educators are not yet in a position to develop guidelines like those in medicine. Unlike medicine, in which practice is supported by extensive scientific research, teaching practices are not always tested and analyzed with the same level of rigor. Therefore, it may not be possible to develop educational practice guidelines immediately. Initial guideline development could occur in areas with a more extensive research base, such as the teaching of reading, while the field generates more support and funding for best-practices research in other areas.

Public Reporting of Health Care Performance Data Facilitates Consumer Choice

Independent organizations also help to keep health care providers accountable for their performance by collecting and releasing performance data to the public. The publication of health care "report cards" and performance data allows market forces to effect change. Providers with better records can market themselves on their positive performance; those with inferior records can choose to improve or risk a loss in market share.

Under NCLB, educators are now required to publish performance data similar to the optional health care report cards. Health care organizations often respond to health care report cards by using the information to change their practices or to market their organizations. NCLB requires every state, school, and district to issue a report card every year describing teacher qualifications and student performance in reading and mathematics. The law also gives students at failing schools the option to transfer to other schools, thus providing choices similar to those in health care, where dissatisfied consumers can switch to other physicians or hospitals. The RAND researchers suggest that putting this power in the hands of education consumers may increase the leverage of public reporting over education providers.

Educators Must Develop an Integrated Strategy to Establish Accountability and Improve Performance

Based on their analysis of practices in other sectors, the researchers recommend that an educational improvement strategy include the following:

  • Methods for focused institutional self-assessment
  • Thorough understanding of the value added at each stage in the educational process
  • Plans for broadening the scientific knowledge base and developing guidelines about best practices
  • Methods to empower teachers to test new strategies and contribute directly to improvement efforts.

Developing and adopting such a strategy in education will require time and effort, but educators can draw from examples in other sectors to inform their policies.

Notes

This report is part of the RAND Corporation research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.