An Evaluation of the California Injury and Illness Prevention Program

by John Mendeloff, Wayne B. Gray, Amelia Haviland, Regan Main, Jing Xia

This Article

RAND Health Quarterly, 2012; 2(1):16

Abstract

The Injury and Illness Prevention Program (IIPP) requirement has been the most frequently cited standard in California workplace health and safety inspections almost every year since it became effective in July 1991. Every workplace safety inspection must assess compliance with the IIPP. This article presents the results of an evaluation of the IIPP's effects on worker injuries in California and should inform policy both in California and in the federal Occupational Safety and Health Administration (OSHA) program, which has made the adoption of a similar national requirement a top priority. Using data from the Workers' Compensation Information System, OSHA Data Initiative statistics, and Workers' Compensation Insurance Rating Bureau of California reports on medical and indemnity claims from single-establishment firms, the evaluation team analyzed the impact of citations for violations of the IIPP on safety performance by (1) using the number of citations as a measure of effectiveness and (2) assessing the number of establishments that were cited for noncompliance and then came into compliance. They found that enforcement of the IIPP appears to prevent injuries only when inspectors cite firms for violations of specific subsections of that standard. Eighty percent of the citations of the IIPP by the California Division of Occupational Safety and Health program are for only a different section, the one that requires employers to have a written IIPP. The specific subsections refer to the provisions that mandate surveying and fixing hazards, investigating the causes of injuries, and training employees to work safely. Because about 25 percent of all inspections cite the IIPP, citations of the specific subsections occur in about 5 percent of all inspections. In those inspections, the total recordable injury rate falls by more than 20 percent in the two years following the inspection.

For more information, see RAND TR-1190-CHSWC at https://www.rand.org/pubs/technical_reports/TR1190.html

Full Text

Occupational safety and health regulation tends to follow one of two paths: enforcement of compliance with hazard-specific standards, as in the United States, or requirements for procedures that more broadly address safety and health, as in most of the European Union. A frequent shortcoming of the U.S. approach, which is enforced by the Occupational Safety and Health Administration (OSHA), is failing to engage employers at a fundamental and comprehensive level. Top management commitment is essential for a strong safety effort, and involvement of frontline supervisors and other employees is also critical. Training workers to understand can also complement efforts to improve safety. Although OSHA offers some training programs and guidelines, it has no general requirements regarding any of these.

Within the United States, some state agencies that enforce occupational health and safety regulations do require general preventive programs. California, for example, has had an injury and illness prevention program (IIPP) standard since 1991. Although not embodying the risk-assessment approach common among European regulators, the IIPP does focus on overall programs employers should adopt. Under the Barack Obama administration, OSHA has made adoption of a similar program its top priority.

Typical elements of such a program can include encouraging employers to think broadly about risks and encouraging employers and employers to communicate about hazards. Specifically, §3203(a) of the California Code of Regulations requires every employer to “establish, implement, and maintain an effective Injury and Illness Prevention Program.” Such a plan must

  • identify those responsible for implementing the program
  • ensure that “employees comply with safe and healthy work practices”
  • ensure that the program is communicated in an understandable form
  • have “procedures for identifying and evaluating work place hazards”
  • “include a procedure to investigate occupational injury or occupational illness”
  • have procedures for correcting hazards when discovered
  • “provide training and instruction” as necessary, including when the program is established, to new or newly assigned employees, and when new hazards arise.

The IIPP is the most frequently cited workplace safety standard in California, with violations in about 25 percent of inspections. Given the importance of this issue in California, the current interest of federal OSHA in adopting some form of IIPP, and the absence of rigorous evaluation of existing approaches, the California Commission on Health and Safety and Workers' Compensation asked RAND to assess the California IIPP's impact on injuries. Because of data limitations, we focused our study on general industry, excluding construction and agriculture.

To most safety professionals, the elements of the California IIPP are all obvious ingredients of a good safety program. Despite that agreement, there is surprisingly little good research that confirms their effectiveness. Moreover, it is not at all clear that a mandate to adopt these practices will result in the same outcomes as when they are adopted voluntarily.

Study Approach

Our study addressed the following questions about implementation and effectiveness. The former are as follows:

  • Has compliance with specific IIPP provisions improved over the years?
  • How does the number of IIPP violations cited vary with the type of establishment and type of inspection?

The questions that attempt to examine effectiveness issues are as follows:

  • Did injury and fatality rates decline in California, relative to other states, after the implementation of the IIPP standard?
  • Do workplaces that do not comply with the IIPP have worse injury, fatality, and loss performance than compliant firms?
  • Did workplaces that had been cited for IIPP violations and then came into compliance improve their injury performance relative to other workplaces?

To answer the last two questions, we relied on three data sources of establishment-level injury or loss data.

Enforcement of the Injury and Illness Prevention Program in California

Inspectors from the California Division of Occupational Safety and Health, better known as Cal/OSHA, are required to assess compliance with the IIPP in almost every inspection. Cal/OSHA inspects about 8,000 to 10,000 establishments per year, out of more than 700,000 establishments in the state. The largest categories of inspections are planned inspections, which are targeted primarily at high-hazard industries, and inspections responding to complaints and to reports of serious injuries. In about two-thirds of the inspections that cite IIPP violations, the only IIPP section cited is §3203(a), which requires employers to implement an effective IIPP and to have a written IIPP plan. Approximately one-fifth of inspections citing §3203 have violations of the subsections of 3203(a)(1) through (a)(7), and another one-sixth have violations of the requirements to document the hazard survey and the training given to employees. In the great majority of cases, no more than a single section of the IIPP is cited in an inspection.

We found that there is an important difference between inspections citing violations of §3203(a) and inspections citing violations for its specific subsections. The former carry small penalties and are cited primarily in first-time inspections, mainly at quite small, nonunion workplaces. The latter are more likely to be cited in accident investigations and are more often cited at larger sites.

An important point is that, according to Cal/OSHA leaders, its inspectors often failed to inquire beyond whether employers had a written IIPP document. As a result, it is not clear whether the workplaces cited only for §3203(a) had other IIPP deficiencies. In contrast, employers cited for the specific subsections of the IIPP clearly did have some significant problem in implementing its provisions.

When we look at trends over time, we see that, after a decline during the first two years following the effective date of the IIPP, the number of IIPP violations per inspection has remained fairly constant both for §3203(a) and for its specific subsections. Disturbingly, the number of §3203(a) violations in first-time inspections has not decreased over time. Thus, either due to lack of information or lack of deterrence, newly inspected establishments are no more likely to have written programs now than newly inspected establishments were 20 years ago. On the other hand, once an establishment has been cited for an IIPP violation, the likelihood of finding another IIPP violation at that establishment declines substantially.

Tests for Finding the Injury and Illness Prevention Program's Effects on Injuries

We first examined changes in fatality rates to see whether California experienced any improvement relative to other states in the years after the IIPP took effect in 1991. We did not find any improvement. Even if we had, it would have been unclear whether the improvement was due to the IIPP or to other factors. The absence of any evident impact at the state level suggested that, if there were impacts of the IIPP, we would need to look only at inspected establishments and compare between those cited or not cited for IIPP violations.

In order to assess the impacts of IIPP citations at inspected establishments, we carried out two different tests. The first test was based on the assumption that, if compliance with the IIPP helped to prevent injuries, then establishments with violations of its provisions should, on average, be those with poorer safety performance. We labeled this the “lookback” test.

The second, more direct test was based on the assumption that, if the IIPP were effective, establishments that were cited for noncompliance and then came into compliance would have improvements in injury performance.

In the hope of producing more-robust results, we carried out these tests on data from several different sources:

  • Workers' Compensation Information System (WCIS), specifically first-report-of-injury forms insurers and third-party administrators are to submit from employers reporting to them
  • OSHA Data Initiative (ODI) statistics, based on reports establishments are required to maintain reporting the number of different types of injuries and illnesses, collected since 1996 and covering establishments with at least 40 employees in manufacturing and a few other industries
  • Workers' Compensation Insurance Rating Bureau of California (WCIRB) reports on medical and indemnity claims from single-establishment firms.*

Each of the three samples has different measures of performance:

  • With data from the WCIS, the numerator of the measure was the total number of first-report-of-injury notices submitted to the California Division of Workers' Compensation by each establishment. We had this figure for each month. We obtained the number of employees during each month at each establishment and calculated the injury rate for the 12 months and 24 months before and after each inspection. The WCIS began collecting reports of injuries in 2000 but became more complete in 2001, which is the first year we use. Although reporting to the WCIS was mandatory, there has been no penalty for failing to report, and substantial gaps remain in reporting.
  • With data from the ODI, the measure was the OSHA total recordable injury and illness rate. The ODI sample included rates beginning in 1996 and targeted establishments with more than 40 employees, although some smaller ones were included. The denominator for the ODI's rates are the hours worked, translated into the number of full-time-equivalent (FTE) workers. For the lookback test, we looked at the rates one year and two years before the year of the inspection. We did not use the year of the inspection because many of the inspections were triggered by accidents, which raise the injury rate for that year. For the change test, we used data from one year before and two years after the year of the inspection.
  • With data from the WCIRB, the measure was the experience modification factor (ex-mod) of the firm. This sample included only single-establishment firms; the ex-mods were based on injury experience dating back to 1991.

For each inspection, we looked at the relationship between the IIPP compliance status and the ex-mod factor for the firm in the policy year two years after the inspection. Because ex-mods are based on three years of injury loss experience beginning one year before the policy year, the ex-mod two years after the inspection seemed to be a reasonable figure to use.

Findings on Injury Impacts

For the lookback test, the WCIRB and ODI samples produced similar results. Employers that were cited for a violation of §3203(a), the basic requirement to have a written IIPP document, actually had better performance (ex-mods or prior injury rates) than firms that had no IIPP violations. In contrast, employers whose only IIPP violations were the specific subsections of §3203(a), especially the requirements to train employees and to investigate accidents, had worse performance than employers that were not cited for any IIPP violation or that were cited only for §3203(a). There were no significant findings with the WCIS data, although the average effect of citing any of the specific requirements came close.

For the change test using the ODI data, we found that citations for noncompliance with the specific subsections of the IIPP were followed by improvements in injury rates. Importantly, we found these decreases both where the inspection citing the subsection was an accident investigation and where it was not. For inspections that were not triggered by accidents, the average effect of citing the specific provisions §§3203(a)(1)–(a)(7) was a 26-percent decrease in injury rates in the following year. The most consistent finding for the subsections was that a citation for failing to provide appropriate training was linked both to poorer performance prior to the inspection and to improved performance (a 53-percent reduction) after the inspection. With the ODI sample, we found no effect when only the §3203(a) provision was cited. With the WCIS data, we found no evidence of any effects. (Because the ex-mod factor for a firm is based on three years of data and thus changes slowly, we excluded the WCIRB sample from the change test.)

In conclusion, we failed to find any clear impact of the IIPP on the total fatality rate in California. We did find sizable effects when the specific subsections of the IIPP were cited, but this occurred in only 5 percent of inspections.

Using the estimated 26-percent reduction in the total recordable injury rate following a citation for the specific subsections (§§3203[a][1]–[a][7]) of the IIPP, we would find, on average, an annual reduction of 0.29 injuries at a workplace with employees and 0.96 at a workplace with 100 when the employer implements the specific subsections of the IIPP.

Because, except for having fewer small workplaces, our sample was representative of inspected workplaces in California, we think that the results are generalizable to that group. However, we expect that the absolute effects of the program in sectors with low injury rates—sectors that typically get few inspections—would be less.

Policy Implications

If we assume that that the safety effects of the IIPP in California have probably been real but not very large, what are the policy implications for California and for other jurisdictions considering similar policies? The answer depends, in part, on the reasons for those results.

It is plausible that higher penalties for failure to have a written IIPP document would have reduced the number of those violations somewhat. Requirements for some form of employee participation in the implementation of the IIPP would probably have helped as well. More important, we believe, based on interviews with Cal/OSHA leaders, that inspectors did not regularly probe to find out whether employers actually had implemented the more-specific subsections of the IIPP. Variability among inspectors played a role here. However, a more important factor was that, despite Cal/OSHA's support for the IIPP standard, its enforcement process often failed to look beyond paper compliance with its provisions.

The traditional OSHA enforcement program is focused on detecting and abating hazard-specific standards—e.g., unguarded machines, slippery floors. A quite different enforcement program would rely solely on the implementation of a safety program. OSHA or Cal/OSHA would examine whether the employer had carried out each of the requirements of the IIPP program but would not focus on hazard-specific standards.

Although possibly quite effective, this second approach carries some risks. It assumes that the process can ensure that major hazards are eliminated. But it may be difficult to assess the quality of the process with a great deal of confidence. Employers may be able to create the image of compliance without the substance. In addition, it is difficult to know, for example, just how effective a particular trainer or training program is. And even if the process is carried out properly, it is not fail-safe. To the extent that hazard-specific standards convey useful information to employers and workers about what precautions to take, that contribution would be undermined by a shift away from relying on those standards.

However, there may be another approach that achieves some of the benefits of both strategies, without the drawbacks. Under this approach, Cal/OSHA would still inspect to identify hazard-specific violations. However, when it did so, the inspector would ask managers, “How did your IIPP allow this hazard to appear in your workplace or allow this injury to occur?” In other words, he or she would try to relate the hazards to the program that the employer is required to implement. Detection of hazards would lead not only to the removal of hazards but also to the strengthening of safety programs.

In no small measure, this middle approach is the one used by the Health and Safety Executive (HSE) in the United Kingdom. In that case, the reference is to the employer's mandatory “risk assessment” rather than to an IIPP, but the principle is the same.

It seems plausible that discussing the relevance of the IIPP to injuries and violations would require inspectors to spend more time on-site. Thus, these inspections would need to be more effective in order to compensate for the prospect that fewer would be conducted. The new approach might provide more long-lasting benefits. Currently, analyses of the effects of enforcement typically find effects only in the year or two following an inspection with a penalty. The motivational effects of a serious violation fade over time, and compliance decays. In contrast, it is plausible, but hardly guaranteed, that efforts to support the practices required by a firm's safety and health program could have more-enduring effects.

Reference

California Code of Regulations, Title 8, Industrial relations, Division 1, Department of Industrial Relations, Chapter 4, Division of industrial safety, Subchapter 7, General industry safety orders, Group 1, General physical conditions and structures, Section 3203, Injury and Illness Prevention Program. As of January 24, 2012:
http://weblinks.westlaw.com/result/default.aspx?cite=8CAADCS3203&db=1000937&findtype=L&fn=_top&pbc=DA010192&rlt=CLID_FQRLT5082392718241&rp=%2FSearch%2Fdefault%2Ewl&rs=WEBL12%2E01&service=Find&spa=CCR-1000&sr=TC&vr=2%2E0

Note

* Establishment, in our usage, is the same as an individual worksite. A firm may have many different establishments, whether factories or commercial facilities.

RAND Health Quarterly is produced by the RAND Corporation. ISSN 2162-8254.