Changes in the Standards for Admitting Expert Evidence

In 1993, the U.S. Supreme Court took up this issue. In Daubert v. Merrell Dow Pharmaceuticals, Inc., the Supreme Court directed federal judges to examine the method or reasoning underlying the admission of expert evidence and to admit only evidence that was reliable and relevant. No longer could judges defer to the appropriate expert community to determine whether the evidence was reliable, nor could they any longer leave it to the jury to decide. Judges now were to act as "gatekeepers," screening expert evidence to ensure that what was admitted was both relevant and reliable.

A new RAND report, Changes in the Standards for Admitting Expert Evidence in Federal Civil Cases Since the Daubert Decision by Lloyd Dixon and Brian Gill, examines the effects of this ruling on the behavior of federal judges, plaintiffs, and defendants. By conducting a statistical analysis of data gathered from 399 federal district court opinions written between 1980 and 1999, the authors found that judges have been taking their new role seriously and applying stricter standards to determine what sort of expert evidence is admissible. They also found that plaintiffs and defendants responded to the change in admissibility standards.

Judges Are Scrutinizing Reliability More Carefully

The Daubert decision provided a list of factors that judges might take into account when evaluating the reliability, or scientific validity, of evidence:

• whether it can be (and has been) tested

• whether it has been subjected to peer review and publication

• its known or potential rate of error

• the existence and maintenance of standards controlling the particular technique's operation

• whether it is generally accepted in the scientific community.

While general acceptance in the scientific community is still a factor, it is only one of many that might enter the judges' assessment of reliability. The Supreme Court added that judges are not obligated to consider what have come to be called the "Daubert factors" in every case, and that other factors may be included in their evaluation.

Figure 1. Trends in Challenges to Reliability of Expert Evidence

Results of the RAND analysis show that after the Daubert decision, reliability was more carefully assessed. Figure 1 shows that judges addressed the issue of reliability more often--and found evidence unreliable more often--after the Daubert decision and up through mid-1997. In subsequent years, those same proportions fell. The reversal, the authors argue, probably came about because plaintiffs and defendants began to tailor the evidence that they proposed.

The RAND study found that in the initial years after Daubert, all five Daubert factors came up more frequently in judges' discussions of reliability. Over time, judges increasingly mentioned other factors in their decisions, such as the clarity and coherence of the expert's explanation of the theory, methods, and procedures underlying the evidence.

Because Daubert de-emphasized the importance of the testimony's "general acceptance" by the expert community, some legal observers initially expected the decision to increase the admissibility of "novel science" that was not yet widely accepted. The report's authors did not find this to be the case. They found that lack of general acceptance was as much a barrier to admission after Daubert as it was before. What's more, they found that general acceptance was no longer sufficient for evidence to be found reliable. After Daubert, judges began to find evidence unreliable even though it was generally accepted.

In the early years after Daubert, when there was uncertainty about whether the ruling applied to just "hard science" or all expert evidence, judges focused most of their increased scrutiny on evidence from the physical sciences. However, they gradually began looking more closely at all types of expert evidence. These changes predated the Supreme Court's 1999 decision in Kumho Tire Co. v. Carmichael, which affirmed that judges were to assess the reliability of all expert evidence.

Judges were not only more rigorous about assessing reliability of the evidence after Daubert, they were also more rigorous about assessing relevance and expert qualifications. Once judges began acting as more watchful gatekeepers, they examined all dimensions of the evidence more closely.

Challenges to Expert Evidence Have Become Increasingly Fatal to Cases

Figure 2. Trends in the Proportion of Challenged Evidence That Is Excluded

The closer scrutiny given expert evidence resulted in an increase in the proportion of evidence that judges excluded after Daubert. For example, as Figure 2 shows, the exclusion rate holding case type, area of the evidence, and federal appellate circuit constant jumped from 53 percent during the two years before Daubert to 70 percent between mid-1996 and mid-1997.

Moreover, challenges to expert evidence are increasingly fatal to a case. The table shows that, after Daubert, parties challenging expert evidence more frequently requested summary judgment on some or all of the issues in a case, and summary judgment was more frequently granted. Although these increases may reflect other trends in litigation practices that have little to do with Daubert--such as judges' incentives to resolve cases more quickly and at lower cost--the authors believe it is likely that the more rigorous standards for evaluating expert evidence encouraged more challengers to expand the scope of their challenges and, in so doing, effectively undermined the entire basis of the opposing party's contention.

Are These Trends Leading to More-Accurate Evidence Screening?

The data gathered for the study did not allow the authors to examine how accurately judges are screening expert evidence. Judges may be more actively evaluating reliability, but that does not mean they are making the right judgments. They may not be knowledgeable enough in the relevant field to exclude evidence that is unreliable and avoid excluding evidence that is reliable.

The authors suggest further analysis to explore these questions:

• How well are judges performing the gatekeeper function? This question could be explored by convening a panel of experts to evaluate how well judges are assessing expert evidence in a sample of cases. The results could provide the basis for a report card on a judge's performance.

• How has Daubert affected case outcomes? Future research might examine how likely it is for cases to be settled or dropped after expert evidence is excluded. Assembling the histories of a representative sample of cases involving expert evidence would provide the data to answer this question.

RAND research briefs summarize research that has been more fully documented elsewhere. This brief describes work done in the RAND Institute for Civil Justice and published as Changes in the Standards for Admitting Expert Evidence in Federal Civil Cases Since the Daubert Decision by Lloyd Dixon and Brian Gill, RAND MR-1439-ICJ, 2001, 114 pp., ISBN: 0-8330-3088-4. Publications are distributed to the trade by NBN. RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis; its publications do not necessarily reflect the opinions or policies of its research sponsors. RAND® is a registered trademark.

The RAND Institute for Civil Justice is an independent research program within RAND. The Institute's mission is to help make the civil justice system more efficient and more equitable by supplying government, private decisionmakers, and the public with the results of objective, empirically based, analytic research. The ICJ facilitates change in the civil justice system by analyzing trends and outcomes, identifying and evaluating policy options, and bringing together representatives of different interests to debate alternative solutions to policy problems. The Institute builds on a long tradition of RAND research characterized by an interdisciplinary, empirical approach to policy issues and rigorous standards of quality, objectivity, and independence.

Westlaw is the exclusive online distributor of RAND/ICJ materials. You may find the full text of many ICJ documents at www.westlaw.com. A profile of the ICJ, summaries of its studies, and electronic order forms can be found on RAND's homepage at www.rand.org/icj.

Acting Director: Robert T. Reville
Development Director: Susan Rice
Communications Director: Laura Zakaras; contact by email at zakaras@rand.org

RAND Institute for Civil Justice
1700 Main Street, P.O. Box 2138
Santa Monica, CA 90407-2138 - (310) 393-0411

©Copyright 2002 by RAND/Institute for Civil Justice