Biased expert testimony is a leading cause of wrongful convictions, and new techniques are needed to reduce such biases. RAND researchers conducted an experimental investigation of two potential contributors to biased testimony within adversarial litigation involving forensic evidence: (1) experts' knowledge of their party representation (i.e., prosecution vs. defense counsel), and (2) lack of input from the relevant scientific community.

The Potential of Blind Collaborative Justice

Testing the Impact of Expert Blinding and Consensus Building on the Validity of Forensic Testimony

Carolyn Wong, Eyal Aharoni, Gursel Rafig oglu Aliyev, Jacqueline Du Bois

ResearchPublished Aug 7, 2015

Biased expert testimony is a leading cause of wrongful convictions, and new techniques are needed to reduce such biases. This study conducted an experimental investigation of two potential contributors to biased testimony within adversarial litigation involving forensic evidence: (1) experts' knowledge of their party representation (i.e., prosecution vs. defense counsel), and (2) lack of input from the relevant scientific community. A sample of 580 scientists was asked to read a vignette about a hypothetical criminal case and solve a statistical reasoning problem bearing on the case evidence. Participants were randomly assigned to one of three types of party representation (prosecution, defense, or blinded). Approximately half the participants in each representation were given the correct solution in advance of their responses, The correct solution was derived by an independent panel of 12 experts, and presented as "consensus feedback." The other half of participants in each representation received the consensus feedback after providing an initial response, and received an opportunity to change their initial response following that feedback. We found no evidence of an effect of blinding on accuracy. The results revealed a consistent, positive effect of expert consensus feedback on response accuracy. We conclude that expert consensus feedback could improve the validity of expert testimony, and discuss the importance of educating scientists about ways to reduce testimonial bias.

Key Findings

Expert Consensus Feedback Resulted in Improved Performance

  • Expert consensus feedback regarding the correct response to the reasoning problem demonstrated the predicted significant effect on response errors: Delivery of feedback resulted in improved performance.

An Advantage Due to Blinding Was Not Observed

  • There was modest evidence that an expert's mere knowledge of the hiring counsel (prosecution vs. defense) could influence probabilistic reasoning. Increased accuracy tended to be higher in the party least favored by the evidence (the defense, in this case), suggesting a greater motivation to evaluate the evidence critically.
  • However, there was no conclusive evidence that blinding experts to their party representation conferred an advantage — accuracy within this condition was only marginally greater than the prosecution condition and not greater than that of the defense.

Recommendation

  • Much more research is needed to demonstrate the real-world utility and feasibility of expert blinding. To the extent that this study's results can be replicated in more naturalistic trial settings, it suggests that greater focus on common threats to sound scientific reasoning (such as base rate neglect) and engagement of the broader scientific community could improve the validity of expert testimony in court cases. In addition, the effect of expert consensus exposure suggests that expert consensus feedback could be a promising way to reduce individual error in expert testimony. Future research is needed to confirm the effect of consensus exposure and to address practical and fairness issues associated with implementing expert consensus exposure approaches in a criminal justice environment.

Topics

Document Details

  • Copyright: RAND Corporation
  • Availability: Web-Only
  • Year: 2015
  • Pages: 39
  • Document Number: RR-804-1-NIJ

Citation

RAND Style Manual
Wong, Carolyn, Eyal Aharoni, Gursel Rafig oglu Aliyev, and Jacqueline Du Bois, The Potential of Blind Collaborative Justice: Testing the Impact of Expert Blinding and Consensus Building on the Validity of Forensic Testimony, RAND Corporation, RR-804-1-NIJ, 2015. As of October 10, 2024: https://www.rand.org/pubs/research_reports/RR804-1.html
Chicago Manual of Style
Wong, Carolyn, Eyal Aharoni, Gursel Rafig oglu Aliyev, and Jacqueline Du Bois, The Potential of Blind Collaborative Justice: Testing the Impact of Expert Blinding and Consensus Building on the Validity of Forensic Testimony. Santa Monica, CA: RAND Corporation, 2015. https://www.rand.org/pubs/research_reports/RR804-1.html.
BibTeX RIS

The research reported here was conducted in the RAND Safety and Justice Program, a part of RAND Justice, Infrastructure, and Environment.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.