The Potential of Blind Collaborative Justice

Testing the Impact of Expert Blinding and Consensus Building on the Validity of Forensic Testimony

by Carolyn Wong, Eyal Aharoni, Gursel Rafig oglu Aliyev, Jacqueline Du Bois

Download eBook for Free

FormatFile SizeNotes
PDF file 0.4 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. Is bias reduced when experts do not know whether the prosecution or defense is the hiring party?
  2. Is bias reduced by expert consensus feedback, wherein expertise is culled from multiple sources and those sources examine the majority view to move toward a group consensus?

Biased expert testimony is a leading cause of wrongful convictions, and new techniques are needed to reduce such biases. This study conducted an experimental investigation of two potential contributors to biased testimony within adversarial litigation involving forensic evidence: (1) experts' knowledge of their party representation (i.e., prosecution vs. defense counsel), and (2) lack of input from the relevant scientific community. A sample of 580 scientists was asked to read a vignette about a hypothetical criminal case and solve a statistical reasoning problem bearing on the case evidence. Participants were randomly assigned to one of three types of party representation (prosecution, defense, or blinded). Approximately half the participants in each representation were given the correct solution in advance of their responses, The correct solution was derived by an independent panel of 12 experts, and presented as "consensus feedback." The other half of participants in each representation received the consensus feedback after providing an initial response, and received an opportunity to change their initial response following that feedback. We found no evidence of an effect of blinding on accuracy. The results revealed a consistent, positive effect of expert consensus feedback on response accuracy. We conclude that expert consensus feedback could improve the validity of expert testimony, and discuss the importance of educating scientists about ways to reduce testimonial bias.

Key Findings

Expert Consensus Feedback Resulted in Improved Performance

  • Expert consensus feedback regarding the correct response to the reasoning problem demonstrated the predicted significant effect on response errors: Delivery of feedback resulted in improved performance.

An Advantage Due to Blinding Was Not Observed

  • There was modest evidence that an expert's mere knowledge of the hiring counsel (prosecution vs. defense) could influence probabilistic reasoning. Increased accuracy tended to be higher in the party least favored by the evidence (the defense, in this case), suggesting a greater motivation to evaluate the evidence critically.
  • However, there was no conclusive evidence that blinding experts to their party representation conferred an advantage — accuracy within this condition was only marginally greater than the prosecution condition and not greater than that of the defense.

Recommendation

  • Much more research is needed to demonstrate the real-world utility and feasibility of expert blinding. To the extent that this study's results can be replicated in more naturalistic trial settings, it suggests that greater focus on common threats to sound scientific reasoning (such as base rate neglect) and engagement of the broader scientific community could improve the validity of expert testimony in court cases. In addition, the effect of expert consensus exposure suggests that expert consensus feedback could be a promising way to reduce individual error in expert testimony. Future research is needed to confirm the effect of consensus exposure and to address practical and fairness issues associated with implementing expert consensus exposure approaches in a criminal justice environment.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Methods and Data Collection

  • Chapter Three

    Analyses and Findings

  • Chapter Four

    Conclusions and Closing Remarks

  • Appendix

    Supplemental Analysis

The research reported here was conducted in the RAND Safety and Justice Program, a part of RAND Justice, Infrastructure, and Environment.

This report is part of the RAND Corporation research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.