Biased expert testimony is a leading cause of wrongful convictions, and new techniques are needed to reduce such biases. RAND researchers conducted an experimental investigation of two potential contributors to biased testimony within adversarial litigation involving forensic evidence: (1) experts' knowledge of their party representation (i.e., prosecution vs. defense counsel), and (2) lack of input from the relevant scientific community.
- Is bias reduced when experts do not know whether the prosecution or defense is the hiring party?
- Is bias reduced by expert consensus feedback, wherein expertise is culled from multiple sources and those sources examine the majority view to move toward a group consensus?
Biased expert testimony is a leading cause of wrongful convictions, and new techniques are needed to reduce such biases. This study conducted an experimental investigation of two potential contributors to biased testimony within adversarial litigation involving forensic evidence: (1) experts' knowledge of their party representation (i.e., prosecution vs. defense counsel), and (2) lack of input from the relevant scientific community. A sample of 580 scientists was asked to read a vignette about a hypothetical criminal case and solve a statistical reasoning problem bearing on the case evidence. Participants were randomly assigned to one of three types of party representation (prosecution, defense, or blinded). Approximately half the participants in each representation were given the correct solution in advance of their responses, The correct solution was derived by an independent panel of 12 experts, and presented as "consensus feedback." The other half of participants in each representation received the consensus feedback after providing an initial response, and received an opportunity to change their initial response following that feedback. We found no evidence of an effect of blinding on accuracy. The results revealed a consistent, positive effect of expert consensus feedback on response accuracy. We conclude that expert consensus feedback could improve the validity of expert testimony, and discuss the importance of educating scientists about ways to reduce testimonial bias.
Expert Consensus Feedback Resulted in Improved Performance
- Expert consensus feedback regarding the correct response to the reasoning problem demonstrated the predicted significant effect on response errors: Delivery of feedback resulted in improved performance.
An Advantage Due to Blinding Was Not Observed
- There was modest evidence that an expert's mere knowledge of the hiring counsel (prosecution vs. defense) could influence probabilistic reasoning. Increased accuracy tended to be higher in the party least favored by the evidence (the defense, in this case), suggesting a greater motivation to evaluate the evidence critically.
- However, there was no conclusive evidence that blinding experts to their party representation conferred an advantage — accuracy within this condition was only marginally greater than the prosecution condition and not greater than that of the defense.
- Much more research is needed to demonstrate the real-world utility and feasibility of expert blinding. To the extent that this study's results can be replicated in more naturalistic trial settings, it suggests that greater focus on common threats to sound scientific reasoning (such as base rate neglect) and engagement of the broader scientific community could improve the validity of expert testimony in court cases. In addition, the effect of expert consensus exposure suggests that expert consensus feedback could be a promising way to reduce individual error in expert testimony. Future research is needed to confirm the effect of consensus exposure and to address practical and fairness issues associated with implementing expert consensus exposure approaches in a criminal justice environment.
Table of Contents
Methods and Data Collection
Analyses and Findings
Conclusions and Closing Remarks