In this report, researchers explore measurement methods to assess how information-sharing affects criminal justice outcomes and how the practicalities of law enforcement work practices can affect measurement. In addition, the authors offer lessons for future policy evaluation and information system design.

Read Online

Knowing More, But Accomplishing What?

Developing Approaches to Measure the Effects of Information-Sharing on Criminal Justice Outcomes

by Brian A. Jackson, Lane F. Burgette, Caroline Stevens, Claude Messan Setodji, Erinn Herberman, Stephanie Ann Kovalchik, Katie Mugg, Meagan Cahill, Jessica Hwang, Joshua Lawrence Traub

Download

Full Document

FormatFile SizeNotes
PDF file 1.9 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Purchase

Purchase Print Copy

 FormatList Price Price
Add to Cart Paperback74 pages $18.00 $14.40 20% Web Discount

Research Questions

  1. How can we measure the effects of information-sharing?
  2. What lessons can this analysis provide for future measurement efforts and designs for information-sharing systems?

Information-sharing became a central element of the policy debate about U.S. homeland and national security after the September 11, 2001, terrorist attacks. However, sharing of information across jurisdictional lines is just as important for everyday criminal justice efforts to prevent and investigate crime, and systems to provide such capabilities have been in place for many years. Despite widespread belief that information-sharing is valuable, there have been relatively limited efforts to measure its effect on criminal justice outcomes. To help address this need, we examined the measurement of information-sharing effects from the strategic to the tactical levels, with a focus on developing reliable measurements that capture the range of ways sharing can affect outcomes and how the practicalities of law enforcement work practices can affect measurement. In collaboration with an advanced regional information-sharing agency, we developed techniques to examine the effects of multiple types of data-sharing at the officer, case, and offender levels. Analyses showed significant correlations between different types of sharing on the level of interagency involvement in cases for individual offenders, on the timing and likelihood of specific law enforcement events, and on the likelihood of individual police officers to be involved in cross-jurisdictional arrests. In addition, we explored lessons for future policy evaluation and information system design to facilitate measurement.

Key Findings

Measuring the Effects of Information-Sharing

  • Measuring the effects of information-sharing is not straightforward, because information is not being shared simply for the sake of sharing but with the intent of doing something else better. Many typical quantitative measures (e.g., amount of information shared by such systems) measure process or outputs, not necessarily the effect of sharing information.
  • We teamed up with the Automated Regional Justice Information System (ARJIS) to provide a real-world case study in which to experiment with measuring the effects of different types of data-sharing. As part of this effort, we interviewed users of ARJIS information-sharing tools in various roles and agencies to identify how the ways that criminal justice practitioners performed their tasks would shape the challenge of measuring the effect of information-sharing on their efficiency or effectiveness.
  • We also developed measures for three different types of sharing tools provided by ARJIS to its users. The measures examined the effect of ARJIS applications that made specific types of information more prominent for users, allowed sharing of information among groups of users, and offered tools to search cross-jurisdictional databases on the speed and likelihood of different law enforcement events, as well as the involvement of agencies with different jurisdictions in making arrests or managing specific types of offenders.

Recommendations

  • Some of the biggest takeaways of this effort were lessons on how information systems could be designed to make this sort of analysis more straightforward by building in specific features or addressing key technical problems. These lessons would make it easier to replicate and expand on the criminal justice system's ability to link information-sharing to justice outcomes.
  • Challenges we encountered included ways of addressing how differences in law enforcement practitioner roles and work structure would affect measurement and navigating technical challenges of cross-jurisdictional data integration, data quality issues, and other concerns.
  • Among the key lessons learned were, first, for some of the system design issues, our results simply reemphasize the importance of initiatives that are already under way to facilitate sharing of data across agencies. Second, standardization across agencies, by agreeing on and adopting a unique identifier for individuals, would help address difficulties in tracking both criminal activity and actions to address that activity across jurisdictional lines. Third, where storage capabilities make it possible to do so, it would be valuable to retain more administrative data in justice information systems to enable evaluation efforts. Fourth, building system feedback mechanisms that link the use of specific data to outcomes would be an opportunity for systems to make themselves not only easier to assess but also, and more importantly, operationally useful.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    How Can We Measure the Effects of Information-Sharing?

  • Chapter Three

    Developing and Testing Measures with the Automated Regional Justice Information System

  • Chapter Four

    Conclusions

  • Appendix

    User Interview Questions

The research described in this report was sponsored by the National Institute of Justice and conducted by the Justice Policy Program within RAND Justice, Infrastructure, and Environment.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.