The sharing of intelligence and law enforcement information is a central part of U.S. domestic security efforts, yet there are concerns about the effectiveness of information-sharing and fusion activities and their value relative to the public funds invested in them. This report lays out the challenges of evaluating information-sharing efforts that seek to achieve multiple goals simultaneously; reviews past evaluations of information-sharing programs; and lays out a path to improving the evaluation of such efforts.

How Do We Know What Information Sharing Is Really Worth?

Exploring Methodologies to Measure the Value of Information Sharing and Fusion Efforts

Brian A. Jackson

ResearchPublished Jun 18, 2014

Cover: How Do We Know What Information Sharing Is Really Worth?
Download PDF
Additional Downloads

Includes other ebook formats

Since the terrorist attacks of September 11, 2001, the sharing of intelligence and law enforcement information has been a central part of U.S. domestic security efforts. Though much of the public debate about such sharing focuses on addressing the threat of terrorism, organizations at all levels of government routinely share varied types of information through multiagency information systems, collaborative groups, and other links. Given resource constraints, there are concerns about the effectiveness of information-sharing and fusion activities and, therefore, their value relative to the public funds invested in them. Solid methods for evaluating these efforts are lacking, however, limiting the ability to make informed policy decisions. Drawing on a substantial literature review and synthesis, this report lays out the challenges of evaluating information-sharing efforts that frequently seek to achieve multiple goals simultaneously; reviews past evaluations of information-sharing programs; and lays out a path to improving the evaluation of such efforts going forward.

Key Takeaways

Improved Methodologies Are Needed to Measure the Value of Information-Sharing and Fusion Efforts

  • The lack of literature on evaluating information sharing, coupled with passionate arguments both for and against the value of such efforts, has produced a stunted policy debate that is insufficient to support reasoned and reasonable tradeoffs among these programs and other ways to pursue the goals they are designed to advance.
  • With a clearer framing of the evaluable goals sharing programs are pursuing, data on organizational outcomes can be linked to different ways of assessing the "dosage" of exposure to information sharing at different levels.
  • New analytic techniques that enable matching of individual users and comparison of outcomes at a very disaggregated level appear particularly promising for assessing existing initiatives.
  • Systematic approaches like "but-for" analyses and structures that tie sharing to the outcomes it is designed to achieve provide paths toward improved measurement of the value of information-sharing and fusion efforts.

Recommendations

  • Methodologies to measure the value of information-sharing and fusion efforts should focus on outcome measures.
  • Making analytic investments to improve methodologies for assessing the value of information-sharing systems is important if future decisions about the preservation, maintenance, or expansion of such systems is to be based on objective data instead of assumptions and anecdotal evidence of their effects and value.

Topics

Document Details

Citation

Chicago Manual of Style

Jackson, Brian A., How Do We Know What Information Sharing Is Really Worth? Exploring Methodologies to Measure the Value of Information Sharing and Fusion Efforts. Santa Monica, CA: RAND Corporation, 2014. https://www.rand.org/pubs/research_reports/RR380.html.
BibTeX RIS

Research conducted by

This research was conducted within the RAND Homeland Security and Defense Center, a joint center of RAND Justice, Infrastructure, and Environment, and the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.