Congress asked six questions about the extent of, and opportunities to improve, data collection and analytic capabilities in the U.S. Department of Defense to inform acquisition decisions. This report identifies and measures extensive capabilities and recent progress in following commercial best practices. However, barriers to improvement include a culture against data sharing because of concerns about security and oversight burden.

Assessing Department of Defense Use of Data Analytics and Enabling Data Management to Improve Acquisition Outcomes

Philip S. Anton, Megan McKernan, Ken Munson, James G. Kallimani, Alexis Levedahl, Irv Blickstein, Jeffrey A. Drezner, Sydne J. Newberry

ResearchPublished Aug 13, 2019

In the conference report accompanying the National Defense Authorization Act for Fiscal Year 2017, Congress expressed concern that the U.S. Department of Defense (DoD) "does not sufficiently incorporate data into its acquisition-related learning and decision-making" and asked six questions about "the use of data analysis, measurement, and other evaluation-related methods in DoD acquisition programs." In this report, the authors decompose and measure acquisition functions, data governance, and training to assess how data and associated analytics support DoD acquisition decisionmaking.

The authors found that the DoD is applying a breadth of data analytics to acquisition. Capabilities range from simple data archives and plotting to archives integrated with commercial analytic tools. The DoD has implemented an array of data governance and management practices, but major challenges remain, including a culture against data sharing and concerns about security and oversight burden.

Some commercial breakthroughs in advanced analytics sound promising for DoD acquisition, but some might not be applicable; research is ongoing. Advancement should include developing a data analytics strategy across acquisition domains, expanding data governance and data sharing, and continuing to expand and mature data collection, access, and analytic layers. Also, mechanisms are needed to authorize and ensure protected access to data for both the DoD and external analysts. Improved incentives and understanding of data analytics could encourage decisionmakers to make better use of capabilities.

Key Findings

Data analytics support a broad range of acquisition functions

  • The DoD is using a mix of advanced data analytics, commercial-off-the-shelf tools, and simpler approaches.

Data analytics contribute to decisionmaking

  • Data analytics are applied to acquisition decisions by and through a wide range of acquisition functions.
  • These analyses inform acquisition management, insight, oversight, execution, and decisions at all levels, including program managers, contracting officers, engineers, auditors, oversight executives, and DoD leadership.
  • However, data analysis may or may not be equally weighted against other considerations by decisionmakers. At times, there may be other priorities (e.g., politics, optics) that will override analysis.

Capabilities are advancing, but more remains to be done

  • The DoD has made progress in improving its data and analysis capabilities, including implementations of the latest commercial best practices and analytic tools on top of information systems.
  • Although data analytics training is expanding, as in the private sector, the pool of needed experts who understand both the application domain (acquisition) and data analytics is limited.

Recommendations

  • Congress could consider clarifying in 10 U.S.C. 2222(e) that all acquisition and sustainment data are common enterprise data and thus available across the DoD.
  • Congress could make permanent federally funded research and development centers' access to sensitive data under Section 235 of the National Defense Authorization Act for Fiscal Year 2017.
  • The DoD should facilitate data analytics by creating a cross-domain DoD data analysis strategy: identify how to address disincentives to data sharing; plan long-term investments; analyze policies and approaches for granting DoD-wide access to various DoD information systems for government and contractor analysts; identify the data needed, at what level, and for what purposes, given costs and benefits; and perform policy and process analysis on data aggregation and classification upgrades to ensure more-consistent application.
  • Expectations need to be moderated on what data analytics can do for DoD acquisition. Recent commercial breakthroughs in analytics do not always apply to certain acquisition data. Also, problematic decisions can result from strategic considerations rather than from a lack of data analytics.

Order a Print Copy

Format
Paperback
Page count
170 pages
List Price
$49.95
Buy link
Add to Cart

Topics

Document Details

  • Copyright: RAND Corporation
  • Availability: Available
  • Year: 2019
  • Print Format: Paperback
  • Paperback Pages: 170
  • Paperback Price: $49.95
  • Paperback ISBN/EAN: 978-1-9774-0326-1
  • DOI: https://doi.org/10.7249/RR3136
  • Document Number: RR-3136-OSD

Citation

RAND Style Manual
Anton, Philip S., Megan McKernan, Ken Munson, James G. Kallimani, Alexis Levedahl, Irv Blickstein, Jeffrey A. Drezner, and Sydne J. Newberry, Assessing Department of Defense Use of Data Analytics and Enabling Data Management to Improve Acquisition Outcomes, RAND Corporation, RR-3136-OSD, 2019. As of October 15, 2024: https://www.rand.org/pubs/research_reports/RR3136.html
Chicago Manual of Style
Anton, Philip S., Megan McKernan, Ken Munson, James G. Kallimani, Alexis Levedahl, Irv Blickstein, Jeffrey A. Drezner, and Sydne J. Newberry, Assessing Department of Defense Use of Data Analytics and Enabling Data Management to Improve Acquisition Outcomes. Santa Monica, CA: RAND Corporation, 2019. https://www.rand.org/pubs/research_reports/RR3136.html. Also available in print form.
BibTeX RIS

Research conducted by

This research was sponsored by the Director, Acquisition Analytics and Policy, within the OUSD(A&S) and conducted within the Acquisition and Technology Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.