Congress asked six questions about the extent of, and opportunities to improve, data collection and analytic capabilities in the U.S. Department of Defense to inform acquisition decisions. This report identifies and measures extensive capabilities and recent progress in following commercial best practices. However, barriers to improvement include a culture against data sharing because of concerns about security and oversight burden.

Read Online

Assessing Department of Defense Use of Data Analytics and Enabling Data Management to Improve Acquisition Outcomes

by Philip S. Anton, Megan McKernan, Ken Munson, James G. Kallimani, Alexis Levedahl, Irv Blickstein, Jeffrey A. Drezner, Sydne Newberry

Download

Full Document

FormatFile SizeNotes
PDF file 19.5 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Purchase

Purchase Print Copy

 FormatList Price Price
Add to Cart Paperback170 pages $49.95 $39.96 20% Web Discount

Research Questions

  1. What is the extent to which data analytics capabilities have been implemented across the DoD to provide technical support for acquisition program management?
  2. What is the potential to increase the use of analytical capabilities?
  3. What is the current amount of research and development funding for acquisition data analytics capabilities?
  4. What private-sector best practices could be leveraged to minimize the collection and delivery of data?
  5. What steps are being taken to share anonymized acquisition data to researchers and analysts?
  6. Do the curricula at defense acquisition workforce training institutions include appropriate courses on applied and general data analytics and other evaluation-related methods?

In the conference report accompanying the National Defense Authorization Act for Fiscal Year 2017, Congress expressed concern that the U.S. Department of Defense (DoD) "does not sufficiently incorporate data into its acquisition-related learning and decision-making" and asked six questions about "the use of data analysis, measurement, and other evaluation-related methods in DoD acquisition programs." In this report, the authors decompose and measure acquisition functions, data governance, and training to assess how data and associated analytics support DoD acquisition decisionmaking.

The authors found that the DoD is applying a breadth of data analytics to acquisition. Capabilities range from simple data archives and plotting to archives integrated with commercial analytic tools. The DoD has implemented an array of data governance and management practices, but major challenges remain, including a culture against data sharing and concerns about security and oversight burden.

Some commercial breakthroughs in advanced analytics sound promising for DoD acquisition, but some might not be applicable; research is ongoing. Advancement should include developing a data analytics strategy across acquisition domains, expanding data governance and data sharing, and continuing to expand and mature data collection, access, and analytic layers. Also, mechanisms are needed to authorize and ensure protected access to data for both the DoD and external analysts. Improved incentives and understanding of data analytics could encourage decisionmakers to make better use of capabilities.

Key Findings

Data analytics support a broad range of acquisition functions

  • The DoD is using a mix of advanced data analytics, commercial-off-the-shelf tools, and simpler approaches.

Data analytics contribute to decisionmaking

  • Data analytics are applied to acquisition decisions by and through a wide range of acquisition functions.
  • These analyses inform acquisition management, insight, oversight, execution, and decisions at all levels, including program managers, contracting officers, engineers, auditors, oversight executives, and DoD leadership.
  • However, data analysis may or may not be equally weighted against other considerations by decisionmakers. At times, there may be other priorities (e.g., politics, optics) that will override analysis.

Capabilities are advancing, but more remains to be done

  • The DoD has made progress in improving its data and analysis capabilities, including implementations of the latest commercial best practices and analytic tools on top of information systems.
  • Although data analytics training is expanding, as in the private sector, the pool of needed experts who understand both the application domain (acquisition) and data analytics is limited.

Recommendations

  • Congress could consider clarifying in 10 U.S.C. 2222(e) that all acquisition and sustainment data are common enterprise data and thus available across the DoD.
  • Congress could make permanent federally funded research and development centers' access to sensitive data under Section 235 of the National Defense Authorization Act for Fiscal Year 2017.
  • The DoD should facilitate data analytics by creating a cross-domain DoD data analysis strategy: identify how to address disincentives to data sharing; plan long-term investments; analyze policies and approaches for granting DoD-wide access to various DoD information systems for government and contractor analysts; identify the data needed, at what level, and for what purposes, given costs and benefits; and perform policy and process analysis on data aggregation and classification upgrades to ensure more-consistent application.
  • Expectations need to be moderated on what data analytics can do for DoD acquisition. Recent commercial breakthroughs in analytics do not always apply to certain acquisition data. Also, problematic decisions can result from strategic considerations rather than from a lack of data analytics.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Approach and Methodology

  • Chapter Three

    Findings on Congressional Questions

  • Chapter Four

    Conclusions

Research conducted by

This research was sponsored by the Director, Acquisition Analytics and Policy, within the OUSD(A&S) and conducted within the Acquisition and Technology Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.