Research Questions

  1. What is the state of AI relevant to the DoD?
  2. What is the DoD’s current posture in AI?
  3. What internal actions, external engagements, and potential legislative or regulatory actions might enhance the DoD’s posture in AI?

The 2019 National Defense Authorization Act mandated a study on artificial intelligence (AI) topics. In this report, RAND Corporation researchers assess the state of AI relevant to the U.S. Department of Defense (DoD), and address misconceptions about AI; they carry out an independent and introspective assessment of the Department of Defense's posture for AI; and they share a set of recommendations for internal actions, external engagements, and potential legislative or regulatory actions to enhance the Department of Defense's posture in AI.

Key Findings

The state of AI and its implications for DoD

  • Many technologies underpin AI. Recent significant technological advances have been primarily in supervised machine learning, specifically deep learning. Success in deep learning is predicated on the availability of large labeled data sets and significant computing power to train the models. The current crop of approaches are fragile, artisanal, and are optimized for commercial rather than DoD uses.
  • The current state of AI verification, validation, test, and evaluation (VVT&E) is nowhere close to ensuring the performance and safety of AI applications, particularly where safety-critical systems are concerned. Although this is not a problem unique to DoD, it is one that significantly affects DoD.
  • It is important for DoD to maintain realistic expectations for both performance and timelines in going from demonstrations of the art of the possible to deployments at scale. As a rule of thumb, investments made today can be expected to yield at scale deployment in the near-term for enterprise AI, mid-term for most mission support AI, and long-term for most operational AI.

DoD's posture in AI is significantly challenged across all dimensions of the authors' assessment

  • DoD lacks baselines and metrics in conjunction with its AI vision.
  • The Joint Artificial Intelligence Center (JAIC) lacks visibility and authorities to carry out its present role. It also lacks a five-year strategic road map, and a precise objective allowing it to formulate one.
  • The lack of longer-term budget commitments might hinder the JAIC's success.
  • The service AI annexes lack baselines and metrics. When mandates and authorities of the service AI organizations exist, they appear to be limited.
  • Communication channels between the builders and users of AI within DoD are sparse.
  • Innovation within DoD might be hampered by current practices and processes or their implementation. Current practices and processes also inhibit the ability to bring in external innovation.
  • Data are not collected and stored at every opportunity, and access to existing data is limited; the data that exist are not always understandable or traceable. Lack of interoperability in systems across DoD also creates challenges. There is ambiguity in data ownership where external vendors are involved.
  • DoD lacks clear mechanisms for defining and tracking AI talent, and it struggles to grow and cultivate AI talent.

Recommendations

  • DoD should adapt AI governance structures that align authorities and resources with their mission of scaling AI.
  • The JAIC and each centralized service organization should develop a five-year strategic road maps to scale AI.
  • The JAIC, working with DoD partners, should carry out annual or biannual portfolio reviews of DoD-wide investments in AI.
  • The JAIC should organize a technical workshop showcasing AI programs DoD-wide.
  • DoD should advance the science and practice of VVT&E of AI systems, working in close partnership with industry and academia.
  • All funded AI efforts should include a budget for AI VVT&E, including any critically needed testing infrastructure.
  • All agencies within DoD should create or strengthen mechanisms for connecting AI researchers, technology developers, and operators.
  • DoD should recognize data as critical resources, continue instituting practices for their collection and curation, and increase sharing while resolving issues in protecting the data after sharing and during analysis and use.
  • The chief data officer should make a selection of DoD data sets available to the AI community to spur innovation and enhance external engagement with DoD.
  • DoD should embrace permeability, and an appropriate level of openness, as a means of enhancing DoD's access to AI talent.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Analytical Framework and Methodology

  • Chapter Three

    The DoD-Relevant State of Artificial Intelligence

  • Chapter Four

    DoD Posture for Artificial Intelligence

  • Chapter Five

    Recommendations

  • Appendix A

    Details of the Analytic Methodology

  • Appendix B

    Insights from Federal Interviews

  • Appendix C

    Insights from Interviews with Industry and Academia

  • Appendix D

    Insights from Historical Case Studies

  • Appendix E

    Defining Artificial Intelligence

This research was sponsored by the DoD Joint Artificial Intelligence Center (JAIC) and was conducted within the Acquisition and Technology Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.

This report is part of the RAND Corporation research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.