This report, which describes the potential for artificial intelligence (AI) systems to assist in Air Force command and control (C2) from a technical perspective, presents an analytical framework for assessing the suitability of a given AI system for a given C2 problem. The authors also provide metrics that can be used to evaluate AI systems and to demonstrate and socialize their utility.

Cover: Exploring the Feasibility and Utility of Machine Learning-Assisted Command and Control
Read Online

Exploring the Feasibility and Utility of Machine Learning-Assisted Command and Control

Volume 1, Findings and Recommendations

by Matthew Walsh, Lance Menthe, Edward Geist, Eric Hastings, Joshua Kerrigan, Jasmin Léveillé, Joshua Margolis, Nicholas Martin, Brian P. Donnelly

Download

Download eBook for Free

Full Document

FormatFile SizeNotes
PDF file 0.9 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Summary

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Purchase

Purchase Print Copy

 FormatList Price Price
Add to Cart Paperback90 pages $22.50 $18.00 20% Web Discount

Research Questions

  1. Can AI systems that are developed and deployed in academic and commercial contexts be of value in military contexts?
  2. Can an analytical framework be developed for understanding the suitability of different AI systems for different C2 problems and for identifying pervasive technology gaps?
  3. Can sufficient metrics of merit be developed for evaluating the performance, effectiveness, and suitability of AI systems for C2 problems?

This report concerns the potential for artificial intelligence (AI) systems to assist in Air Force command and control (C2) from a technical perspective. The authors present an analytical framework for assessing the suitability of a given AI system for a given C2 problem. The purpose of the framework is to identify AI systems that address the distinct needs of different C2 problems and to identify the technical gaps that remain. Although the authors focus on C2, the analytical framework applies to other warfighting functions and services as well.

The goal of C2 is to enable what is operationally possible by planning, synchronizing, and integrating forces in time and purpose. The authors first present a taxonomy of problem characteristics and apply them to numerous games and C2 processes. Recent commercial applications of AI systems underscore that AI offers real-world value and can function successfully as components of larger human-machine teams. The authors outline a taxonomy of solution capabilities and apply them to numerous AI systems.

While primarily focusing on determining alignment between AI systems and C2 processes, the report's analysis of C2 processes is also informative with respect to pervasive technological capabilities that will be required of Department of Defense (DoD) AI systems. Finally, the authors develop metrics — based on measures of performance, effectiveness, and suitability — that can be used to evaluate AI systems, once implemented, and to demonstrate and socialize their utility.

Key Findings

C2 processes are very different from many of the games and environments used to develop and demonstrate AI systems

  • Game-playing algorithms exploit regularity to achieve superhuman performance, but nature and the adversary intervene to break this simplifying assumption in military tasks.
  • Characterizing and developing representative problems and environments will enable research, development, testing, and evaluation of AI systems under conditions representative of DoD problem sets, thereby increasing transferability to operational environments.

The distinctive nature of C2 processes calls for AI systems different from those optimized for game play

  • Understanding the capabilities and limitations of existing AI systems will allow the Air Force to identify systems that are suitable for different C2 processes.
  • Choosing the right approach at problem outset can substantially reduce application development time, increase solution quality, and decrease risk associated with transitioning the solution.

New guidance, infrastructure, and metrics are needed to evaluate applications of AI to C2

  • Establishing and operationalizing measures of merit will enable the evaluation and comparison of potential AI systems.
  • Additionally, measures of merit provide a way to communicate the return on investment of AI-enabled C2.

Hybrid approaches are needed to deal with the multitude of problem characteristics that are present in C2 processes

  • Given the generality of the analytical framework and the emergence of Joint All-Domain C2, all these conclusions and recommendations extend to the pursuit of AI across DoD.

Recommendations

  • Use the structured method described in this report to systematically analyze the characteristics of games, problems, and C2 processes to determine where existing AI test beds are representative and nonrepresentative of C2 tasks.
  • Develop new AI test beds with problem characteristics that are representative of C2 tasks in kind and in intensity.
  • Use the structured method described in this report to identify and invest in high-priority solution capabilities called for across a wide range of C2 processes and not currently available (e.g., robustness and assuredness).
  • Use the structured method described in this report to evaluate alignment between the characteristics of potential AI systems and particular C2 processes to prioritize which systems to develop.
  • Develop metrics for AI solutions that assess capabilities beyond algorithm soundness and optimality (e.g., robustness and explainability).
  • Use the structured method described in this report to identify key measures of performance, effectiveness, and suitability for a given C2 process.
  • Perform a comprehensive assessment of AI systems for a given C2 process based on identified measures of merit.
  • Identify, reuse, and combine algorithmic solutions that confer critical AI system capabilities.

Table of Contents

  • Chapter One

    Introduction and Project Overview

  • Chapter Two

    Taxonomy of Problem Characteristics

  • Chapter Three

    Taxonomy of Solution Capabilities

  • Chapter Four

    Mapping Problem Characteristics to Solution Capabilities

  • Chapter Five

    Metrics for Evaluating Artificial Intelligence Solutions

  • Chapter Six

    Conclusion and Recommendations

Research conducted by

The research described in this report was prepared for the the Air Force Research Laboratory, Information Directorate (AFRL/RI) and conducted by the Force Modernization and Employment Program within RAND Project AIR FORCE.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.