For the purposes of capability development planning, RAND Project AIR FORCE focuses on analytical methods that can be used for decisionmaking under conditions of deep uncertainty about the future threat environment, rate of technological advancement, and budgeting. Such methods can also inform investment trades across warfighting domains and functional areas, as well as other responsibilities of the Air Force Warfighting Integration Capability.

Air Force Capability Development Planning

Analytical Methods to Support Investment Decisions

James A. Leftwich, Debra Knopman, Jordan R. Fischbach, Michael J. D. Vermeer, Kristin Van Abel, Nidhi Kalra

ResearchPublished Dec 11, 2019

In view of uncertainties about the future threat environment, trajectories of technological development, and shifting budgeting priorities, RAND Project AIR FORCE examined analytical methods that would best guide the recently established Air Force Warfighting Integration Capability in capability development planning (CDP), as well as concept development and future force design. In their review, researchers found that specific methods of decisionmaking under deep uncertainty (DMDU) can provide the most suitable means for arriving at solutions that are flexible, adaptable, and robust and guiding investment pathways and modernization efforts.

The method highlighted, Robust Decisionmaking, rests on a simple concept. Rather than using models and data to assess policies under a single set of assumptions, RDM runs models over hundreds to thousands of different sets of assumptions about the problem space with the aim of understanding how plans perform under many plausible conditions. Each of the four steps of RDM — decision-framing, case generation, vulnerability assessment and scenario discovery, and trade-off analysis — feeds into the next, providing stakeholders and decisionmakers with a more informed tradespace. By exposing vulnerabilities of different options under different scenarios, RDM can enable stakeholders to engage in a rich dialogue about which risks or vulnerabilities are acceptable, as well as to review assumptions made in framing the problem and make adjustments as needed.

Key Findings

  • Compared with existing methods currently in use for CDP, DMDU methods — and specifically the RDM paradigm — could support investment decisionmaking in situations when evidence is lacking to quantify the probability of future threats and risks and in the face of deep uncertainties.
  • A workshop in which RAND experts and Air Force stakeholders explored ways to frame a decision problem revealed the layers of complexity embedded within the CDP challenge to produce decision-quality analysis to support design and planning choices.
  • Workshop participants identified technical limitations to implementing RDM, including availability of models and their fidelity.
  • Participants also noted that the practicalities of developing competencies in a new approach for which the capabilities may need to be built rather than simply adapted will be a challenge. A related issue is the adequacy of internal staffing and capacity to support CDP.
  • RDM and similar methods represent a fundamental shift in problem-solving from prediction-then-act analysis and decisionmaking to a risk management approach of seeking robust solutions across a range of future threat environments and other uncertainties. The key output of this type of approach is not a single answer but rather trade-off curves or other visualizations that compare performance of alternative strategies or portfolios of options with the status quo or other baseline in terms of metrics that reflect what is most important to the Air Force.

Recommendations

  • A first step in strategic developing planning and experimentation would be to choose a timely and relevant investment decision problem and demonstrate the robustness of a number of alternative investment portfolios using a mostly qualitative approach that would alleviate the need for sophisticated — or nonexistent — models capable of encompassing a range of emerging technologies, such as quantum computing, artificial intelligence, hypersonics, or directed energy. The analysis should consider performance under a wide range of future conditions and visualize trade-offs among these alternatives across the performance metrics relevant to Air Force decisionmakers.
  • To be workable, a test application will need to be amenable to analysis. Models should be understandable and appropriate for exploring broad-ranging uncertainty analysis; in the absence of numerical models, conceptual models may be used.
  • In the test application, there should be basic access to data that could be used to establish performance under baseline or status-quo concepts of operation and existing capabilities.
  • In the test application, classification levels should be managed to enable serious analysis and reveal key developments underway but not overly restrict the necessary audience of key analysts and decisionmakers.
  • Air Force stakeholders should be willing to engage in an iterative process in developing a decision-framing matrix, including the identification of performance metrics, key uncertainties, modeling tools, and investment/divestment strategies (levers).
  • The problem can be scoped to accommodate a qualitative analysis within 8–12 months of initiation, including all relevant data collection, modeling, analysis, visualization, and interactive review of and response to results.

Order a Print Copy

Format
Paperback
Page count
94 pages
List Price
$30.00
Buy link
Add to Cart

Topics

Document Details

  • Copyright: RAND Corporation
  • Availability: Available
  • Year: 2019
  • Print Format: Paperback
  • Paperback Pages: 94
  • Paperback Price: $30.00
  • Paperback ISBN/EAN: 978-1-9774-0308-7
  • DOI: https://doi.org/10.7249/RR2931
  • Document Number: RR-2931-AF

Citation

RAND Style Manual
Leftwich, James A., Debra Knopman, Jordan R. Fischbach, Michael J. D. Vermeer, Kristin Van Abel, and Nidhi Kalra, Air Force Capability Development Planning: Analytical Methods to Support Investment Decisions, RAND Corporation, RR-2931-AF, 2019. As of September 23, 2024: https://www.rand.org/pubs/research_reports/RR2931.html
Chicago Manual of Style
Leftwich, James A., Debra Knopman, Jordan R. Fischbach, Michael J. D. Vermeer, Kristin Van Abel, and Nidhi Kalra, Air Force Capability Development Planning: Analytical Methods to Support Investment Decisions. Santa Monica, CA: RAND Corporation, 2019. https://www.rand.org/pubs/research_reports/RR2931.html. Also available in print form.
BibTeX RIS

Research conducted by

The research reported here was commissioned by the Air Force Strategic Development Planning and Experimentation Office and conducted by the Resource Management Program within RAND Project AIR FORCE.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.