RAND researchers worked to understand the costs and benefits of digital engineering in the U.S. Department of Defense (DoD) and develop a decision support framework for digital engineering activities in weapon system programs. The authors reviewed existing literature and developed decision support frameworks incorporating (1) established DoD cost-benefit analysis approaches and (2) established systems engineering decision methodologies.

A Framework for Assessing the Costs and Benefits of Digital Engineering

A Systems Approach

N. Peter Whitehead, Thomas Light, Adrian Luna, Jim Mignano

ResearchPublished Mar 13, 2024

RAND researchers worked to understand the costs and benefits of digital engineering in the U.S. Department of Defense (DoD) and develop a decision support framework for digital engineering activities in weapon system programs. To prepare, the authors reviewed the literature and interviewed stakeholders to understand the current state of digital engineering practice and prior efforts to assess the costs and benefits of digital engineering and model-based systems engineering. They then developed decision support frameworks incorporating (1) established DoD cost-benefit analysis approaches and (2) established systems engineering decision methodologies. Along the way, the authors noted critical issues with rigor and risks in the practice of DoD digital engineering and added that aspect to the study.

This research suggests that cost-benefit decision support for digital engineering is possible at any stage of a weapon system program life cycle if program data have been collected accordingly or if goal-based systems engineering principles are leveraged. Calculating definitive costs and benefits of digital engineering is imperfect because no analyst will have access to an identical weapon system program developed without digital engineering — the counterfactual scenario.

Key Findings

  • Absent a consensus about what constitutes digital engineering, scholars and practitioners lack a common foundation on which to conduct analysis of costs and benefits.
  • Tracking digital engineering investments and distinguishing them from other costs (e.g., systems engineering, multipurpose IT investments) does not appear to be a standard practice.
  • Basing program decisions for digital engineering resource allocation on a systems thinking approach might produce optimal program outcomes.
  • Much effort has been devoted to making the case for digital engineering by projecting possible benefits without, at the same time, evaluating costs.
  • Some authors — and some services — are looking to a digital engineering maturity model as a path to improving outcomes without any cost data, causality map, or risk analysis to support that conclusion.

Recommendations

  • Develop consistency and goal-focused consensus in what digital engineering is (and is not).
  • Establish clear digital engineering program goals and system boundaries to eliminate pretending and to eliminate applying software principles and standards to complex systems.
  • Collect program goal-derived data to support assessment of digital engineering costs and benefits.
  • Eliminate aspirational and general guidance (top-down direction) for implementing digital engineering in such terms as to the maximum extent practical.
  • Promote an environment in which digital engineering is a useful tool among many others to be used in good systems engineering practice and weapon system project management.
  • Objectively analyze enterprise digital engineering tools in their operational context.
  • Promote digital engineering practice and culture where learning from mistakes is as important as achieving success.
  • Establish policy whereby understanding and mitigating risks is a key facet of digital engineering and MBSE practice.
  • Establish a roadmap for identifying and achieving absorptive capacity needs across DoD and the respective services in conjunction with the respective digital transformations.
  • Develop a framework for establishing the hierarchical goals for leveraging data and intellectual property in a weapon system program.
  • Establish DoD weapon system program life cycle decision and milestone gate criteria for effectively leveraging modeling and simulation based on established systems engineering practice.
  • Objectively study and develop recommendations for the long-term life cycles of multi-level classification systems, data, and infrastructures as required.

Order a Print Copy

Format
Paperback
Page count
122 pages
List Price
$33.00
Buy link
Add to Cart

Topics

Document Details

  • Copyright: RAND Corporation
  • Availability: Available
  • Year: 2024
  • Print Format: Paperback
  • Paperback Pages: 122
  • Paperback Price: $33.00
  • Paperback ISBN/EAN: 1-9774-1298-X
  • DOI: https://doi.org/10.7249/RRA2418-1
  • Document Number: RR-A2418-1

Citation

RAND Style Manual
Whitehead, N. Peter, Thomas Light, Adrian Luna, and Jim Mignano, A Framework for Assessing the Costs and Benefits of Digital Engineering: A Systems Approach, RAND Corporation, RR-A2418-1, 2024. As of October 10, 2024: https://www.rand.org/pubs/research_reports/RRA2418-1.html
Chicago Manual of Style
Whitehead, N. Peter, Thomas Light, Adrian Luna, and Jim Mignano, A Framework for Assessing the Costs and Benefits of Digital Engineering: A Systems Approach. Santa Monica, CA: RAND Corporation, 2024. https://www.rand.org/pubs/research_reports/RRA2418-1.html. Also available in print form.
BibTeX RIS

Research conducted by

This research was sponsored by the Under Secretary of Defense for Research and Engineering, Office of Systems Engineering and Architecture, and conducted within the Acquisition and Technology Policy Program of the RAND National Security Research Division (NSRD).

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.