Through analysis of alternative courses of action and their effects on probability distributions estimated for performance, schedule, and cost, acquisition decisionmakers can understand the implications of certain risk-mitigating actions. They can examine technology, schedule, and cost trades between or within individual systems.
Download
Download eBook for Free
Full Document
| Format | File Size | Notes |
|---|---|---|
| PDF file | 2.1 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Research Synopsis
| Format | File Size | Notes |
|---|---|---|
| PDF file | 0.1 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Purchase
Purchase Print Copy
| Format | List Price | Price | |
|---|---|---|---|
| Add to Cart | Paperback184 pages | $32.50 | $26.00 20% Web Discount |
Research Questions
- What are the current methodologies, processes, and results of the U.S. Army Materiel Systems Analysis Activity Risk Integrated Product Team?
- What are the team's key analysis capability gaps?
- What would a risk-adjusted trade-space methodology and tool look like, and how would they work?
This report describes the methodology for risk-informed trade-space analysis developed by the U.S. Army Materiel Systems Analysis Activity Risk Integrated Product Team and researchers from the RAND Corporation and the first iteration of the associated Risk-Informed Trade Analysis Model. The framework combines elements of system engineering, production economics, and risk analysis to functionally and probabilistically relate performance, schedule, and cost outcomes and their uncertainties holistically and understandably. The technology development process is conceptualized as one in which the physical system is described as a portfolio of technologies with associated technical capabilities, and the completion of each technology's development is a discrete random variable. The performance characteristics of the final system are stochastic. In addition, the time of technology development is also stochastic and, in part, drives the overall cost of the system. In a departure from previous analyses, the authors incorporate technology-specific courses of action, or risk-mitigation behaviors, that are assumed to take place in the event that the technology is not developed at the milestone date. For example, one might assume that a lesser-performing but existing substitute could replace a particular developmental technology or that, if that technology is of critical importance, the schedule might be allowed to slip. Through analysis of alternative courses of action and their effects on the resultant probability distributions estimated for performance, schedule, and cost, decisionmakers have a means to understand the implications of certain risk-mitigating actions. Technology, schedule, and cost trades can be examined between or within individual systems.
Key Findings
The Technical Risk Assessment, Risk Workshops, and Schedule Risk Data Decision Methodology Deliver Inputs to Determine Acquisition Risks
- The U.S. Army Materiel Systems Analysis Activity (AMSAA) Technical Risk Assessment measures the risk that a technology relevant to an Army acquisition system will not be delivered within the desired time frame, cost target, and performance designation. AMSAA also developed a system-level schedule risk modeling approach called Schedule Risk Data Decision Methodology.
The Performance, Schedule, and Cost Dimensions of the Methodologies Are Treated as Independent
- AMSAA's incumbent methodologies lack a system-level approach in which performance, schedule, and cost are interdependent.
The Risk-Informed Trade Analysis Model (RTRAM) and Methodology Provide a System-Level Approach to Acquisition Risk
- RTRAM and its underlying methodology provide an approach to quantitatively and consistently estimate output distributions related to performance, schedule, and cost based on AMSAA's risk workshop and previously developed methodologies. It allows users to investigate multidimensional trade-offs both within and between weapon systems prior to production, using elements of system engineering, production economics, and risk analysis to functionally and probabilistically relate performance, schedule, and cost outcomes and their related uncertainties holistically and understandably.
Recommendations
- AMSAA should develop and integrate baseline-independent technical performance metrics into the risk workshop process. These measures could be objective and relative to an appropriate measurement scale where available, objective and relative to a baseline technology, or subjective and relative to a baseline technology. The important attribute of such measures is that they can be used to compare alternative technologies relative to some standardized baseline.
- The team needs a continuous distribution that describes stochastic system-level schedule outcomes, rather than the discrete probability and consequence of not meeting a particular deadline for a particular technology. Also, the three broad outcomes of interest are functionally interrelated, at both the technology and the system levels. Ideally, a fully formed trade-space analysis should recognize these interconnections, such that any proposed changes in design variables percolate through all outcomes in each dimension.
- Because current Army costing procedures do not typically incorporate explicit structural relationships between performance, schedule, and cost estimates and the task of performing cost analysis typically falls to different organizations that perform technical and schedule analysis, the team needs to develop additional cost methodologies that are consistent with the technological and schedule risks.
Table of Contents
Chapter One
Introduction
Chapter Two
Current Cost, Schedule, and Performance Risk Methodologies Within the Army
Chapter Three
Theoretical Framework for Risk-Informed Trade-Space Analysis
Chapter Four
Proposed Conceptual Model of Risk-Informed Trade-Space Analysis in the Acquisition of Weapon Systems
Chapter Five
The Model
Chapter Six
An Example Application of the Risk-Informed Trade Analysis Model Using a Demonstration Tool
Chapter Seven
Conclusions
Appendix A
User Manual for the Risk-Informed Trade Analysis Model Demonstration Tool, Version 1.0
Appendix B
Outline of Specific Assumptions and Methods in the Risk- Informed Trade Analysis Model
Appendix C
Expert Elicitation and Minimizing Bias from Heuristics
Appendix D
Computer Code
Research conducted by
This research was sponsored by Randolph Wheeler, AMSAAWR, and conducted within the Force Development and Technology program within the RAND Arroyo Center.
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.