Download

Download eBook for Free

Full Document

FormatFile SizeNotes
PDF file 2.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Synopsis

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Purchase

Purchase Print Copy

 FormatList Price Price
Add to Cart Paperback184 pages $32.50 $26.00 20% Web Discount

Research Questions

  1. What are the current methodologies, processes, and results of the U.S. Army Materiel Systems Analysis Activity Risk Integrated Product Team?
  2. What are the team's key analysis capability gaps?
  3. What would a risk-adjusted trade-space methodology and tool look like, and how would they work?

This report describes the methodology for risk-informed trade-space analysis developed by the U.S. Army Materiel Systems Analysis Activity Risk Integrated Product Team and researchers from the RAND Corporation and the first iteration of the associated Risk-Informed Trade Analysis Model. The framework combines elements of system engineering, production economics, and risk analysis to functionally and probabilistically relate performance, schedule, and cost outcomes and their uncertainties holistically and understandably. The technology development process is conceptualized as one in which the physical system is described as a portfolio of technologies with associated technical capabilities, and the completion of each technology's development is a discrete random variable. The performance characteristics of the final system are stochastic. In addition, the time of technology development is also stochastic and, in part, drives the overall cost of the system. In a departure from previous analyses, the authors incorporate technology-specific courses of action, or risk-mitigation behaviors, that are assumed to take place in the event that the technology is not developed at the milestone date. For example, one might assume that a lesser-performing but existing substitute could replace a particular developmental technology or that, if that technology is of critical importance, the schedule might be allowed to slip. Through analysis of alternative courses of action and their effects on the resultant probability distributions estimated for performance, schedule, and cost, decisionmakers have a means to understand the implications of certain risk-mitigating actions. Technology, schedule, and cost trades can be examined between or within individual systems.

Key Findings

The Technical Risk Assessment, Risk Workshops, and Schedule Risk Data Decision Methodology Deliver Inputs to Determine Acquisition Risks

  • The U.S. Army Materiel Systems Analysis Activity (AMSAA) Technical Risk Assessment measures the risk that a technology relevant to an Army acquisition system will not be delivered within the desired time frame, cost target, and performance designation. AMSAA also developed a system-level schedule risk modeling approach called Schedule Risk Data Decision Methodology.

The Performance, Schedule, and Cost Dimensions of the Methodologies Are Treated as Independent

  • AMSAA's incumbent methodologies lack a system-level approach in which performance, schedule, and cost are interdependent.

The Risk-Informed Trade Analysis Model (RTRAM) and Methodology Provide a System-Level Approach to Acquisition Risk

  • RTRAM and its underlying methodology provide an approach to quantitatively and consistently estimate output distributions related to performance, schedule, and cost based on AMSAA's risk workshop and previously developed methodologies. It allows users to investigate multidimensional trade-offs both within and between weapon systems prior to production, using elements of system engineering, production economics, and risk analysis to functionally and probabilistically relate performance, schedule, and cost outcomes and their related uncertainties holistically and understandably.

Recommendations

  • AMSAA should develop and integrate baseline-independent technical performance metrics into the risk workshop process. These measures could be objective and relative to an appropriate measurement scale where available, objective and relative to a baseline technology, or subjective and relative to a baseline technology. The important attribute of such measures is that they can be used to compare alternative technologies relative to some standardized baseline.
  • The team needs a continuous distribution that describes stochastic system-level schedule outcomes, rather than the discrete probability and consequence of not meeting a particular deadline for a particular technology. Also, the three broad outcomes of interest are functionally interrelated, at both the technology and the system levels. Ideally, a fully formed trade-space analysis should recognize these interconnections, such that any proposed changes in design variables percolate through all outcomes in each dimension.
  • Because current Army costing procedures do not typically incorporate explicit structural relationships between performance, schedule, and cost estimates and the task of performing cost analysis typically falls to different organizations that perform technical and schedule analysis, the team needs to develop additional cost methodologies that are consistent with the technological and schedule risks.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Current Cost, Schedule, and Performance Risk Methodologies Within the Army

  • Chapter Three

    Theoretical Framework for Risk-Informed Trade-Space Analysis

  • Chapter Four

    Proposed Conceptual Model of Risk-Informed Trade-Space Analysis in the Acquisition of Weapon Systems

  • Chapter Five

    The Model

  • Chapter Six

    An Example Application of the Risk-Informed Trade Analysis Model Using a Demonstration Tool

  • Chapter Seven

    Conclusions

  • Appendix A

    User Manual for the Risk-Informed Trade Analysis Model Demonstration Tool, Version 1.0

  • Appendix B

    Outline of Specific Assumptions and Methods in the Risk- Informed Trade Analysis Model

  • Appendix C

    Expert Elicitation and Minimizing Bias from Heuristics

  • Appendix D

    Computer Code

Research conducted by

This research was sponsored by Randolph Wheeler, AMSAAWR, and conducted within the Force Development and Technology program within the RAND Arroyo Center.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.