Implementing risk management principles for large defense acquisition programs is a priority for the U.S. defense acquisition community. To assist decisionmakers responsible for identifying major weapons programs' risks, RAND developed a methodology, an Excel risk tool (the "Assessor Tool"), and this users' manual. The tool offers an approach to the evaluation of system integration risk for defense staff assessors.

An Excel Tool to Assess Acquisition Program Risk

Lauren A. Mayer, Mark V. Arena, Michael McMahon

ToolPublished Oct 31, 2013

Implementing risk management principles to manage large defense acquisition programs is a priority for the U.S. defense acquisition community. To assist those decisionmakers responsible for identifying the risk associated with major weapons programs, RAND researchers developed a methodology and accompanying Excel, information-based risk tool (the "Assessor Tool"). A description of the methodology and the tool are available in a companion document, A Risk Assessment Methodology and Excel Tool for Acquisition Programs (by Lauren A. Fleishman-Mayer, Mark V. Arena, and Michael E. McMahon, RR-262-OSD, 2013). The present document is the users' manual for the Assessor Tool. The Assessor Tool offers an Office of the Secretary of Defense (OSD)-level approach to the evaluation and measurement of system integration risk. That is, it is meant for assessors, such as OSD personnel, who may not be especially familiar with the specific program under evaluation but still may need to make judgments about the program's risk. It is based on a tractable and comprehensive set of questions that can help evaluate integration risk at each point in the acquisition process. More specifically, the tool enables users to see how well integration risk is being managed by providing a standards-based valuation of integration issues that can lead to cost growth, schedule growth, and program performance. The Assessor Tool and its methodology may also be generalizable to an entire set of information-based risk assessment applications. Overall, the methodology and tool have many strengths, including being based on well-grounded theories, allowing for reproducibility and traceability, and the extensive flexibility to be used to evaluate risk for many different types of programs. To provide a benchmarking and validation of the risk scores calculated by the tool, future work could include the tool's validation by tracking its output against a program's performance.

Key Findings

The Assessor Tool and Its Information-Based Risk Assessment Methodology Offer an Office of the Secretary of Defense (OSD)-Level Valuation of Program Risk

  • The Assessor Tool is designed for those staff involved more generally with weapon systems acquisition who need access to a systematic method of determining a program's ability to meet its goals and manage risks, and to provide one basis to report on the success of the department's compliance with system integration risk management as directed by the Weapon Systems Acquisition Reform Act.
  • The risk valuation is based on established documentation that allows for a level of traceability, verifiability, and objectiveness.

The Assessor Tool Design Is Based Upon OSD-Level Assessment Questions and Utility Theory

  • The methodology assumes that a set of knowledge-based standards has been developed against which to measure program risk and that a risky outcome may result if a question is not satisfied.
  • The knowledge-based standards are based on the existence and completeness of Department of Defense artifacts and checklists that would be readily available to an assessor at the OSD level.

The Assessor Tool Can Be Adapted for Additional Risk-Related Assessments

  • The tool and methodology are also generalizable to an entire set of information-based risk assessment applications.
  • The tool could easily be tailored to insert user-determined review elements specific to previously identified technical or integration risk issues.
  • Using the blank template of the tool, a user could adapt it to evaluate risk related to many other types of programs with documented knowledge-based standards in place.

Recommendations

  • The reproducible and documented tool for integration risk assessment may be considered for program office reporting to meet Weapon Systems Acquisition Reform Act compliance as well as for other acquisition reviews and for adaptation into other program assessment tools.
  • To provide a benchmarking and validation of hte risk scores calculated by the tool, future work could include the tool's validation by tracking its output against a program's performance.

Topics

Document Details

  • Copyright: RAND Corporation
  • Availability: Web-Only
  • Year: 2013
  • Pages: 27
  • Document Number: TL-113-OSD

Citation

RAND Style Manual
Mayer, Lauren A., Mark V. Arena, and Michael McMahon, An Excel Tool to Assess Acquisition Program Risk, RAND Corporation, TL-113-OSD, 2013. As of September 20, 2024: https://www.rand.org/pubs/tools/TL113.html
Chicago Manual of Style
Mayer, Lauren A., Mark V. Arena, and Michael McMahon, An Excel Tool to Assess Acquisition Program Risk. Santa Monica, CA: RAND Corporation, 2013. https://www.rand.org/pubs/tools/TL113.html.
BibTeX RIS

Research conducted by

This research was conducted within the Acquisition and Technology Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.

This publication is part of the RAND tool series. RAND tools include models, databases, calculators, computer code, GIS mapping tools, practitioner guidelines, web applications, and various other toolkits and applied research products. All RAND tools undergo rigorous peer review to ensure both high data standards and appropriate methodology in keeping with RAND's commitment to quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.