The National Aeronautics and Space Administration (NASA) confronts a variety of organizational-level risks within its programs that could influence the success of its missions or programs. This report provides NASA's Office of Strategy and Plans with a risk assessment methodology that integrates risk factors tailored to NASA's management, operations, and acquisition structures, allowing decisionmakers to compare risks within a common framework.
Developing a Risk Assessment Methodology for the National Aeronautics and Space Administration
Download
Download eBook for Free
| Format | File Size | Notes |
|---|---|---|
| PDF file | 2.3 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Download Support Files
Appendix B and C
Microsoft Excel spreadsheets
| Format | File Size | Notes |
|---|---|---|
| zip file | 0.1 MB | The file(s) provided above are ZIP-formatted archives, which most modern systems can natively unpack. If your computer does not unpack the archive when you double-click it, you may need to use a separate decompression program such as UnZip. |
Purchase
Purchase Print Copy
| Format | List Price | Price | |
|---|---|---|---|
| Add to Cart | Paperback112 pages | $40.00 | $32.00 20% Web Discount |
Research Questions
- What is risk? What are existing methods to assess it?
- What are risk factors and components that NASA must consider?
- What should be included in one overarching, risk-informed decision support methodology for NASA?
The National Aeronautics and Space Administration (NASA) confronts a variety of organizational-level risks within its programs that could influence the success of its missions or programs. Comparing, contrasting, and mitigating these risks require developing a common lens through which to view them. Such an evaluation can increase overall understanding of the risks associated with NASA-level decisions. This report provides NASA's Office of Strategy and Plans with a risk assessment methodology that integrates risk factors and risk management approaches tailored to NASA's management, operations, and acquisition structures. While NASA has deep experience in conducting risk assessments on highly technical issues for individual programs and projects, it has not developed an overarching methodology for thinking about overall NASA-level risk associated with its decisions. For this effort, a methodology was developed for normalizing and comparing supply chain, external dependency, cost and schedule, human capital, organizational and managerial, political, and technical risks. The goal ultimately was to develop a single, overarching, risk-informed decision support methodology for looking at disparate risks through a common lens. The methodology presented is for a non-expert audience so that any practitioner or decisionmaker with any level of training can use it.
Key Findings
Seven Key Risk Factors Examined
- The risk factors considered were: supply chains, dependency on external sources, acquisition costs and schedules, workforce issues, organizational and managerial considerations (e.g., the issues arising from NASA's distributed management between mission directorates and research centers), domestic and international political situations (e.g., partnership instabilities, congressional equities), and technical issues including information security.
Structured Methodology Provides Benefits Beyond Final Calculations
- Considering not only risks but also mitigations at an early stage in the analysis can prompt a structured thinking process that allows for clearer and more-thoughtful decisions.
- The use of a normalization process allows comparison of disparate risk issues that contribute to important decisions. Without such normalization, decisionmakers would be presented with a number of risk factors using different scales (and likely different grading and weighting) with little ability to understand comparisons within or among them or the overall risk burden associated with a decision.
- The result of such an analysis is a graphical representation of normalized relative risks designed to allow for understanding the NASA organizational-level risk associated with its decisions.
Case Studies Validate Suggested Methodology
- Each discrete risk assessment requires that the methodology be changed to reflect the particulars of the issue under consideration.
- Two decisions — one past and one future — were considered as a way to test and partially validate the methodology. The previous decision considered was the cancellation of the Space Shuttle program and commercialization of the transportation to the International Space Station, while the future decision focused on Cislunar habitation.
Recommendations
- Each discrete assessment requires that the methodology be changed to reflect the particulars of the issue under consideration.
- Extensive resources may be required to develop the inputs for this methodology but once inputs are in hand, the significant remaining task is to elicit weights from decisionmakers. Data for each risk factor may be obtained from different experts in each area. Therefore, while resource-intensive, much of the work may occur concurrently.
- Separating the experts from the decisionmakers is imperative for assuring the methodology produces unbiased results. Experts are essential in developing the risk factor component charts; decisionmakers are essential in developing weights. Decisionmakers should approach the methodology without being encumbered by the discussions associated with completing the chart and with the opportunity to synthesize the material and think at a higher level of abstraction about the results. That is, there should be an independent role for the decisionmaker, reflecting a broader pattern of thinking on a risk-informed decision issue.
- The steps developed in the methodology provide a structured way to consider a risk-informed decision. While we were able to conduct abbreviated case studies validating it, conducting a more robust analysis for a future NASA decision is important. Given the work we have begun on it, applying this model to a more expanded analysis of the pending Cislunar habitation decision may be most appropriate. Such work would provide further validation of the methodology and additional insights into the risk (and options) associated with this decision.
Table of Contents
Chapter One
Introduction
Chapter Two
Risk Assessment and Risk-Informed Decisionmaking Methods
Chapter Three
Definition and Discussion of the Risk Factors and Components
Chapter Four
Development of the Risk-Informed Decision Methodology
Chapter Five
Case Studies
Chapter Six
Findings and Conclusions
Appendix A
List of Potential Cases for Consideration
Appendix B
Cancellation of the Space Shuttle Program and Commercialization of Transport to the ISS Case Study
Appendix C
Cislunar Habitat Case Study
Appendix D
Human Capital Risk Methodology
The research reported here was prepared for the National Aeronautics and Space Administration and conducted by the Science, Technology and Policy Program within RAND Justice, Infrastructure, and Environment.
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.