Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief

Abstract

A new RAND project enhances the methods of portfolio analysis—an approach to allocating resources across multiple objectives while managing risk. The enhancements are particularly applicable to decisions made regarding the development of military capabilities. The modified approach recognizes the need to test system alternatives against a range of scenarios, to supplement narrow military evaluation criteria with socioeconomic and political criteria, and to allow easy decisionmaker input to assumptions and weighting factors. The enhanced approach should be particularly helpful in high-level reviews seeking to integrate requirement-setting, technology assessments, and cost analyses.

The Department of Defense (DoD) works continuously to improve its strategic planning for future forces, their capabilities, and their readiness. An important objective of this planning is confronting and managing diverse risks. Considerable RAND research has been devoted to these topics, including in-depth work on what has come to be called capabilities-based planning. In our view, good capabilities-based planning encourages a "FAR strategy," i.e., a strategy assuring capabilities that are flexible (suitable for different strategic or military purposes), adaptive (modifiable for different circumstances), and robust (resistant or resilient to negative events).

Capabilities-based planning is facilitated by portfolio analysis, the goal of which is to invest in a mix (a portfolio) of capabilities that, while perhaps not the best for any single objective, is effective for as broad a range of possibilities as is feasible under the budget. Such a portfolio is consistent with a FAR strategy.

Contributions of the New Research

The specific emphasis of RAND's latest research on portfolio-analysis methods is on helping inform senior DoD decisionmakers regarding acquisition of capabilities such as conventional global strike, ballistic-missile defense, and long-endurance surveillance, with choices made within a budget that may be flexible but that is always limited. The work was stimulated by senior-leader requests to RAND for an analytic framework that would systematize how analysis is presented to them and how they are able to interact.

A new monograph describes and illustrates the analytic framework. Superficially, the process suggested is familiar: Define a capability area and a set of test cases, define concepts of operations, generate and evaluate options, characterize shortfalls, iterate to improve both options and appreciation of objectives, evaluate again, and inform decisions. The specifics, however, represent a significant advance:

Evaluations Should Be Subject to Challenge by Drilling Down to Lower Levels and Changing Assumptions

NOTE: Colors indicate effectiveness or some other measure, running from red (poor) through orange and yellow to green (excellent).

  1. The Right Test Cases. The test cases of the approach described in this study are chosen analytically to constitute a "spanning set." If an option tests well against all of the options in this test set, it will quite likely be appropriate for real-world crises, even though they are different from test-set cases. The key is having a test set that stresses the options in all of the critical dimensions, making it possible quickly to see what a given option can and cannot do well. Historically, DoD's defense planning scenarios have not had this goal.
  2. Diverse Evaluation Factors. A broad set of evaluation factors is used. For example, in evaluating options for prompt global strike, it is not sufficient to worry only about the likelihood of destroying targets on a timely basis. Options should also be evaluated for side effects and for their strategic, operational, and developmental risks.
  3. Effective Communication with Senior Leaders. In this approach, top-level summaries use familiar stoplight charts, color-coded to indicate the relative goodness of an option by a given criterion. These are cognitively very effective, but unsatisfying to discerning decisionmakers unless the basis of the scorecard evaluations is clear and unless key assumptions can be readily identified and challenged. A tool that accompanies the enhanced approach allows senior leaders reviewing analyses to "drill down"—to ask, for example, "Why does Option B get only an orange rating? It was supposed to be great." The drill-down brings up a second-level scorecard showing the factors contributing to the top-level evaluation (see the figure). Option B is shown to get a red (poor) rating on test case 1b. The decisionmaker, upon understanding the issue, may ponder and conclude, "We need to be more realistic about that case; let's reduce the objective to a more reasonable level." Further drill-downs are also possible and sometimes very useful.

    Some such challenges will reveal that results depend upon assumptions or judgments that are inherently in the realm of decisionmakers, such as degree of concern about certain strategic or operational risks. Senior reviewers should be able to see the effect of changing the assumptions in real time. The result is then true communication, not just reporting of nominal results.

  4. Highlighting Important Alternative Perspectives. The relative merit of options will typically depend critically upon "perspective," such as the relative importance of particular missions, scenario classes, or risks. These are, again, in the province of senior leaders. In the new approach, analysis shows how results vary with the more important perspectives. This material should be front and center, not relegated to backup status.
  5. Assuring Rigor of Staff Work. Structuring analysis and its presentation to permit such interactive drill-down and iteration is important for assuring rigor and quality of staff work. If senior leaders require such reporting and do enough spot checking with drill-downs to reinforce the point in some meetings, analysis processes will respond accordingly, and the quality of analysis and its presentation will improve.

Applications and Next Steps

Although the analytical methods we have developed are applicable across a range of organizational structures, we see them as particularly useful in the new DoD concept decision reviews. These high-level reviews are intended to get beyond the tendency of acquisition processes to separate requirement-setting, assessment of technology options, and economic considerations. The enhanced approach described in this study is well suited to an early integrative process that considers all three factors at once. The options considered can thus be better informed technically, the requirements can be established with a full appreciation of what can be accomplished affordably, and programming and budgeting can allow for long-lead-time activities such as making room for a new program.

Finally, the study indicates future directions for this line of analysis. Among the most important is developing the capability for analysis across capability areas. This is necessary for big-ticket items such as bombers and space-based radars that contribute to multiple capability areas. Another possibility is the application of portfolio analysis to higher-level strategic decisionmaking in DoD. For example, in deciding whether to recapitalize, modernize, or restructure for low-intensity conflict, force planners will have to balance concerns about regional instabilities with those about the long-term competition in East Asia.

This report is part of the RAND Corporation research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.