Jan 23, 2008
A new RAND project enhances the methods of portfolio analysis—an approach to allocating resources across multiple objectives while managing risk. The enhancements are particularly applicable to decisions made regarding the development of military capabilities. The modified approach recognizes the need to test system alternatives against a range of scenarios, to supplement narrow military evaluation criteria with socioeconomic and political criteria, and to allow easy decisionmaker input to assumptions and weighting factors. The enhanced approach should be particularly helpful in high-level reviews seeking to integrate requirement-setting, technology assessments, and cost analyses.
The Department of Defense (DoD) works continuously to improve its strategic planning for future forces, their capabilities, and their readiness. An important objective of this planning is confronting and managing diverse risks. Considerable RAND research has been devoted to these topics, including in-depth work on what has come to be called capabilities-based planning. In our view, good capabilities-based planning encourages a "FAR strategy," i.e., a strategy assuring capabilities that are flexible (suitable for different strategic or military purposes), adaptive (modifiable for different circumstances), and robust (resistant or resilient to negative events).
Capabilities-based planning is facilitated by portfolio analysis, the goal of which is to invest in a mix (a portfolio) of capabilities that, while perhaps not the best for any single objective, is effective for as broad a range of possibilities as is feasible under the budget. Such a portfolio is consistent with a FAR strategy.
The specific emphasis of RAND's latest research on portfolio-analysis methods is on helping inform senior DoD decisionmakers regarding acquisition of capabilities such as conventional global strike, ballistic-missile defense, and long-endurance surveillance, with choices made within a budget that may be flexible but that is always limited. The work was stimulated by senior-leader requests to RAND for an analytic framework that would systematize how analysis is presented to them and how they are able to interact.
A new monograph describes and illustrates the analytic framework. Superficially, the process suggested is familiar: Define a capability area and a set of test cases, define concepts of operations, generate and evaluate options, characterize shortfalls, iterate to improve both options and appreciation of objectives, evaluate again, and inform decisions. The specifics, however, represent a significant advance:
Effective Communication with Senior Leaders. In this approach, top-level summaries use familiar stoplight charts, color-coded to indicate the relative goodness of an option by a given criterion. These are cognitively very effective, but unsatisfying to discerning decisionmakers unless the basis of the scorecard evaluations is clear and unless key assumptions can be readily identified and challenged. A tool that accompanies the enhanced approach allows senior leaders reviewing analyses to "drill down"—to ask, for example, "Why does Option B get only an orange rating? It was supposed to be great." The drill-down brings up a second-level scorecard showing the factors contributing to the top-level evaluation (see the figure). Option B is shown to get a red (poor) rating on test case 1b. The decisionmaker, upon understanding the issue, may ponder and conclude, "We need to be more realistic about that case; let's reduce the objective to a more reasonable level." Further drill-downs are also possible and sometimes very useful.
Some such challenges will reveal that results depend upon assumptions or judgments that are inherently in the realm of decisionmakers, such as degree of concern about certain strategic or operational risks. Senior reviewers should be able to see the effect of changing the assumptions in real time. The result is then true communication, not just reporting of nominal results.
Although the analytical methods we have developed are applicable across a range of organizational structures, we see them as particularly useful in the new DoD concept decision reviews. These high-level reviews are intended to get beyond the tendency of acquisition processes to separate requirement-setting, assessment of technology options, and economic considerations. The enhanced approach described in this study is well suited to an early integrative process that considers all three factors at once. The options considered can thus be better informed technically, the requirements can be established with a full appreciation of what can be accomplished affordably, and programming and budgeting can allow for long-lead-time activities such as making room for a new program.
Finally, the study indicates future directions for this line of analysis. Among the most important is developing the capability for analysis across capability areas. This is necessary for big-ticket items such as bombers and space-based radars that contribute to multiple capability areas. Another possibility is the application of portfolio analysis to higher-level strategic decisionmaking in DoD. For example, in deciding whether to recapitalize, modernize, or restructure for low-intensity conflict, force planners will have to balance concerns about regional instabilities with those about the long-term competition in East Asia.