Paul K. Davis
RAND Graduate School

Santa Monica, CA

pdavis@rand.org

In this paper I describe certain core problems of defense planning (Section 2), which have much in common with more general planning problems that arise in dealing with complex adaptive systems (CAS) characterized by unpredictability. I then describe concepts and methods that my colleagues and I have brought to bear with not-inconsiderable success. The concepts relate to managing risk and uncertainty (Section 3). These involve portfolio management and an emphasis on building blocks and at-the-time assembly of building blocks. The methods use measures of effectiveness focused on achieving flexibility, robustness, and adaptiveness. They involve exploratory analysis in large scenario spaces, and multi resolution modeling to facilitate such analysis. Such models often need adaptive agents. Human gaming, as part of a family-of-models approach, can also be helpful and even essential.

Ultimately, planning is about control: we seek, by decisions that we make today, to improve future circumstances. The enemy of planning, of course, is uncertainty. Although this fact is sometimes suppressed, strategic planners are often faced with massive and ubiquitous uncertainty in many dimensions. Moreover, many of the uncertainties involve people and organizations, some of them in competition. That is, the relevant systems are not only complex, but also adaptive.

To make things worse, when planners consider alternative strategies, they typically attempt to do so by predicting the consequences in future system behavior of their choices. However, they often discover–if they have the courage and integrity to address the issue–that they are unable to make meaningful predictions: future behavior of their complex adaptive systems is sensitive to a myriad of uncertainties. Sometimes, this relates to the celebrated sensitivity to initial conditions, but it may instead be due to sensitivity to subsequent random events (what Gell Mann has called "life’s frozen accidents"), or to various nonlinearities. Or it may be due to the sheer magnitude of the uncertainties: outcomes are uncertain because uncertainties are large. In any case, predictiveness is sometimes quite poor, which creates a dilemma. After all, decisions must be made, even if uncertainties are high.

In today's defense planning, the uncertainty issue arises at the highest level as people argue about whether China will become a troublemaking regional power, whether Russia will revert to a more nationalist course, and whether Saddam Hussein will try again to invade and conquer the Persian Gulf. Others argue about whether U.S. forces will continue to be engaged in numerous smaller-scale contingencies across the entire globe, with their many effects on personnel readiness, morale, and ability to retain personnel. Still others argue about what the future will bring with respect to weapons of mass destruction, attacks against the U.S. homeland, or different types of conflict such as wars against non-state terrorist groups or drug lords.

At the next level of planning, the uncertainties are more technical, but equally large and troublesome. Will potentially hostile nations develop and deploy enough advanced air defenses so that many of the horribly expensive F-22 stealthy fighter aircraft will be needed? Can we count on future conflicts providing significant strategic and tactical warning, or must we invest to reduce our dependence on such warning? Will American aircraft carriers still be able to sail into troubled waters as part of an effort to stem crises, or will they be vulnerable to long-range missiles, in which case the U.S. might need to invest in a new generation of submerged platforms.

We should remember the abysmal past record of forecasts. Few predicted the fall of the Soviet Union, the reunification of Germany, Saddam Hussein's invasion of Kuwait in 1990, or American involvement in wars in both Bosnia and Kosovo. Suffice it to say, higher level military planning is beset with problems of uncertainty. These uncertainties are not minor annoyances to be dealt with by technicians, but matters of the highest significance. For discussion of international affairs, defense planning, and war itself as examples of complex adaptive systems, see Alberts and Czerwinski (1997).

The challenge of planning under uncertainty is hardly new (see particularly Morgan and Henrion, 1992). Some of the most important generic mechanisms for coping with uncertainty are (Davis and Hillestad, forthcoming):

The principal point here is that even massive and ubiquitous uncertainty need not be paralyzing. Indeed, if this were not so, we might not none of us arise from our bed in the morning.

Nonetheless, it is common for the strategic-planning groups of large organizations to suppress uncertainty and become "forecasters." The result is plans that are essentially "optimized" against the particular image of the future represented by those forecasts, which are usually simple projections in disguise. In business, such planning may lead to rigid long-term investment in facilities and capital equipment, extreme centralization to achieve advantages of scale in production, and an emphasis on perfecting one's product rather than R&D to develop successor products of a different character. Such forms of business planning have been discredited over the last decade or two–so much so that most businesses have deleted or drastically cut back on strategic planning of the bureaucratic, forecasting and optimization-oriented variety (Mintzberg, 1993).

Defense planning has not had the advantage of a fierce market competition. As a result, some of the dysfunctional aspects of strategic planning persist in the Pentagon. In particular, the DoD’s Planning, Programming, and Budgeting System (PPBS) is massive, ponderous, and supported by studies of the sort deplored here. Nonetheless, much has been accomplished in recent years to reform defense planning. In what follows I shall discuss efforts to help the department plan for adaptiveness, which relate primarily to the last three of the above bullets.

There are at least three key elements in an effort to achieve adaptiveness in defense planning. These are (Davis, Gompert, and Kugler, 1996):

Over the decades between 1961 and 1990, the principal challenge of defense planning was seen by many as sizing the force: How many divisions, carrier battle groups, and tactical fighter wings should the U.S. have? How many ICBMs? And so on. A fairly rich theory was developed to assist in such work, one that recognized uncertainty, but argued from the premise that the United States should buy forces for bounding Soviet threats because, if it did, it would not only be prudently prepared for those threats, but would also have enough general capability to deal with lesser-included cases (Davis, 1994).

Planning exclusively around an image of one or two big wars has made little sense since the end of the Cold War. The disconnect between that image and what U.S. forces must actually do is simply too large. In thinking afresh about a framework for defense planning, my colleagues and I have emphasized the portfolio-management approach and elevating to the top level of visibility those DoD activities that deserve to be there. Figure 1 shows a top-level view of what we recommended.

Anyone who thinks about adaptive capabilities soon learns about the centrality of building-block approaches in which–instead of designing in exquisite detail for a particular requirement–one instead develops the building blocks that allow dealing with diverse challenges as they arise. This is related, of course, to using the modules of "nearly hierarchical decomposition" (Simon, 1996; Holland, 1995).

That systems are nearly hierarchically decomposable should be nothing new to those interested in complex adaptive systems, but it is difficult for planners to know whether they have the "right" building blocks or should create new ones. Space does not allow elaborating here, but suffice it to say that a major issue for the Department of Defense is rethinking all of its building blocks to "transform the force" for the decades ahead (Davis, et al., 1998). For example, it is by no means evident that the centuries-old concept of the Army division is still appropriate. And, with only 10 active divisions, the U.S. Army has much less flexibility than it needs. This becomes apparent when one realizes that it takes three divisions to have one that is ready to do something (a second division is typically recovering from a period of assignment, and the third is training). In any case, such issues are fodder for other forums.

One of the most important aspects of adaptive planning is changing the measures of effectiveness used to evaluate alternative courses of action. In the Department of Defense, the traditional measures used for strategic planning have been outcomes of simulated future wars. That is logical enough, but what has been much more problematic is the DoD’s use of what are called defense planning guidance scenarios. Figure 2 shows schematically what such a scenario might look like:

Figure 2–Schematic of an Illustrative Planning Scenario

There is nothing wrong with using such a scenario test. The problem has been that the defense planning guidance scenarios have been treated as necessary and sufficient "requirements," or "specifications" by the many thousands of officers and contractors who receive them in guidance. That is, the tendency has often been to plan forces and programs to do well in the official test scenarios without much regard for other measures of capability.

Clearly, if one wants military forces that are superbly adaptive, a good deal more effort must go into examining the consequences of uncertainty. An effective way to do that is by assessing forces in a scenario space–so named to convey the sense of considering an enormous range of scenarios that place different demands on the forces. The measure of an alternative force’s goodness then becomes how much of the credible region of scenario space can be handled adequately. In this framework, it is far better to add an innovation that provides capability for a credible scenario that was previously unworkable, than to add an innovation that provides for a marginally better outcome in any one scenario, even if standard (Davis, 1994; Davis, et al., 1996).

Figure 3 illustrates the notion of expanding analysis from point scenarios, as in Figure 2, to a scenario space. Figure 4 (Gritton, Davis, et al., 2000) shows one depiction of capabilities in a scenario space. Capabilities are good in the green region, poor in the red region. The intention should be to invest and otherwise proceed so as to turn as much of the red region green as possible.

Figure 3–Assessments in a Scenario Space

This approach puts a great deal of weight on what we call exploratory analysis (closely related to what colleagues call exploratory modeling (Bankes, 1993). The emphasis is on assessments that are very broad in scope, rather than detailed but narrow (National Research Council, 1997). This type of assessment is especially suitable for strategic planning under uncertainty and is to be recommended much more generally in policy planning. It is by no means unique to defense.

various forces and weapons; what is the physical environment, such as weather; and what other model assumptions are being made, some of them deep within the algorithms of the model itself?). These abstractions can be translated into highly concrete variables such as the size assumed for a future enemy’s army. We have used this structured approach to exploratory analysis in a number of studies. The result has been to look at issues, and obtain insights about them, that would simply have been overlooked in more conventional efforts. The result has been to highlight issues of adaptiveness.

Figure 4–Measuring Outcomes in a Scenario Space

Although this approach may appear straightforward, it is radical within a planning system familiar with stereotyped analyses that suppress major types of uncomfortable uncertainty.

The preceding sections have emphasized the central role of uncertainty in strategic planninge. It is perhaps rather obvious that this characteristic of strategic planning has much in common with the study of many complex adaptive systems–not just in defense, or even in planning problems, but much more generally. Potentially unpredictable behavior is almost a defining characteristic of complex-adaptive-system research.

Exploratory analysis methods are also applicable to a wide range of CAS problems. After all, we don’t want to forgo the hope of controlling our environment–at least to some extent–merely because there are CAS phenomena at work. We must, however, understand better what can and cannot be understood, controlled, or modulated. For example, we may wish to identify regions in which system behavior is unacceptably uncertain precisely so that we can avoid operating in those regions.

If exploratory analysis is desirable, what implications does this have for modeling, analysis, and related technology? I have written elsewhere on this subject, but a few observations are appropriate here:

This last item might seem obvious to CAS aficionados, but it is not obvious to many planners, who may believe it intuitive that, in comparing Option A and Option B, one should hold everything else constant. That is, one should have a model and a scenario, and one should compare results for Option A and Option B making no other changes of assumption. The problem, of course, is that if the real system of interest is adaptive because it includes humans such as violently competitive military commanders, then if one side gets a new capability or changes its strategy or tactics in some particular way, then the other side will change its behavior.

Even if convinced of the need to include adaptation, workers are often stymied because they do not know how to represent behavioral factors in their algorithms. Some of the difficulties here are cultural: many modelers dislike dealing with "soft factors" like the vagaries of human behavior. But the soft factors, such as the opponent’s ability to learn and adapt, often dominate the problem!

Although my colleagues and I have done a good deal of agent-based modeling over the years in defense problems, which I shall touch upon below, an important conclusion before getting to that is that an enlightened analysis of planning problems in complex adaptive systems should be done with families of "models" with "model" construed to mean not just the garden-variety closed simulations that analysts like to build, but also interruptible simulations allowing humans to make key decisions and even people-intensive war games, with or without computer support. Figure 5 summarizes a family-of-models-and-games scheme recommended for DoD’s current work on "transforming the force" (Davis, Bigelow, and McEver, 1999). As the figure indicates, different kinds of models vary drastically in what they are good for. Relatively simple and abstract analytical models (e.g., ones that might be done in closed form or in models built on spreadsheets or Analytica) are superb for broad exploration, but quite poor for uncovering new phenomena. For that, one typically needs more detailed models, games, or both. On the other hand, detailed models and complex war games are typically too ponderous and complicated for exploration, and they can bury users in a mass of facts that obscure forest for trees.

Figure 5–The Relative Virtues of a Family of Models and Games in Military Analysis

By and large, DoD models and simulations are "scripted models" in which strategy and tactics are hard-wired in input data. As a result, the simulated entities are often not very adaptive. There have been exceptions, however, and much can be done. I shall only mention a few of the possibilities here.

As one might expect from the agent-based approaches, behaviors are sometimes "emergent," in the sense that sensible and striking aggregate-level behavior was not dictated by the model, but rather a consequence of events and lower-level interactions. Nonetheless,

This paper has shown that some of the core concepts of CAS research play dominant roles in strategic planning, as illustrated with examples from defense planning. It is of interest that some of the CAS concepts, such as that of chaotic systems and the sensitivity of dynamical systems to initial conditions, are perhaps less central to planning that larger issues of dealing with uncertainty. Outcome uncertainty stems not just from chaos-creating initial conditions, but also from sensitiity to subsequent events and the sheer magnitude of uncertainty in many key parameters and model features. If one wishes to plan under uncertainty, an excellent methodology for doing so is exploratory analysis, which examines outcomes across huge regions of "scenario space" rather than allegedly representative point cases. This is akin to testing against a highly diverse fitness landscape. The ability to accomplish such exploratory analysis, however, is facilitated by specially designed multiresolution, multiperspective models (MRMPM), agent-based methods for representing human and organizational behaviors and adaptations, and by modeling and analysis technology making it easier to search and view results across large outcome spaces.

Alberts, David S. and Thomas J. Czerwinski (ed.)(1997), Complexity, Global Politics, and National Security, Institute for National Security Studies, National Defense University, Washington, D.C.

Bankes, Stephen C., "Exploratory Modeling for Policy Analysis," Operations Research, Vol. 41, No. 3, 1993.

Cohen, William (1997), Report of the Quadrennial Defense Review, Department of Defense, Washington, D.C.

Davis, Paul K. and Bigelow, James A. (1998), Experiments in Multiresolution Modeling, RAND, Santa Monica, CA.

Davis, Paul K. (ed.), New Challenges in Defense Planning: Rethinking How Much Is Enough, RAND, Santa Monica, CA, 1994

Davis, Paul K. and Hillestad, Richard (2000), Exploratory Analysis for Strategy Problems With Massive Uncertainty, RAND, Santa Monica, CA, draft book, forthcoming in 2000.

Davis, Paul K., Bigelow, James A., and McEver, Jimmie (1999), Analytic Methods for Studies and Experiments on Transforming the Force, RAND, Santa Monica, CA.

Davis, Paul K., Gompert, David, and Kugler, Richard (1996), Adaptiveness in National Defense: the Basis of a New Framework, RAND Issue Paper, Santa Monica, CA.

Davis, Paul K., Gompert, David, Hillestad, Richard, and Johnson,Stuart (1998) Transforming the Force: Suggestions for DoD Strategy, RAND Issue Paper, Santa Monica, CA, 1998.

Holland, John (1995), Hidden Order, Addison Wesley, Reading, MA.

Ilachinski, Andrew (1996), Land Warfare and Complexity, Center for Naval Analyses, Arlington, VA.

Lempert, Robert, Schlesinger, Michael E., and Bankes, Steven C., "When We Don’t Know the Costs or the Benefits: Adaptive Strategies for Abating Climate Change," Climatic Change, Vol. 33, No. 2, 1996.

Mintzberg, Henry (1993), The Rise and Fall of Strategic Planning:, The Free Press, NY.

Morgan, Granger and Henrion, Max Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press, Cambridge, Mass., 1992. Reprinted in 1998.

National Research Council, Modeling and Simulation, Volume 9 of Technology for the United States Navy and Marine Corps, 2000—2035, National Academy Press, Washington, D.C., 1997.

Simon, Herbert (1996), Sciences of the Artificial, MIT Press, 3d edition, Cambridge, MA.