The Utility of Modeling and Analysis in the Iraq and Afghanistan Wars

Modeling, simulation, and operations analysis, as well as systems analysis, have long been used to support decisionmaking at all levels. In Iraq and Afghanistan, modelers and analysts provided both in-theater and homeland support ("reachback") to commanders and staffs making decisions across the spectrum of counterinsurgency and irregular warfare (COIN and IW) operations. The Office of the Secretary of Defense asked RAND to review the usage and effectiveness of analysis and modeling in these recent wars. RAND researchers examined the decision support literature and interviewed commanders and analysts. The RAND team found that modeling and analysis have supported four categories of activity:

1. Force protection, which encompasses efforts to reduce casualties and damage to friendly forces — including armor improvements and tactics.
2. Logistics, support for which ranged from simple tactical calculations to theater-level modeling and which focused on such issues as the movement of supplies, aerial transport deployments, and the location of specialty surgical teams.
3. Campaign assessment — that is, the commander's effort to determine progress against mission objectives to optimize planning and resource allocation. Assessments support decisions such as how to allocate forces and when to change strategy.
4. Force structuring, which resolves such issues as how many and what kinds of troops are needed and how to use them.

How effective were modeling and operations analysis at supporting these functions?

The way commanders in the field reacted to their support varied with the support's purpose. Some commanders were enthusiastic as they described the contributions of modeling and analysis to their efforts. Most stated that they particularly benefited from decision support for activities countering improvised explosive devices. Whether developed for that purpose or modified from previous work, modeling and other analytic techniques saved countless lives, preserved millions of dollars in material, and increased the freedom of movement of friendly forces. Logistics modeling and analysis also saved money and directly reduced threat of injury or death to coalition military personnel.

Commanders and analysts expressed far less enthusiasm for the results achieved in support of campaign assessment and strategic force structuring. These opinions were also reflected in the relevant literature, which was sparse, conflicting, and generally inconclusive. Those interviewed by the RAND team reported frustration due to poorly defined problems, inadequate data, and a lack of common, validated methods. Commanders, for example, wanted to know if they were winning — if a particular decision would get them closer to their objective — and that kind of knowledge has proved to be beyond what quantitative modeling can provide in the COIN/IW environment.

As to force-structuring, commanders are responsible for providing policymakers with a clear rationale for their requirements, but a methodology providing such a rationale is lacking in COIN and IW. None of the approaches tried to date have been considered generally sound and effective by policymakers, commanders, or the analytic community. This is partly a problem of applying complex models where the data are not sufficient to support them. The level of analysis should be chosen to match the objectives in the field that are under consideration.

A number of other key findings emerged from the RAND research

• It is not clear that mathematical methods from operations research or systems analysis are applicable to all COIN and IW problems. Analysts have had great success with quantifiable problems — less so in applying the same approach to nonlinear, complex COIN and IW problems that do not lend themselves to quantification. In some cases, other analysts (e.g., sociocultural and intelligence analysts) have compensated for this shortfall, but the Department of Defense (DoD) has not addressed this issue in depth.
• Many commanders and analysts praised the role of reachback support for COIN and IW. However, this support is bounded by the relative lack of situational awareness in remote analytic shops and the general inability of reachback analysts to support decisions within a 72-hour window.
• Some commanders are not well prepared to use analysts or analyses. Many only partially understand the capabilities, limitations, and demands of analysis. This hindered analysis in many cases, but it also gave some of the more entrepreneurial analysts the opportunity to do some excellent initiative-based work.
• Unfortunately, the ability of analysts to take initiative was often constrained by the need to generate recurring reports. Once an analytic report was generated and deemed valuable, it was often demanded on a recurring basis, regardless of its recurring value. Recurring report generation is time-consuming and can reduce analysts' ability to provide relevant, specifically targeted support.
• Data in Iraq and Afghanistan were generally poor in quality — incomplete, inaccurate, and inconsistent. This stems in great part from the fact that in nearly all cases, data are reported to support operations, not analysis. Also, there was no clear way for analysts to determine the degree to which aggregated datasets were complete or accurate. Data-quality issues were sometimes manageable at the tactical level, but rarely at the strategic level.
• Simulation has helped analysts think about the challenges associated with complex and seemingly intransigent problems such as operational force structuring, and it has helped prepare commanders and staffs for deployment. Tactical simulation has been useful for training, and also in support of real-world problems like logistics and force protection. However, the complexity of the IW environment and the lack of good, consistent data preclude the use of simulation as a real-world, real-time decision support tool at the strategic level.

What do the RAND findings imply for further development of models and other analytic techniques for IW and COIN?

Should DoD weight its investment toward those purposes for which models have already proven useful? Should it attempt to focus investment toward capability gaps? Or should it spread its assets to achieve some kind of parity? The RAND team concluded as follows: Given that campaign assessment and force-structuring are the least amenable to the kinds of quantitative models and tools typically associated with modeling and simulation and operational analysis, further investment in structured techniques of similar purpose and type — without a reconsideration of assessment and force-structuring theory — is putting good money after bad. Further such investments should be selective until issues with theory, methods, and data are resolved. Each investment in modeling, simulation, and analysis should be predicated on the understanding that the COIN and IW mission and environment places restraints on applicability of many commonly used methods, particularly on those involving quantitative analysis.

Other recommendations for different players

DoD should consider supplementing the operations researchers and systems analysts on its decision support teams with social scientists, intelligence analysts, civilian advisers, and planning experts.

Modelers and analysts should explicitly identify for commanders and other consumers the limits of various approaches, methods, and tools. That will help users understand where and how modeling, simulation, and analysis can be helpful, and where the complexity of a problem begins to impose limits on efficacy.

Commanders should make every effort to provide analysts with clear articulations of their key decisions. Analysts were more effective when they had a good understanding of the commander's needs. Commanders should insist on objectivity in their assessments and should be aware that recommended decisions resulting from an analytic effort will be subject to caveats and assumptions that they should not ignore.

Topics

• Afghanistan
• Asymmetric Warfare
• Cyber and Data Sciences
• Data Science
• Decisionmaking
• Intelligence Analysis
• Iraq
• Military Force Deployment
• Modeling and Simulation
• Peacekeeping and Stability Operations