Acquiring Simulation Training

For decades, the Department of Defense (DoD) has used simulations to train uniformed and civilian defense personnel.^[1] By using computer hardware and software to portray and model a variety of tasks and scenarios, the DoD believes that—for many skills—it can train personnel more quickly and at less cost and risk than it can by conducting live training.

Beginning in 2002, the DoD undertook a two-year analytical process to identify ways to improve how it obtains simulations and simulation training support. At the conclusion of this process, called the Training Capabilities Analysis of Alternatives, the DoD decided to pursue a prototype of one alternative that it had identified. This alternative, known as Alternative #4 (Alt#4), proposes that the DoD abandon its traditional practice of acquiring and owning training simulators, software, and teaching materials in favor of contracting for training services from private sector vendors.

A recent RAND study evaluated Alt#4, compared it to other approaches to buying simulations and simulation training support, reviewed relevant economic theories, and provided detailed implementation and evaluation plans for a prototype. The study assembled case summaries of relevant programs and acquisition approaches based on open-source literature reviews and on interviews of industry and DoD personnel familiar with simulation acquisition issues. It used these case studies and economic theories to examine and validate Alt#4's underlying logic and core principles in light of other methods used to acquire simulation support and services.

Alt#4: Breaking from the Traditional Approach to Acquiring Simulation

Traditionally, the DoD has purchased simulation tools directly from developers with "cost-plus" contracts and often has paid those same developers to run training exercises using these tools.^[2] This has meant that the DoD has ended up owning these simulation tools, frequently at great cost. Some observers have characterized this approach as being both fiscally wasteful and a hindrance to innovation because it ties the DoD to the specific tools and simulations that it has acquired, often for lengthy periods of time.

Alt#4 proposes that the DoD avoid these problems by buying only training support, not simulation tools and training support, and by using firm-fixed-price (FFP) contracts tied to training outcomes rather than cost-plus contracts. The DoD no longer would buy simulation tools; instead, providers of training would buy or license tools and the DoD would contract with them for simulation training services. The DoD would get out of the ownership cycle. But although the main responsibility for funding the development of simulation tools would shift to the private sector under Alt#4, the DoD could contribute through seed money investments.

How Does Alt#4 Compare with Other Approaches to Buying Simulations and Simulation Training?

The study compared Alt#4 to other approaches that government entities use to acquire simulation tools and training. The accompanying table shows different training systems that were looked at on the vertical axis and the array of approaches taken in their acquisitions on the horizontal axis.

Some acquisitions have involved the purchase of simulation tools, simulation training, or both tools and training; others have bought availability of simulation tools. Buyers have used a variety of contract vehicles of differing durations to acquire simulation training. Some have obtained both tools and training from the same provider; others have obtained them from different sources. And funding for tool development has run the gamut from public (funded by the defense entity) to private (funded by contractors, tool makers, or original equipment manufacturers).

Is Alt#4 Economically Sound?

The study also reviewed economic theory related to Alt#4 and other business approaches. Its main conclusion: Alt#4 is based on sound economic principles and has good prospects for delivering efficiencies to DoD's training community. However, theory suggests that Alt#4 is most likely to realize the cost efficiencies and innovations of the private sector when it is applied to technologies with commercial applications and to relatively straightforward training needs.

How Can the DoD Implement and Evaluate Alt#4?

Additionally, the study sketched out plans to implement and evaluate a prototype of Alt#4. To implement the prototype, the DoD needs to establish four key components: a governance/oversight entity; a simulation tool catalog standards, sustainment, and investment entity we call the "catalog conductor"; an advisory board; and a contracting and grants office. The DoD needs to work through one or more existing users of simulation training inside the DoD and existing private sector simulation tool and service providers.

The study's implementation plan recognizes that the prototype will have a limited budget—$15 million over three years. These funds need to support the operation of the core prototype components (governance, catalog conductor, advisory board, and contracting/grants support), provide seed money for the catalog conductor to invest in the tool vendor market, and cover costs associated with compliance testing.

Recommendations

The study recommends that DoD take two concrete steps with respect to a prototype of Alt#4:

• Proceed with implementing the prototype. Similar approaches used by other entities have succeeded. Moreover, the observed balance between the plausibility and risks of this approach makes it an ideal candidate program for the DoD to test, pilot, or prototype.
• Arrange for an impartial outsider to collect and track data. To evaluate the prototype, the DoD needs an outside entity able to evaluate the process and outcomes and assess the prototype's efficiency. Such evaluations will require that the DoD keep careful records on the component side and survey both users and providers of training about their perceived levels of satisfaction and effectiveness.

Notes

• [1] Simulations connote hardware or software that help the DoD conduct live (e.g., real people operating real systems), virtual (real people operating simulated systems), or constructive (simulated people operating simulated systems) training. Heavyweight simulations involve high levels of fidelity and realism; lightweight simulations involve less fidelity and realism.
• [2] "Cost-plus" is short for "cost-plus-fixed (or incentive or award) fee," an arrangement whereby contractors are reimbursed for the real costs they incur, plus a fixed profit or profit margin.