Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.2 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief

Abstract

Between 2002 and 2004, the Department of Defense (DoD) investigated ways to improve how it acquires simulations and simulation training support. Under one alternative the DoD considered, the department no longer would buy and own simulation materials; instead, it would contract with the private sector for simulation goods and services. RAND found that this alternative rests on sound economic principles and resembles other successful acquisition approaches. RAND recommended that the DoD implement a prototype of this approach and use impartial outside evaluators to gauge its success.

For decades, the Department of Defense (DoD) has used simulations to train uniformed and civilian defense personnel.[1] By using computer hardware and software to portray and model a variety of tasks and scenarios, the DoD believes that—for many skills—it can train personnel more quickly and at less cost and risk than it can by conducting live training.

Beginning in 2002, the DoD undertook a two-year analytical process to identify ways to improve how it obtains simulations and simulation training support. At the conclusion of this process, called the Training Capabilities Analysis of Alternatives, the DoD decided to pursue a prototype of one alternative that it had identified. This alternative, known as Alternative #4 (Alt#4), proposes that the DoD abandon its traditional practice of acquiring and owning training simulators, software, and teaching materials in favor of contracting for training services from private sector vendors.

A recent RAND study evaluated Alt#4, compared it to other approaches to buying simulations and simulation training support, reviewed relevant economic theories, and provided detailed implementation and evaluation plans for a prototype. The study assembled case summaries of relevant programs and acquisition approaches based on open-source literature reviews and on interviews of industry and DoD personnel familiar with simulation acquisition issues. It used these case studies and economic theories to examine and validate Alt#4's underlying logic and core principles in light of other methods used to acquire simulation support and services.

Alt#4: Breaking from the Traditional Approach to Acquiring Simulation

Traditionally, the DoD has purchased simulation tools directly from developers with "cost-plus" contracts and often has paid those same developers to run training exercises using these tools.[2] This has meant that the DoD has ended up owning these simulation tools, frequently at great cost. Some observers have characterized this approach as being both fiscally wasteful and a hindrance to innovation because it ties the DoD to the specific tools and simulations that it has acquired, often for lengthy periods of time.

Alt#4 proposes that the DoD avoid these problems by buying only training support, not simulation tools and training support, and by using firm-fixed-price (FFP) contracts tied to training outcomes rather than cost-plus contracts. The DoD no longer would buy simulation tools; instead, providers of training would buy or license tools and the DoD would contract with them for simulation training services. The DoD would get out of the ownership cycle. But although the main responsibility for funding the development of simulation tools would shift to the private sector under Alt#4, the DoD could contribute through seed money investments.

How Does Alt#4 Compare with Other Approaches to Buying Simulations and Simulation Training?

The study compared Alt#4 to other approaches that government entities use to acquire simulation tools and training. The accompanying table shows different training systems that were looked at on the vertical axis and the array of approaches taken in their acquisitions on the horizontal axis.

Some acquisitions have involved the purchase of simulation tools, simulation training, or both tools and training; others have bought availability of simulation tools. Buyers have used a variety of contract vehicles of differing durations to acquire simulation training. Some have obtained both tools and training from the same provider; others have obtained them from different sources. And funding for tool development has run the gamut from public (funded by the defense entity) to private (funded by contractors, tool makers, or original equipment manufacturers).

Characteristics of Different Business Models Used to Acquire Simulations and Simulation Training

Model Who Buys Tools Who Funds Tool Development Who Builds Tools Who Owns Intellectual Property/Assembled Simulators "Units" Tools Provided In Who Provides Training "Units" Training Provided In Length of Contracts
JSIMS DoD DoD Contractor DoD (full government rights)/DoD Cost-plus Same contractor Billable contractor hours Long and locked in
SAGIS DoD DoD Contractor DoD (full government rights)/DoD Cost-plus Uniformed personnel Classes Acquisition length for tools, no contracts for training
VCCT Contractor Contractor Contractor Contractor/Contractor (a) Same contractor FFP person-hours of training Relatively short
DMO DoD Contractor Contractor Contractor/Contractor FFP hours of simulator availability Uniforms or different contractors Varies Long, performance extended tools contracts
FFTU PFI contractor OEMs OEMs, subcontractor OEM/PFI contractor (a) PFI contractor (with transferred MOD personnel) FFP training days; excess capacity sold for MOD & PFI profit Long
MCTS PFI contractor OEMs OEMs, subcontractor OEM/PFI contractor Fixed-price for tool availability RN uniforms and PFI contractor FFP training days Long
Alt#4 TSP contractor OEMs, perhaps with catalog conductor seed money OEMs OEM/TSP contractor (a) TSP contractor FFP per training outcome Short

NOTES: JSIMS: Joint Simulation System; SAGIS: Special Operation Forces Air Ground Interface Simulator; VCCT: Virtual Convoy Combat Trainer; DMO: Distributed Mission Operations; FFTU: United Kingdom Ministry of Defence (MOD) Naval Recruiting and Training Agency—Fire-Fighting Training Units; MCTS: MOD Maritime Composite Training System; OEM: original equipment manufacturer; PFI: private finance initiative; TSP: training service providers. Blank cells in the "Units" Tools Provided In column means that the DoD or MOD is buying training outcome, not tools.

Is Alt#4 Economically Sound?

The study also reviewed economic theory related to Alt#4 and other business approaches. Its main conclusion: Alt#4 is based on sound economic principles and has good prospects for delivering efficiencies to DoD's training community. However, theory suggests that Alt#4 is most likely to realize the cost efficiencies and innovations of the private sector when it is applied to technologies with commercial applications and to relatively straightforward training needs.

How Can the DoD Implement and Evaluate Alt#4?

Additionally, the study sketched out plans to implement and evaluate a prototype of Alt#4. To implement the prototype, the DoD needs to establish four key components: a governance/oversight entity; a simulation tool catalog standards, sustainment, and investment entity we call the "catalog conductor"; an advisory board; and a contracting and grants office. The DoD needs to work through one or more existing users of simulation training inside the DoD and existing private sector simulation tool and service providers.

The study's implementation plan recognizes that the prototype will have a limited budget—$15 million over three years. These funds need to support the operation of the core prototype components (governance, catalog conductor, advisory board, and contracting/grants support), provide seed money for the catalog conductor to invest in the tool vendor market, and cover costs associated with compliance testing.

Recommendations

The study recommends that DoD take two concrete steps with respect to a prototype of Alt#4:

  • Proceed with implementing the prototype. Similar approaches used by other entities have succeeded. Moreover, the observed balance between the plausibility and risks of this approach makes it an ideal candidate program for the DoD to test, pilot, or prototype.
  • Arrange for an impartial outsider to collect and track data. To evaluate the prototype, the DoD needs an outside entity able to evaluate the process and outcomes and assess the prototype's efficiency. Such evaluations will require that the DoD keep careful records on the component side and survey both users and providers of training about their perceived levels of satisfaction and effectiveness.

Notes

  • [1] Simulations connote hardware or software that help the DoD conduct live (e.g., real people operating real systems), virtual (real people operating simulated systems), or constructive (simulated people operating simulated systems) training. Heavyweight simulations involve high levels of fidelity and realism; lightweight simulations involve less fidelity and realism.
  • [2] "Cost-plus" is short for "cost-plus-fixed (or incentive or award) fee," an arrangement whereby contractors are reimbursed for the real costs they incur, plus a fixed profit or profit margin.

This report is part of the RAND Corporation research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.