Report
Implementing and Evaluating an Innovative Approach to Simulation Training Acquisitions
Mar 6, 2006
Evaluating and Implementing an Innovative Approach
Format | File Size | Notes |
---|---|---|
PDF file | 0.1 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Between 2002 and 2004, the Department of Defense (DoD) investigated ways to improve how it acquires simulations and simulation training support. Under one alternative the DoD considered, the department no longer would buy and own simulation materials; instead, it would contract with the private sector for simulation goods and services. RAND found that this alternative rests on sound economic principles and resembles other successful acquisition approaches. RAND recommended that the DoD implement a prototype of this approach and use impartial outside evaluators to gauge its success.
For decades, the Department of Defense (DoD) has used simulations to train uniformed and civilian defense personnel.[1] By using computer hardware and software to portray and model a variety of tasks and scenarios, the DoD believes that—for many skills—it can train personnel more quickly and at less cost and risk than it can by conducting live training.
Beginning in 2002, the DoD undertook a two-year analytical process to identify ways to improve how it obtains simulations and simulation training support. At the conclusion of this process, called the Training Capabilities Analysis of Alternatives, the DoD decided to pursue a prototype of one alternative that it had identified. This alternative, known as Alternative #4 (Alt#4), proposes that the DoD abandon its traditional practice of acquiring and owning training simulators, software, and teaching materials in favor of contracting for training services from private sector vendors.
A recent RAND study evaluated Alt#4, compared it to other approaches to buying simulations and simulation training support, reviewed relevant economic theories, and provided detailed implementation and evaluation plans for a prototype. The study assembled case summaries of relevant programs and acquisition approaches based on open-source literature reviews and on interviews of industry and DoD personnel familiar with simulation acquisition issues. It used these case studies and economic theories to examine and validate Alt#4's underlying logic and core principles in light of other methods used to acquire simulation support and services.
Traditionally, the DoD has purchased simulation tools directly from developers with "cost-plus" contracts and often has paid those same developers to run training exercises using these tools.[2] This has meant that the DoD has ended up owning these simulation tools, frequently at great cost. Some observers have characterized this approach as being both fiscally wasteful and a hindrance to innovation because it ties the DoD to the specific tools and simulations that it has acquired, often for lengthy periods of time.
Alt#4 proposes that the DoD avoid these problems by buying only training support, not simulation tools and training support, and by using firm-fixed-price (FFP) contracts tied to training outcomes rather than cost-plus contracts. The DoD no longer would buy simulation tools; instead, providers of training would buy or license tools and the DoD would contract with them for simulation training services. The DoD would get out of the ownership cycle. But although the main responsibility for funding the development of simulation tools would shift to the private sector under Alt#4, the DoD could contribute through seed money investments.
The study compared Alt#4 to other approaches that government entities use to acquire simulation tools and training. The accompanying table shows different training systems that were looked at on the vertical axis and the array of approaches taken in their acquisitions on the horizontal axis.
Some acquisitions have involved the purchase of simulation tools, simulation training, or both tools and training; others have bought availability of simulation tools. Buyers have used a variety of contract vehicles of differing durations to acquire simulation training. Some have obtained both tools and training from the same provider; others have obtained them from different sources. And funding for tool development has run the gamut from public (funded by the defense entity) to private (funded by contractors, tool makers, or original equipment manufacturers).
Model | Who Buys Tools | Who Funds Tool Development | Who Builds Tools | Who Owns Intellectual Property/Assembled Simulators | "Units" Tools Provided In | Who Provides Training | "Units" Training Provided In | Length of Contracts |
---|---|---|---|---|---|---|---|---|
JSIMS | DoD | DoD | Contractor | DoD (full government rights)/DoD | Cost-plus | Same contractor | Billable contractor hours | Long and locked in |
SAGIS | DoD | DoD | Contractor | DoD (full government rights)/DoD | Cost-plus | Uniformed personnel | Classes | Acquisition length for tools, no contracts for training |
VCCT | Contractor | Contractor | Contractor | Contractor/Contractor | (a) | Same contractor | FFP person-hours of training | Relatively short |
DMO | DoD | Contractor | Contractor | Contractor/Contractor | FFP hours of simulator availability | Uniforms or different contractors | Varies | Long, performance extended tools contracts |
FFTU | PFI contractor | OEMs | OEMs, subcontractor | OEM/PFI contractor | (a) | PFI contractor (with transferred MOD personnel) | FFP training days; excess capacity sold for MOD & PFI profit | Long |
MCTS | PFI contractor | OEMs | OEMs, subcontractor | OEM/PFI contractor | Fixed-price for tool availability | RN uniforms and PFI contractor | FFP training days | Long |
Alt#4 | TSP contractor | OEMs, perhaps with catalog conductor seed money | OEMs | OEM/TSP contractor | (a) | TSP contractor | FFP per training outcome | Short |
NOTES: JSIMS: Joint Simulation System; SAGIS: Special Operation Forces Air Ground Interface Simulator; VCCT: Virtual Convoy Combat Trainer; DMO: Distributed Mission Operations; FFTU: United Kingdom Ministry of Defence (MOD) Naval Recruiting and Training Agency—Fire-Fighting Training Units; MCTS: MOD Maritime Composite Training System; OEM: original equipment manufacturer; PFI: private finance initiative; TSP: training service providers. Blank cells in the "Units" Tools Provided In column means that the DoD or MOD is buying training outcome, not tools.
The study also reviewed economic theory related to Alt#4 and other business approaches. Its main conclusion: Alt#4 is based on sound economic principles and has good prospects for delivering efficiencies to DoD's training community. However, theory suggests that Alt#4 is most likely to realize the cost efficiencies and innovations of the private sector when it is applied to technologies with commercial applications and to relatively straightforward training needs.
Additionally, the study sketched out plans to implement and evaluate a prototype of Alt#4. To implement the prototype, the DoD needs to establish four key components: a governance/oversight entity; a simulation tool catalog standards, sustainment, and investment entity we call the "catalog conductor"; an advisory board; and a contracting and grants office. The DoD needs to work through one or more existing users of simulation training inside the DoD and existing private sector simulation tool and service providers.
The study's implementation plan recognizes that the prototype will have a limited budget—$15 million over three years. These funds need to support the operation of the core prototype components (governance, catalog conductor, advisory board, and contracting/grants support), provide seed money for the catalog conductor to invest in the tool vendor market, and cover costs associated with compliance testing.
The study recommends that DoD take two concrete steps with respect to a prototype of Alt#4:
This report is part of the RAND Corporation Research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.