Using the National Training Center Instrumentation System to Aid Simulation-Based Acquisition

by Andrew Cady

Full Document

FormatFile SizeNotes
PDF file 2.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Though current data sources for simulation models used in the United States Department of Defense (DoD) acquisition process are many and varied, none adequately represents how weapon systems behave in combat in a robust, quantifiable manner, leading to uncertainty in the acquisition decision making process. The objective of this dissertation is to improve this process by developing empirically derived measures of direct fire behaviors from U.S. Army National Training Center (NTC) data and by demonstrating how these measures can be used to support acquisition decisions based on the output of simulation-based modeling. To accomplish this, I employ a three-part methodology.

First, I identify the current data sources for models and simulations used in the defense acquisition process. Of these four data sources — historical combat, operational testing, other simulations, and subject matter expert (SME) judgment — no single source can adequately describe combat behaviors of weapon systems across a wide range of operational environments.

Second, I turn to the NTC data as a potential solution to this gap in data sources. I first examine prior NTC-based research and lessons this literature holds for current and future research. I examine the doctrinal underpinnings of maneuver combat behaviors, deriving five important aspects of direct fire — two of which are operationalized in this dissertation: direct fire engagement and movement and maneuver. I examine the NTC instrumentation system data generation process, strengths, and drawbacks to determine if it could measure these. Finally, I describe four measures that I derive to describe maneuver combat behavior — weapon system probability of hit, weapon system rate of fire, unit dispersion, and unit speed.

Third, I compare the measures derived in this dissertation against baseline measures from the Joint Conflict and Tactical Simulation (JCATS) simulation model to determine the difference the two sources of measures — actual NTC behavior and JCATS baseline — make in three outcomes: exchange ratio, drawdown of forces rate, and volume of fire. To perform this comparison, I create a scenario based on prior simulation studies along with four excursions to test the influence of changes in mission, enemy, and terrain on the impact of data source. I analyze the results of 300 runs of the JCATS model using a series of linear regressions. For each excursion, regression models indicate a highly significant effect of data source on each model outcome.

I conclude this dissertation with a recommendation that the measures described herein form the basis for a larger system of NTC-based behavioral measurement for modeling and simulation (M&S) data. I also recommend several software and hardware improvements to the NTC instrumentation system that could improve its utility as both a data source and a training resource. As future research, I recommend applying advanced analytic techniques to these data, applying these methods to other combat training centers and applying these measures to training and tactics development.

Table of Contents

  • Chapter One


  • Chapter Two

    Current Sources of Data for Combat Simulation Modeling

  • Chapter Three

    Deriving Behavioral Combat Measures from NTC-IS

  • Chapter Four

    Testing the Difference in JCATS Model Outcomes from Using NTC-based Data

  • Chapter Five

    Conclusions and Policy Recommendations

  • Appendix A

    Pairing Fires and Hits in NTC-IS

  • Appendix B

    Line of Sight Algorithm Description and Assumptions

  • Appendix C

    Additional NTC-IS Measures Not Tested in this Research

  • Appendix D

    Verification and Validation of the NTC-IS Data

  • Appendix E

    Full Results of NTC-IS Analysis

  • Appendix F

    Regression Model Specifications and Diagnostics

Research conducted by

This document was submitted as a dissertation in September 2017 in partial fulfillment of the requirements of the doctoral degree in public policy analysis at the Pardee RAND Graduate School. The faculty committee that supervised and approved the dissertation consisted of Bryan Hallmark (Chair), Joe Martz, and Randall Steeb.

This report is part of the RAND Corporation Dissertation series. Pardee RAND dissertations are produced by graduate fellows of the Pardee RAND Graduate School, the world's leading producer of Ph.D.'s in policy analysis. The dissertations are supervised, reviewed, and approved by a Pardee RAND faculty committee overseeing each dissertation.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.