This report describes RAND's assistance to the Air Force in designing physical task simulations (PTSs) to measure the occupationally relevant physical requirements for battlefield airmen (BA) specialties and setting standards for BA physical performance on the PTSs. This research will provide the foundation for Air Force performance measures and tests that meet scientific, technical, and best practice standards.

Physical Task Simulations

Performance Measures for the Validation of Physical Tests and Standards for Battlefield Airmen

Sean Robson, Maria C. Lytell, Anthony Atler, Jason H. Campbell, Carra S. Sims

ResearchPublished Feb 20, 2020

In January 2013, the Chairman of the Joint Chiefs of Staff and the Secretary of Defense rescinded the 1994 Direct Ground Combat Definition and Assignment Rule and mandated that "[v]alidated gender-neutral occupational standards will be used to assess and assign Service members not later than September 2015." In support of this mandate, the U.S. Air Force asked RAND to assist its development and validation of gender-neutral tests and standards for six battlefield airmen (BA) specialties, which were the only occupational specialties that remained closed to women in the Air Force at the time of the study (now open to women). This report describes RAND's assistance to the Air Force on two fronts: (1) designing physical task simulations (PTSs) to measure the occupationally relevant physical requirements for BA specialties and (2) setting standards for BA physical performance on the PTSs. This research will provide the foundation for Air Force performance measures and tests that meet scientific, technical, and best practice standards.

Key Takeaways

PTSs are measures of job performance

  • PTSs can be useful measures of job performance when job tasks are difficult to observe, as is the case with many BA tasks on operational missions. Because they simulate job tasks, stakeholders, such as job incumbents, generally view work simulations as job-relevant.

Five principles emerged for PTS development

  • Our research identified five important principles for developing PTSs: develop PTSs where effective performance is influenced by physical ability; develop PTSs that are representative of tasks, abilities, and mission types; standardize PTSs to the extent possible; to the extent possible, have PTSs reflect how tasks are performed in actual mission environments; and design reliable and accurate measurement of PTS performance.
  • For PTS assessments to be reliable and accurate, different performance levels of participants need to be detectable.
  • Setting PTS times too fast may subsequently result in setting the predictive physical test scores too high. On the other hand, setting PTS cutoff times too slow may subsequently result in increased operational risks because of insufficient physical readiness.

Recommendations

  • The Air Force should conduct additional research to explore development of other measures of physical performance.
  • The Air Force should conduct additional research to examine the relationship between physical ability test performance and other important organizational outcomes.
  • Specifically, the Air Force should consider addressing the following questions in future research: How do scores on the predictive physical tests correlate with other important organizational outcomes, such as injuries, promotion rates, and retention? How can individuals best prepare physically for testing and training in each specialty? How can individuals best prepare to perform the critical physical tasks (CPTs) in each specialty? How much do individuals' scores on the predictive physical tests change over time and the course of their careers? Does individuals' improvement on the predictive physical tests correspond to improvement in performance on occupationally relevant physical tasks?
  • The Air Force is encouraged to explore such questions and periodically reexamine its physical readiness standards to help ensure operators are physically prepared and capable of performing occupationally relevant CPTs.

Topics

Document Details

Citation

Chicago Manual of Style

Robson, Sean, Maria C. Lytell, Anthony Atler, Jason H. Campbell, and Carra S. Sims, Physical Task Simulations: Performance Measures for the Validation of Physical Tests and Standards for Battlefield Airmen. Santa Monica, CA: RAND Corporation, 2020. https://www.rand.org/pubs/research_reports/RR1595.html.
BibTeX RIS

Research conducted by

The work in this report was co-sponsored by the Air Force Directorate of Military Force Management Policy, Deputy Chief of Staff for Manpower, Personnel, and Services (AF/A1P), the Vice Commander in Air Education and Training Command (AETC/CV), the Vice Commander in Air Force Special Operations Command (AFSOC/CV), and the Directorate of Air and Space Operations (ACC/A3). This research was conducted within the Manpower, Personnel, and Training Program within RAND Project AIR FORCE.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.