This report describes RAND's assistance to the Air Force in designing physical task simulations (PTSs) to measure the occupationally relevant physical requirements for battlefield airmen (BA) specialties and setting standards for BA physical performance on the PTSs. This research will provide the foundation for Air Force performance measures and tests that meet scientific, technical, and best practice standards.
Physical Task Simulations
Performance Measures for the Validation of Physical Tests and Standards for Battlefield Airmen
Download
Download eBook for Free
| Format | File Size | Notes |
|---|---|---|
| PDF file | 6.6 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Purchase
Purchase Print Copy
| Format | List Price | Price | |
|---|---|---|---|
| Add to Cart | Paperback172 pages | $49.95 | $39.96 20% Web Discount |
Research Questions
- How can PTSs be designed to best approximate BA critical physical tasks (CPTs)?
- Are PTSs representative of tasks, abilities, and mission types?
- How can PTSs be standardized to facilitate comparisons between participants' performance on each PTS?
- Should PTSs reflect how tasks are performed in actual mission environments?
- What features are important to ensure that PTS assessments are reliable and accurate?
- What standards should be used to differentiate between successful and unsuccessful PTS performance?
In January 2013, the Chairman of the Joint Chiefs of Staff and the Secretary of Defense rescinded the 1994 Direct Ground Combat Definition and Assignment Rule and mandated that "[v]alidated gender-neutral occupational standards will be used to assess and assign Service members not later than September 2015." In support of this mandate, the U.S. Air Force asked RAND to assist its development and validation of gender-neutral tests and standards for six battlefield airmen (BA) specialties, which were the only occupational specialties that remained closed to women in the Air Force at the time of the study (now open to women). This report describes RAND's assistance to the Air Force on two fronts: (1) designing physical task simulations (PTSs) to measure the occupationally relevant physical requirements for BA specialties and (2) setting standards for BA physical performance on the PTSs. This research will provide the foundation for Air Force performance measures and tests that meet scientific, technical, and best practice standards.
Key Findings
PTSs are measures of job performance
- PTSs can be useful measures of job performance when job tasks are difficult to observe, as is the case with many BA tasks on operational missions. Because they simulate job tasks, stakeholders, such as job incumbents, generally view work simulations as job-relevant.
Five principles emerged for PTS development
- Our research identified five important principles for developing PTSs: develop PTSs where effective performance is influenced by physical ability; develop PTSs that are representative of tasks, abilities, and mission types; standardize PTSs to the extent possible; to the extent possible, have PTSs reflect how tasks are performed in actual mission environments; and design reliable and accurate measurement of PTS performance.
- For PTS assessments to be reliable and accurate, different performance levels of participants need to be detectable.
- Setting PTS times too fast may subsequently result in setting the predictive physical test scores too high. On the other hand, setting PTS cutoff times too slow may subsequently result in increased operational risks because of insufficient physical readiness.
Recommendations
- The Air Force should conduct additional research to explore development of other measures of physical performance.
- The Air Force should conduct additional research to examine the relationship between physical ability test performance and other important organizational outcomes.
- Specifically, the Air Force should consider addressing the following questions in future research: How do scores on the predictive physical tests correlate with other important organizational outcomes, such as injuries, promotion rates, and retention? How can individuals best prepare physically for testing and training in each specialty? How can individuals best prepare to perform the CPTs in each specialty? How much do individuals' scores on the predictive physical tests change over time and the course of their careers? Does individuals' improvement on the predictive physical tests correspond to improvement in performance on occupationally relevant physical tasks?
- The Air Force is encouraged to explore such questions and periodically reexamine its physical readiness standards to help ensure operators are physically prepared and capable of performing occupationally relevant CPTs.
Table of Contents
Chapter One
Introduction
Chapter Two
Principles for Developing Physical Task Simulations
Chapter Three
Developing Physical Task Simulations
Chapter Four
Support to Pilot Testing and Validation
Chapter Five
Setting Standards for BA Physical Performance
Chapter Six
Conclusions
Appendix A
Descriptions of Battlefield Airmen Specialties
Appendix B
Descriptions of Physical Task Simulations
Appendix C
Administrator Checklists
Appendix D
Score Sheets and Feedback Forms
Research conducted by
The work in this report was co-sponsored by the Air Force Directorate of Military Force Management Policy, Deputy Chief of Staff for Manpower, Personnel, and Services (AF/A1P), the Vice Commander in Air Education and Training Command (AETC/CV), the Vice Commander in Air Force Special Operations Command (AFSOC/CV), and the Directorate of Air and Space Operations (ACC/A3). This research was conducted within the Manpower, Personnel, and Training Program within RAND Project AIR FORCE.
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.