This report describes RAND's assistance to the Air Force in designing physical task simulations (PTSs) to measure the occupationally relevant physical requirements for battlefield airmen (BA) specialties and setting standards for BA physical performance on the PTSs. This research will provide the foundation for Air Force performance measures and tests that meet scientific, technical, and best practice standards.
- How can PTSs be designed to best approximate BA critical physical tasks (CPTs)?
- Are PTSs representative of tasks, abilities, and mission types?
- How can PTSs be standardized to facilitate comparisons between participants' performance on each PTS?
- Should PTSs reflect how tasks are performed in actual mission environments?
- What features are important to ensure that PTS assessments are reliable and accurate?
- What standards should be used to differentiate between successful and unsuccessful PTS performance?
In January 2013, the Chairman of the Joint Chiefs of Staff and the Secretary of Defense rescinded the 1994 Direct Ground Combat Definition and Assignment Rule and mandated that "[v]alidated gender-neutral occupational standards will be used to assess and assign Service members not later than September 2015." In support of this mandate, the U.S. Air Force asked RAND to assist its development and validation of gender-neutral tests and standards for six battlefield airmen (BA) specialties, which were the only occupational specialties that remained closed to women in the Air Force at the time of the study (now open to women). This report describes RAND's assistance to the Air Force on two fronts: (1) designing physical task simulations (PTSs) to measure the occupationally relevant physical requirements for BA specialties and (2) setting standards for BA physical performance on the PTSs. This research will provide the foundation for Air Force performance measures and tests that meet scientific, technical, and best practice standards.
PTSs are measures of job performance
- PTSs can be useful measures of job performance when job tasks are difficult to observe, as is the case with many BA tasks on operational missions. Because they simulate job tasks, stakeholders, such as job incumbents, generally view work simulations as job-relevant.
Five principles emerged for PTS development
- Our research identified five important principles for developing PTSs: develop PTSs where effective performance is influenced by physical ability; develop PTSs that are representative of tasks, abilities, and mission types; standardize PTSs to the extent possible; to the extent possible, have PTSs reflect how tasks are performed in actual mission environments; and design reliable and accurate measurement of PTS performance.
- For PTS assessments to be reliable and accurate, different performance levels of participants need to be detectable.
- Setting PTS times too fast may subsequently result in setting the predictive physical test scores too high. On the other hand, setting PTS cutoff times too slow may subsequently result in increased operational risks because of insufficient physical readiness.
- The Air Force should conduct additional research to explore development of other measures of physical performance.
- The Air Force should conduct additional research to examine the relationship between physical ability test performance and other important organizational outcomes.
- Specifically, the Air Force should consider addressing the following questions in future research: How do scores on the predictive physical tests correlate with other important organizational outcomes, such as injuries, promotion rates, and retention? How can individuals best prepare physically for testing and training in each specialty? How can individuals best prepare to perform the CPTs in each specialty? How much do individuals' scores on the predictive physical tests change over time and the course of their careers? Does individuals' improvement on the predictive physical tests correspond to improvement in performance on occupationally relevant physical tasks?
- The Air Force is encouraged to explore such questions and periodically reexamine its physical readiness standards to help ensure operators are physically prepared and capable of performing occupationally relevant CPTs.
Table of Contents
Principles for Developing Physical Task Simulations
Developing Physical Task Simulations
Support to Pilot Testing and Validation
Setting Standards for BA Physical Performance
Descriptions of Battlefield Airmen Specialties
Descriptions of Physical Task Simulations
Score Sheets and Feedback Forms