Research Brief
How Effective Are Military Academy Admission Standards?
Jul 22, 2016
United States Service Academy Admissions
Selecting for Success at the Air Force Academy and as an Officer
Download
Download eBook for Free
| Format | File Size | Notes |
|---|---|---|
| PDF file | 0.7 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Purchase
Purchase Print Copy
| Format | List Price | Price | |
|---|---|---|---|
| Add to Cart | Paperback94 pages | $17.50 | $14.00 20% Web Discount |
Research Question
- Can the selection formula for admission to the United States Air Force Academy (which has been in place for decades) be improved? The selection formula, which is only part of the admission decision, is based on scores calculated as a weighted combination of three elements: academic composite, leadership composite, and selection panel score.
This report examines admission standards at the United States Air Force Academy (USAFA) and officer outcomes to evaluate whether the USAFA admissions formula should be adjusted. USAFA admissions are partly determined based on scores calculated as a weighted combination of three elements: academic composite, leadership composite, and selection panel score. The authors explored relationships between various admissions factors (SAT/ACT composite scores, high school rank divided by class size, selection composite score, leadership composite score, academic composite score, and selection panel score) and USAFA and officer outcomes (grade point average; failure to graduate for academic reasons; failure to graduate because of a desire for a career change; military performance average; overall performance average; and promotion to O-4, O-5, and O-6). The data included records on the nearly 35,000 cadets who attended USAFA from 1980 to 2011 from three Air Force data sources: USAFA registrar admissions records, USAFA cadet records, and Air Force personnel records. Recommendations include removing the selection panel score from the selection composite formula, collecting additional information on applicants, and improving data retention and maintenance.
Key Findings
Most of the Admissions Formula Elements Are Useful in Predicting Outcomes at the United States Air Force Academy (USAFA); Selection Panel Score Is the One Exception
- SAT, academic composite, leadership composite, and selection composite are significant predictors of failure to graduate for academic reasons, graduation versus choosing a career change, grade point average (GPA), military performance average (MPA), and overall performance average (OPA).
- In the overwhelming majority of the analyses, selection panel score was significant, but in the wrong direction: Higher scores were associated with a lower likelihood of success in many of the outcomes.
The Analyses Showed Interesting Differences Depending on Which Outcome Was Predicted
- The admissions factors do a much better job of predicting GPA and OPA than they do of predicting MPA. They also do a better job of predicting graduation versus failure for academic reasons than they do of predicting graduation versus choosing a career change.
- For predicting failure for academic reasons, the academic composite should be weighted more heavily than leadership composite, and the selection panel score should be excluded. For predicting graduating versus choosing a career change, the results suggest weighting leadership composite slightly less than the academic composite.
- Academic composite should be weighted twice that of leadership composite for predicting MPA and nine times that of the leadership composite for predicting OPA and GPA.
- For predicting promotions to O-4, O-5, and O-6, USAFA outcomes (GPA, MPA, and OPA) were much better predictors than the admissions factors, and MPA was a slightly better predictor than USAFA GPA.
Recommendations
- Adjust the selection composite formula. The authors saw repeated evidence that the use of the selection panel score (generated from the selection panel's evaluation of the candidate, which includes a review of admissions liaison officer evaluations, writing samples, teacher evaluations, recommendations, and the candidate fitness assessment) was not helping to select the best candidates; in fact, it was hindering it. They recommend removing the selection panel score from the selection composite formula.
- Collect additional information on applicants. A wide variety of measures (including personality tests, situational judgment tests, critical thinking performance tasks, and writing tests) have been identified as useful tools for predicting performance in workplace settings. Some of these measures may add value to the United States Air Force Academy (USAFA) selection process as well.
- Improve data retention and maintenance. USAFA's archival data sources should contain scores on all possible measures taken at the time of application to the institution, should include data on people who are not admitted to USAFA, and should track cadet and officer performance over time. The authors also recommend that the Air Force consider developing a method for capturing systematic performance evaluations on its officers at various points in their careers for research purposes only. If the Air Force hopes to conduct a study such as this one again in the future, it should ensure that the data being collected and retained today are maintained in such a way that they can provide data-driven recommendations for policy changes in the future.
Related Products
Table of Contents
Chapter One
Introduction
Chapter Two
Overview of USAFA's Admissions Process and Available Data
Chapter Three
Components of the Admissions Process
Chapter Four
Statistical Approach and Data Sources
Chapter Five
Regression Analysis Results
Chapter Six
Recommendations
Appendix A
Average Marginal Effects, Unstandardized Beta Coefficients, and Fit Indexes
Appendix B
Means and Standard Deviations for Each Analysis Population
Appendix C
Estimating New Recommended Formula Weights
Research conducted by
This research was sponsored by the Director of Accession Policy in the Office of the Under Secretary of Defense for Personnel and Readiness and conducted within the Forces and Resources Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.
This report is part of the RAND Corporation research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.