Learning About Quality: How the Quality of Military Personnel Is Revealed Over Time
Jan 1, 2003
|PDF file||0.1 MB||
Use Adobe Acrobat Reader version 10 or higher for the best experience.
RAND has developed a broader measure of enlisted military personnel quality that incorporates information about performance on the job, as revealed through the speed of promotions. Performance reflects the quality of the job match between the member and the military and depends on the member's effort, ability, and preference for military service. The promotion process reveals this quality by establishing criteria that apply to all members and by promoting faster those members who are soonest to meet and surpass the criteria. According to the extended definition of quality, the services are retaining higher-quality members.
Policymakers and analysts traditionally assess the quality of enlisted personnel by relying on two measures: high school diploma graduate status and Armed Forces Qualification Test (AFQT) score. These measures are valid predictors of the completion of training, completion of the first term of service, and satisfactory performance on written and hands-on tests of skills and knowledge. They do not, however, provide information about actual performance during a service member's first term, i.e., about quality as revealed on the job. Such information is necessary if the military is to have a full understanding of whether it is keeping the members with the best performance and potential.
To provide a clearer picture of how the quality of military personnel is revealed over time, RAND Corporation researchers implemented a new, broader measure of quality that incorporates information from the military promotion system. Using this measure to examine the quality of U.S. enlisted personnel in the first term of service, they found that higher-quality members of the service are indeed more likely to reenlist. In contrast, by the traditional measure of quality—AFQT score—higher-quality members are somewhat less likely to reenlist.
The broader quality measure is not simply a measure of ability or aptitude alone. It also depicts the quality of the job match between the member and the military. Ability, as reflected in the AFQT score, is one important piece of the match quality, but another is the member's own quality factor. Thus, the broader measure of quality depends on the member's AFQT and quality factor. Although AFQT can be observed, the quality factor must be inferred from data.
The quality factor reflects the interaction of the member's taste for the military, level of effort, and aspects of ability not measured by the AFQT score. A high quality factor indicates that the member has made the effort to perform consistently well and that the service, having viewed the member in comparison with other members being judged by the same criteria in the same occupation, recognizes the superior performance.
The researchers used data on each member's AFQT score and speed of promotion to E-4 and E-5 to infer information about the member's quality. (Enlistees move through the first three ranks at virtually the same pace). Each service's promotion system bases advancement on criteria that reflect acquired skills and knowledge, physical fitness, duty performance, awards and decorations, and education and training. The promotion process reveals quality by applying the criteria to all members and by promoting faster those members who are soonest to meet the criteria. High-AFQT members are promoted faster, and, controlling for the effect of AFQT, fast promotion to E-4 and to E-5 reveals that the member's quality factor is likely to be high.
The relationship between promotion times and personnel quality is reflected in the figure, which shows the cumulative percentage of E-4s who have been promoted to E-5 against the number of months in E-4. Personnel are divided into those members of higher quality upon entry (i.e., those with a high school diploma and a score in the upper half of the AFQT score distribution), and those of lower quality upon entry (the remainder); each of these groups has also been divided into "fast" or "slow" to E-4 (shorter or longer than the median time of those in their entering cohort who reached E-4). As shown, personnel of higher quality upon entry are promoted to E-5 sooner than those of lower quality upon entry. More to the point, holding AFQT score constant, those promoted fast to E-4 are also promoted fast to E-5, suggesting that the first fast promotion was not a random outcome.
The analysis used a data set of all enlisted members joining the military between fiscal year (FY) 1979 and FY 1992. Using data on the number of months a member took to reach E-4 and E-5 promotion, the researchers estimated a model in which promotion speeds depends on quality, and quality depends on the member's AFQT and quality factor. (Separate models were estimated by year of entry, specialty, and service.) The model allows each member's quality measure to be updated from month to month, so it is possible to track a member's estimated quality over time and see how it changes in comparison to the quality of other members.
The study found that, as expected, AFQT score was positively related to the new measure of quality. Among the other key findings:
Higher-quality members are more likely to reenlist. According to the new measure, the services are retaining higher-quality members. This is so despite the fact that studies of the relationship between reenlistment and AFQT generally find that higher-AFQT members are somewhat less likely to stay in the military than lower-AFQT members. The research indicates that by the end of the first term, members have sorted themselves into civilian and military careers based on the quality of job match as revealed over the term. Service members with a higher overall quality of job match tend to stay in the military, whereas those with a lower quality of job match tend to leave.
On average across occupations, the member-specific quality factor accounts for a large fraction of the variance of quality among members at the end of the first term. The quality factor accounts for 92, 54, and 87 percent of the variance in quality for the Army, Air Force, and Marine Corps, respectively; AFQT accounts for the remainder. Therefore, most of the variation in quality across members does not arise from the information observed at entry into service (AFQT and high school graduation status), but rather from information revealed during the first term of military service. This finding strongly supports theoretical literature that emphasizes the role of firms' learning about the quality of workers through their performance on the job.
Future work can extend the application of the new measure of quality. For example, the measure can be used to determine whether members with a high quality of job match to the military are more likely to reach positions of higher rank and greater responsibility than those with a low quality of job match. It can also be used to determine whether a given policy has a different effect on higher-quality members than on lower-quality members.
In the future, additional data about such topics as members' reenlistment decisions or promotions to E-6 and E-7 could be incorporated into the model to improve its usefulness. The model might also be extended to allow for comparisons across services.
This report is part of the RAND Corporation Research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.