The Effect of the Assessment of Recruit Motivation and Strength (ARMS) Program on Army Accessions and Attrition

In February 2005, the U.S. Army allowed six Military Entrance Processing Station (MEPS) locations—Atlanta, Buffalo, Chicago, Sacramento, San Antonio, and San Diego—to enlist Army applicants who did not meet applicable weight-for-height and body fat percentage standards but who passed a test known as the Assessment of Recruit Motivation and Strength (ARMS) test.* ARMS has two components: a step test and a pushup test (initially, it also had a lift component). Successfully completing these tests is meant to indicate that a recruit has the physical and motivational endurance needed to serve in the Army. The Army expanded the use of the ARMS test to eight additional MEPS in February 2006 and to the remaining 51 MEPS in April 2006.

The decision to allow ARMS waivers nationwide was made in a difficult recruiting environment and at a time when the Army was seeking to grow active-duty end strength. The decision was also made with the knowledge that America's obesity epidemic was adversely affecting the supply of eligible recruits and with the belief that ARMS complements existing physical fitness tests used to identify individuals who will and will not fare well in the military. According to data available from the Military Entrance Processing Command, between 1988 and 2007, the mean body mass index (BMI) of Army male applicants increased from 23.8 to 24.9, and the mean BMI of female applicants increased from 22.3 to 23.9 (Figures 1 and 2). Even-larger increases in BMI are apparent among the heaviest applicants. For example, BMI at the 75th percentile of the applicant BMI distribution increased from 26.1 to 27.7 for males and from 23.8 to 25.9 for females. BMI in the overall U.S. youth population increased by even more during this period (Asch et al., 2009).

The Army granted waivers to overweight and over–body fat applicants who passed the ARMS test, hoping that this would increase enlistments without adversely affecting attrition and other measures of recruit readiness. The research reported in this document investigates whether implementation of ARMS succeeded in meeting this goal by examining military personnel data obtained from the Military Entrance Processing Command and the United States Army Accessions Command on nearly 260,000 individuals who applied to the Army between 2004 and 2007.

Methods

One way to measure the effect of ARMS on accessions would be simply to count the number of Army recruits who enlisted with an ARMS waiver. However, there are two main reasons why this measure is not likely to provide a reasonable estimate of the effect of ARMS on Army accessions. First, Army recruits who fail weight and body fat standards at their first medical exam are allowed to return at a later date for retesting. Our data indicate that, before ARMS was implemented, about 45 percent of Army applicants who initially failed weight and body fat standards later met those standards, and 89 percent of those applicants accessed within 30 days of their last physical exam. Thus, it seems likely that some fraction of recruits who enlisted with an ARMS waiver would have enlisted in the absence of ARMS by losing the weight and body fat necessary to meet Army standards. Second, it is possible that the availability of ARMS had a broader effect than just increasing the number of accessions of recruits who failed to meet weight and body fat standards. The availability of ARMS might have encouraged some individuals who were overweight but within body fat standards to apply when they might otherwise not have.

To capture the full effect of ARMS on accessions, we compared changes in accessions over time at a set of MEPS that did implement ARMS with changes at a set of MEPS that did not implement the test. This difference-in-differences approach assumes that the accession experience of MEPS that did not implement ARMS can serve as the counterfactual experience of MEPS that did implement ARMS (i.e., that the former would have been the experience of the latter had the latter not implemented ARMS).

The reader will recall that ARMS was first implemented at six MEPS in February 2005. These six MEPS were the only MEPS authorized to grant ARMS waivers to overweight and over–body fat applicants between February 2005 and January 2006. Thus, our approach was to compare the change in accessions between 2004 and 2005 at the six ARMS study sites with the change in accessions between 2004 and 2005 at the other 59 MEPS.

The Effect of ARMS on Army Accessions

The difference-in-differences estimate of the effect of ARMS on Army accessions is most easily understood in simple tabular form. In section A of Table 1, we see that male accessions occurring within 30 days of the last observed medical exam fell by 11.6 percent between 2004 and 2005 in nonstudy sites but increased by 6.6 percent in study sites.** This means that accessions in study sites increased by 6.6 – (–11.6) = 18.3 percent relative to nonstudy sites during that period. Female accessions in study sites increased by 24 percent relative to nonstudy sites. In section B, we see that this relative increase in accessions at study sites was not attributable to a relative increase in the accession rate. In fact, our data indicate that the accession rate in study sites fell relative to nonstudy sites between 2004 and 2005. This suggests that the relative growth in accessions must have been attributable to a relative increase in applications, which is exactly what we see in section C. Male applications at study sites grew by 21 percent relative to nonstudy sites between 2004 and 2005, and female applications grew by 28 percent.

It is notable that the relative growth in both applicants and accessions at ARMS study sites was primarily among overweight applicants and accessions. Male and female overweight but within–body fat applications at the ARMS study sites grew by 21 and 30 percent, respectively, relative to the nonstudy sites (section E). Male and female over–body fat application at the ARMS study sites increased by 268 and 197 percent, respectively, relative to the nonstudy sites (section F). Note also that within-weight applications at ARMS study sites grew relative to nonstudy sites (section D), although by a substantially smaller amount (8.5 and 3.9 percentage points for males and females, respectively) than overweight applications. The same pattern is evident when we examine accessions (sections G, H, and I). Finally, section J shows that the strong relative increase in the number of overweight and over–body fat applications at ARMS study sites was not correlated with a change in the Armed Forces Qualification Test (AFQT): The percentage of Category I-IIIA recruits fell by about 8 percentage points at both the study and nonstudy sites.

We examined the robustness of the findings reported in Table 1 by controlling for differences in local economic conditions and recruiting resources and for the possibility that some of the relative growth in applications and accessions at ARMS study sites was attributable to the fact that these sites drew applicants and accessions away from nearby MEPS. The basic pattern of results, however, is unaffected by these considerations: The number of overweight applications and accessions, but not the accession rate, grew strongly at ARMS study sites relative to nonstudy sites between 2004 and 2005. Taking these factors into account, our estimate is that ARMS increased overweight but within–body fat male (female) accessions by 13 (26) percent and overweight and over–body fat male (female) accessions by 350 (192) percent. Overall, our estimates imply that ARMS increased overweight male (female) accessions by 35 (62) percent. These estimates also imply that ARMS had no statistically significant effect on the number of within-weight Army applicants or accessions.

The Effect of ARMS on Army Attrition

Our analyses suggest that ARMS was effective in increasing accessions, but did that increase in accessions come at the cost of higher attrition? To answer this question, we began by examining how 6- and 18-month attrition rates varied with weight and body fat as measured at an applicant's first medical exam and whether that applicant passed the ARMS test. Table 2 shows that attrition rates were elevated among overweight but within–body fat male enlistees. For example, in this sample, the 6-month attrition rate of male enlistees who were more than 15 pounds overweight at the time of their first medical exam was 7.5 percent, compared with an average of 5.5 percent for within-weight enlistees. Table 2 further shows that 6-month attrition rates were even higher among over–body fat male enlistees who either did not take or failed the ARMS test (8.0 and 9.0 percent, respectively). However, of great significance is the fact that the attrition rate of over–body fat male enlistees who passed the ARMS test was not statistically different from the attrition rate of within-weight enlistees. The 18-month attrition rate of female enlistees who passed ARMS was actually somewhat lower than the 18-month attrition rate of within-weight female enlistees. These data suggest that the ARMS test is effective at identifying over–body fat applicants who are as likely to complete initial training as within-weight applicants.

Although it would appear from the results reported in Table 2 that overweight enlistees who passed ARMS had relatively low attrition rates, it is not clear from this evidence alone that ARMS results in lower attrition rates overall. We know that accessions increased in study sites relative to nonstudy sites and that those accessions were disproportionately overweight and over–body fat. The net effect of this change in the composition of accessions on attrition rates is unclear. On the one hand, attrition was higher among overweight accessions, which would tend to increase attrition rates under the ARMS program. On the other hand, attrition was lower among overweight accessions who passed the ARMS test. Thus, to the extent that ARMS screens out applicants who might otherwise have accessed and separated, ARMS could result in lower overall attrition rates. To test whether ARMS affected overall attrition in the six study sites, we employed the same difference-in-differences framework we used to study accessions. These difference-in-differences estimates suggest that ARMS had no net effect on male or female attrition rates either in the overall accession population or in the population of overweight accessions.

Although we find that ARMS had no effect on attrition rates, it is nonetheless possible that those who accessed through an ARMS waiver separated for different reasons than those who did not. In particular, it might be the case that ARMS accessions were more susceptible to injury than their non-ARMS counterparts and were therefore more likely to separate for medical reasons. However, an analysis of separation codes available in our administrative data suggest that male accessions who passed the ARMS test and separated within 18 months of accession were somewhat less likely than within-weight accessions to separate for medical reasons but somewhat more likely to separate because they did not meet physical (e.g., weight and body fat) standards (see Table 3). Curiously, the same was true of male accessions who failed the ARMS test and so presumably met weight standards prior to accession. Female accessions who took the ARMS test prior to accession were also less likely than within-weight accessions to separate for medical reasons. This evidence, then, suggests that, if anything, ARMS accessions were less susceptible than non-ARMS accessions to injury that resulted in separation. However, it might still be the case that ARMS accessions were more likely than non-ARMS accessions to suffer other types of injuries, including those that impede performance but do not result in separation. We did not have access to data that would allow us to investigate this issue.

Conclusion

When the Army implemented ARMS at the six study sites in 2005 and then at the remaining MEPS in 2006, the hope was that the test would increase accessions among overweight and over–body fat applicants without adversely affecting attrition. The evidence reported here suggests that implementation of ARMS at the six study sites succeeded in doing just that. Our difference-in-differences estimates imply that the implementation of ARMS increased male accessions by 13 percent and female accessions by 20 percent in 2005 and that virtually all of that percentage increase came from overweight and over–body fat accessions. Despite the fact that ARMS resulted in a large increase in the proportion of applicants who were overweight and over–body fat, our estimates imply that ARMS had no effect on attrition rates. This suggests that the ARMS test is effective in identifying overweight and over–body fat recruits who are as likely as within-standards recruits to complete initial training. Moreover, ARMS has been quite inexpensive to implement. Our estimates imply that the cost of ARMS per additional accession was $163 in fiscal year 2007, which compares very favorably with the estimated per-accession cost of other Army recruiting initiatives.

We temper this overall conclusion with several caveats. First, we cannot say for certain whether the broader implementation of ARMS since 2005 has been as successful as it was at the six study sites. However, at a minimum, the available evidence indicates that overweight and over–body fat applicants who pass ARMS are no more likely to separate than are applicants who meet weight and body fat standards. Second, it remains to be seen whether ARMS accessions in the longer run will turn out to be as productive on average as within-standards accessions. Although our tabulations suggest that ARMS accessions are, if anything, somewhat less likely than non-ARMS accessions to separate for medical reasons, it may be that they are more prone to injuries (e.g., heat illness, musculoskeletal injury) that do not result in separation but that make these accessions less productive. Moreover, it is important to acknowledge that ARMS appears to increase the number of overweight but within–body fat accessions. These individuals would not be subject to the ARMS test, and our evidence suggests that they are somewhat less likely than within-weight recruits to complete initial training.

The decision to implement ARMS was made in a weak recruiting environment. Today, the recruiting environment is much stronger (largely because of the weak civilian labor market), and, as a result, the Army decided to suspend ARMS as of October 2009. However, even in a very strong recruiting environment like the current one, ARMS can serve a highly useful role by identifying enlistees who, despite weight problems, can be productive members of the Army enlisted force. The success of the ARMS test suggests what might appear obvious in hindsight: The population of overweight and over–body fat individuals is quite heterogeneous. Some of these individuals are truly unfit for service, but many others possess the desire and ability to serve their country in the armed forces and, given the chance, will succeed in that capacity. In both weak and strong recruiting environments, then, the ARMS test offers a simple, cost-effective way to separate the fit from the unfit.

Reference

Asch, Beth J., Christopher A. Buck, Jacob Alex Klerman, Meredith Kleykamp, and David S. Loughran, Military Enlistment of Hispanic Youth: Obstacles and Opportunities, Santa Monica, Calif.: RAND Corporation, MG-773-OSD, 2009. As of May 20, 2011:
http://www.rand.org/pubs/monographs/MG773.html

Notes

* Hereafter, we use weight standards to refer to weight-for-height standards.

** All counts are expressed in natural logs. The difference in these log counts approximate percentage changes. Here, the difference rounds to 18.3 percent.