Over the past 15 years, the suicide rate among members of the U.S. armed forces has doubled, with the greatest increase observed among soldiers in the Army (Mancha et al., 2014). This increasing rate is paralleled by a smaller increase the general U.S. population (Curtin, Warner, and Hedegaard, 2016), observed across both genders, in virtually every age group, and in nearly every state. An empirical question exists: What is the extent or degree to which the suicide trend in the Army is unique to the Army, relative to what is observed in the general population?
The Army has typically attempted to address this question by standardizing the general population to look like the military population on demographic characteristics (e.g., age and gender are used most frequently [ Watkins et al., 2018; Reimann and Mazuchowski, 2018] and race/ethnicity as well on occasion) (Ramchand et al., 2011). Standardization aims to make the general population look like the Army population on the characteristics being used in the procedure, thereby allowing for comparisons that are done on populations with the same characteristics and minimizing the ability of the included characteristics to explain the observed rate differences.
However, given the rise in suicide rates over the past decade, the Army wanted to better understand whether standardization based solely on age and gender is enough. Expanding the characteristics on which the general population is standardized to match the Army could be useful to gain a better understanding of the suicide trends in the Army. However, such a change also brings with it some challenges. First, changing the characteristics that are included in the standardization inherently changes the underlying suicide rate for the general U.S. population since there is a shift in the type of Army population with which the matched general population is being compared. In addition, expansion of the characteristics still results in having a large number of unmeasured factors that cannot be included in this type of analysis.
In this study, RAND Arroyo Center investigated how accounting for additional population risk factors beyond age and gender affects suicide rate differences between soldiers and a comparable subset of the general U.S. population. This is a technical report that surveys data and methods available to improve how the suicide rates of the two populations are being compared. The report does not aim to estimate the causal effects of Army service on suicide.
Conceptually, there are three different types of factors on which we might aim to match or standardize the general U.S. population to look like Army.
Set I includes demographic characteristics, such as age, gender/sex, race/ethnicity, and geography of origin. These factors represent stable characteristics that differ between the Army and the general U.S. population because those who choose (or are allowed) to enter the Army are not fully representative of the general population.
Set II includes characteristics that might be somewhat influenced by Army service, such as marital status and education. These factors are largely determined by individual service members' personal interests and aptitudes that generally predate their military service but may also reflect military policy or opportunities that occurred as part of military service.
Set III includes characteristics that are known to be directly affected by military policy or experiences, such as access to firearms, mental health, or occupation.
Standardization on the first set is a minimum requirement for identifying the effect of Army service on suicide risk. However, most research to date considers only age and gender when comparing Army suicide risk with the general population (Watkins et al., 2018; Reimann and Mazuchowski, 2018). If the goal of the comparison is to determine whether suicide risk is higher or lower for soldiers than it would have been if they had not been in the Army, it is critical to control for the fact that individuals who chose to join the Army look substantially different on these core demographic differences from the general population, even before they joined. However, even matching on a full set of demographic variables could be misleading given the number of unmeasured factors on which Army servicemembers differ from the general U.S. population. For example, the kinds of people who join the Army may be more “psychologically resilient” than individuals in the general U.S. population but may face much higher stresses than individuals in the general population, resulting in the adjusted suicide rates (for the measured covariates we standardized on) being the same for soldiers as the general U.S. population even though the suicide rate of soldiers would have been lower than for individuals in the general population in the absence of the unadjusted special stressors to which Army personnel are exposed.
In contrast, controlling for the characteristics in the third set needs to be carefully considered because matching on these types of characteristics would dramatically alter how one interprets any differences between the Army and the matched general population suicide rates. For example, if the goal of an analysis is to estimate the effect of Army service on suicide risk, including such controls as access to firearms and mental health should be avoided. This is because these types of characteristics represent the specific mechanism by which Army service could affect suicide risk (e.g., through availability of military firearms or mental health problems due to military trauma). An analysis that would attempt to match on these types of factors would obscure the fact that suicide risk is higher or lower for soldiers specifically because they joined the Army and gained access to these mechanisms. Any differences in suicide rates between the Army and matched general population group in these types of analyses should therefore be interpreted cautiously. However, there may be some research purposes for which one does want to control for such factors. For example, if one wants to examine suicide differences between the Army and the general population for a subgroup of individuals with specific mental health diagnoses or to understand how much of the effect of Army service on suicide risk is mediated through access to military firearms, then it may be important to match the populations on these factors.
Characteristics in the second set also warrant careful consideration before inclusion in the standardization process. The decision to include factors in Set II that might be somewhat influenced by Army service, such as marital status and education, depends primarily on the underlying assumptions about the relationship between Army service and these factors. For example, soldiers are more likely to be married than similarly aged individuals in the general population. One potential explanation for this difference is that the Army attracts individuals who also have an interest in getting married (perhaps they are more religious or socially conservative than the general population, or they have a greater interest in having children). If this is the correct theory, marital status should be included in the matched comparisons between the Army and the general population in the same way as variables in Set I. Alternatively, if the higher marriage rate among soldiers reflects Department of Defense (DoD) policies designed to promote marriage under the belief that marriage makes for healthier soldiers, then matching on marital status will produce a comparison between Army and the general population that ignores the effect of Army policies designed to promote soldier well-being. Under this second type of theory, one should avoid matching on this factor when trying to estimate the effect of Army service on suicide risk in the same way that they should be careful with variables in the third set of factors. In short, there are some characteristics for which it is not completely clear whether they should or should not be controlled for in any comparison between soldiers and the general population. Those decisions will need to be informed by the researchers' and Army's theory about the relationship of such factors to Army service and the specific goals of the analysis.
In this study, we explored the various characteristics included in these three sets in more detail. For many characteristics of interest, data are lacking that would allow for a full exploration of the implications of matching on factors that are included in Sets II and III. Nonetheless, we identified six factors that are related to suicide in the general population and/or the Army, that differ in frequency between the two populations, and that have data available for comparing the Army and general population suicide population. These “matchable factors” are gender, age, time, race/ethnicity, marital status, and educational attainment. We explored the impact of including each factor in comparisons that standardize a subset of the general population to look like the Army as well as the conceptual implications of different sets of “matchable” factors.
Several databases are available to examine suicide risk in the general population. Our goal was to select one that was representative of the U.S. population, or a subset of it, and that included an expanded set of factors with which to match to the Army sample. In terms of suicides themselves, the National Violent Death Reporting System (NVDRS) is the only available state-based reporting system that pools data from multiple sources into a usable, publicly available database on violent deaths. However, the NVDRS contains only detailed data on suicides. To establish whether marital status, education level, or other key factors are associated with suicide risk among the general U.S. population, we also needed information on whether those characteristics are over- or underrepresented among the suicide cases as compared with the more general population from which the suicide cases arose. Therefore, we merged the suicide cases from the NVDRS with the Current Population Survey (CPS), a general population database that contains representative data of the general populations living in the states included in the NVDRS in each year. The CPS is a nationally representative and state-by-state representative survey providing high-quality information about the characteristics of the general U.S. population overall and the population of the NVDRS states.
A current limitation of the NVDRS is that it contains only suicide information on a subset of states during our study period. Currently the NVDRS contains data on just 27 states, excluding some of the most populous states in the country. Despite this limitation, we opted to use the NVDRS because of the rich covariate information given on each suicide case. Future plans include expanding the NVDRS to 40 states that will allow future work replicating our methods to have greater generalizability than the results we report. We note that we subset the CPS to only those states included in the NVDRS in each year. Thus, the combined data sets (our NVDRS-CPS sample) provide us with usable information on marriage and educational categories for those who died by suicide and for those who did not, a necessary condition for estimating suicide risk. Because geographic differences exist in state suicide rates, suicides in the NVDRS-CPS sample are likely to differ in systematic ways from suicides nationally, so the NVDRS-CPD sample used in this report is not representative of the entire general U.S. population. As a result, a comparison of Army rates to the NVDRS-CPS sample drawn from just the states participating in the NVDRS may not be used to understand how risk in the Army differs from the risk typically experienced nationally. We note that the subset of NVDRS states have slightly higher suicide rates than the general U.S. population. Nonetheless, we believe the lessons learned on different sets of matching factors using the NVDRS-CPS sample provide meaningful information for future efforts that match the Army to the general population since NVDRS will be expanding to more states and both NVDRS and CPS offer rich data sources for future comparisons between the Army and the general U.S. population.
Figure 1 illustrates how using different factors to standardize our NVDRS-CPS sample to the Army in a given calendar year affects the implications one might draw from comparing Army suicide rates with the general U.S. population. First, as is well known, both populations have experienced an increase in suicide rates since 2003, with the scale of the increase being larger in the Army. When adjusting for only age and gender, the Army suicide rate is significantly lower than the NVDRS-CPS suicide rate before 2008. After 2008, the confidence bands for these two curves (Army versus NVDRS-CPS, adjusted for gender and age) are largely overlapping, suggesting little difference between the two populations within each year.
Figure 1 Army and Civilian Suicide Rates for Unweighted NVDRS-CPS, NVDRS-CPS Weighted Using Standard Factors (Age Plus Gender), and NVDRS-CPS Weighted Using Augmented Factors (Age, Gender, Race/Ethnicity, Marital Status, and Educational Attainment)
When we expand the standardization to include race/ethnicity, education, and marital status, the “expected” suicide rate in the weighted NVDRS-CPS sample is consistently lower in each calendar year than in the NVDRS-CPS curve that used only age and gender in the adjustment. This conceptually makes sense since we have expanded the characteristics on which we want to make our NVDRS-CPS sample look like the Army, including characteristics that might themselves be impacted by military service like education and marriage. In this fully adjusted analysis, the Army again has significantly lower rates in 2003, 2004, and 2005 than the weighted NVDRS-CPS sample. Confidence bands for the two curves generally overlap after 2005 (except in 2012), though the Army consistently has higher rates of suicide than the fully adjusted NVDRS-CPS sample. This suggests potential evidence of higher average rates in Army if one were to test across years rather than within years, as shown in the graphic.
In our analysis, we also identified five additional (unmatchable) factors—geography, parenthood, occupation, mental health, and firearm availability. These could be important when comparing Army with the general U.S. population depending on the type of question being addressed, with the needed caveats described earlier. However, we lacked the data needed to include these factors in our analyses.
We offer four recommendations, based on our assessment of data availability and our analysis of how different weighting factors for the general population affect the comparison between Army and NVDRS-CPS suicide rates.
Given that comparisons will be made between the Army's suicide rate and that of the general population, those comparisons should adjust for age, gender, and year, and for the additional matchable factors of race/ethnicity, educational attainment, and marital status.
As noted, accounting for factors such as race/ethnicity, educational attainment, and marital status notably shifted the estimated suicide rate for the NVDRS-CPS population in large part to changing the underlying Army population characteristics to which we are trying match the general population, affecting the conclusions one might draw from the Army-civilian comparison. As noted above, the decision to include marital status and education depends primarily on the underlying assumed theory about the relationship between Army service and these factors. If the theory is that the Army attracts individuals based on their education levels and marital status/aspirations, then both should be included in the matched comparisons between the Army and the general population. If education and/or marital status is being driven to change by DoD policies, it would be best not to include them directly in the matching because their inclusion might obscure the impact of serving in the Army on suicide risk.
The Army should collaborate with the U.S. Census Bureau, the Centers for Disease Control and Prevention (CDC), and the U.S. Department of Labor to improve occupation/industry coding for general population deaths.
A soldier's job-related duties and operational tempo (“unmatchable” factors) are other factors that may distinguish the Army from general populations. However, we were unable to draw parallels between general population and Army job categories due to limitations in how occupation is coded in the mortality data available on the general population. Given that occupation is a known risk factor for suicide in both populations, better quality data on the general U.S. population would be useful to obtain. A collaboration between the Army, Census Bureau, CDC, and Department of Labor could increase the priority assigned to more accurate coding of occupation and industry in death records for the general population. Additionally, extensive work would be needed to decide how to determine which general population occupations best align with military occupations. Preliminary work in this area has been done by Wenger et al., 2017, and could be used as a basis for this work.
The Army should collect voluntary data on soldiers who own personal firearms and should encourage the CDC or another federal agency to resume collecting voluntarily provided survey data on gun ownership and use in the general population.
Soldiers may differ from their general population counterparts regarding ownership of or access to personally owned firearms, the suicide method used in the majority of Army suicides. Adjusting for this factor may also be important for making comparisons between the Army and general population. As noted, this factor falls into our third set of characteristics for which careful consideration is warranted before inclusion in the standardization process. If the goal of an analysis is to estimate the effect of Army service on suicide risk, including such controls as access to firearms should be avoided because this characteristic represents the specific mechanism by which Army service affects suicide risk (e.g., through availability of military firearms). However, there may be some research purposes for which one does want to control for firearm access, for example, to directly study how much of the effect of Army service on suicide risk is mediated through access to military firearms. Unfortunately, high-quality data in both the general population and the Army is fundamentally lacking. The lack of data on personally owned firearms among soldiers and the general populations impedes the Army's ability to adjust for or study a potentially important factor that may distinguish soldiers from members of the general population and that is correlated with suicide.
Future research should examine the suicide risk among those with mental health diagnoses in the Army relative to similar individuals in the general U.S. population.
The Army and general U.S. population may differ with respect to mental health conditions, which are among the strongest risk factors for suicide. For example, the 2014 Army Study to Assess Risk and Resilience in Servicemembers (STARRS) showed that lifetime prevalence estimates of a variety of mental disorders were significantly higher among new soldiers who were surveyed during their first few days after reporting for duty than similarly matched individuals from the U.S. population on age, gender, education, and race/ethnicity in 2011–2012. This highlights the underlying challenge in comparing the Army's suicide rate with the general U.S. population in that the kinds of people who joined the Army are generally at substantially higher risk of suicide related to history of mental disorders than similar individuals from the general U.S. population who could have, but did not, enlist. Even if these types of differences do not extrapolate to all years (e.g., it might also be that all those new soldiers with high burden of prior psychopathology never made it past their first year of service and did not contribute to the high Army suicide rate), there is a great need to be able to match the two populations on mental health to be able to better understand the role mental health plays in suicide rates and the comparison of rates between the Army and the general U.S. population. Data deriving from medical claims may be most easily linked to death data and have detailed information on mental health diagnoses and thus may be the most fruitful avenue for future research. The Army could replicate the methods used in this study to examine the rate of suicide among those with mental health diagnoses in the Army relative to individuals in the general population with the same diagnoses, adjusting for the sociodemographic characteristics described above. Such research will likely require partnership with an existing health system or data system like the National Inpatient System that not only reports mental health diagnoses within an insured population but also links mental health information to cause of death data. To do this, data on diagnoses will be needed not just for suicide cases but also for the entire Army and general U.S. populations at risk. Additionally, care will need to be taken to address any differences in general population and Army/military health systems regarding coding of psychiatric diagnoses.
