Readiness of Soldiers and Adult Family Members Who Receive Behavioral Health Care

Identifying Promising Outcome Metrics

by Kimberly A. Hepner, Carol P. Roth, Heather Krull, Lea Xenakis, Harold Alan Pincus

This Article

RAND Health Quarterly, 2022; 9(3):19

Abstract

Behavioral health (BH) conditions—such as posttraumatic stress disorder, depression, and anxiety—are the second most common medical reasons for nondeployability in the U.S. Army. The authors of this report aimed to identify promising metrics to assess readiness among soldiers and adult family members who receive BH care. These metrics would expand the Army's outcome monitoring, which currently includes symptom improvement metrics, for patients who received BH care. The authors developed rigorous criteria to evaluate candidate readiness metrics, conducted interviews with stakeholders (Army subject-matter experts and BH providers), reviewed existing sources of data that could support the development of a readiness metric, and conducted a literature review to identify instruments that have been used to measure readiness-related domains in both military and civilian populations. The authors found that no existing data source or patient self-report instrument met criteria for implementation of a readiness metric for soldiers, but one instrument, the Walter Reed Functional Impairment Scale (WRFIS), is promising. No existing data source or patient self-report instrument met criteria for Army-wide implementation of a readiness metric for adult family members. Stakeholders reported that psychiatric symptoms, diagnosis, treatment, and impaired functioning are important indicators of lack of readiness among soldiers and adult family members. BH providers reported variability in assessing readiness and applying profiles, but behavioral experts provided suggestions for improving readiness assessment. The authors recommend that the Army conduct a pilot evaluation of a soldier readiness metric based on the WRFIS and increase standardization in applying profiles by continuing provider training.

For more information, see RAND RR-4268-A at https://www.rand.org/pubs/research_reports/RR4268.html

Full Text

Behavioral health (BH) conditions, such as posttraumatic stress disorder (PTSD), depression, and anxiety, are the second most common medical reason for nondeployability in the U.S. Army. Force readiness is tied not only to individual soldiers’ health but also to the well-being of their families. For these reasons, the health and readiness of soldiers and their families is a top concern for the Army, as is ensuring that soldiers and their families have access to high-quality, effective BH care.

A chief strategy that the Army uses to monitor and improve BH treatment is to assess changes in psychological symptoms experienced over the course of BH specialty care. It does so by tracking the proportion of patients who have experienced clinically significant decreases in symptoms. Although such symptom-based metrics are useful, metrics that specifically assess soldier and family readiness could further strengthen the Army's awareness of soldiers’ BH treatment outcomes and their readiness to deploy.

To support its efforts to monitor and improve BH care, the Army asked RAND Arroyo Center, part of the RAND Corporation, to identify promising metrics to assess readiness among soldiers and adult family members who receive BH care. Ideally, such metrics would capture whether soldiers and their adult family members returned to a high level of functioning following treatment. Thus, the goal of these metrics would be to expand the Army's outcome monitoring efforts beyond symptoms to include one or more readiness-related metrics, providing additional information to assess and improve the effectiveness of BH care.

Although this project was conducted for the Army, the findings and recommendations will likely be of interest to a broader audience across service branches and the military medical community.

Assessing the Behavioral Health Needs of Soldiers and Their Families and Identifying Metrics to Monitor Readiness

BH diagnoses are common across the U.S. military. Among all active-duty service members, 20 percent received a BH diagnosis through the Military Health System (MHS) in fiscal year (FY) 2016, with the highest within-service-branch proportion being in the Army (26 percent compared with 15–18 percent for other service branches). This pattern was similar for rates of anxiety, depression, and PTSD diagnoses (Deployment Health Clinical Center, 2017).

Deployments and postdeployment reintegration can be stressful for both service members and their families. A recent survey of 9,845 married couples across service branches found that 36 percent of spouses met criteria for at least one psychiatric condition (Steenkamp et al., 2018). In a convenience sample of 171 Army and Marine Corps families, 15–25 percent of spouses experienced symptoms of stress, anxiety, and depression during their service member's deployment; 10–18 percent of spouses whose service member had recently returned from a deployment experienced these symptoms (Lester et al., 2010). Although spouses were more willing to seek help for their BH problems than soldiers, their main source of BH care was their on-post primary care clinic rather than a BH specialty care provider (Eaton et al., 2008).

A core component of soldier readiness is the ability to deploy. Soldiers who have a temporary or permanent medical condition that may render them not medically ready to deploy are put on a profile. In 2017, the Army issued a memorandum to standardize the application of profiles for BH conditions to appropriately inform commanders of duty limitations and treatment support recommendations. The memorandum instructs health care providers to assess soldiers’ medical readiness for duty during every clinical encounter.

There are a variety of definitions of soldier and family readiness. For the purpose of this project, we defined these concepts as follows:

  • soldier readiness: the ability to perform mission-essential tasks and deploy without limitations from a BH condition
  • family readiness: the state of being prepared to effectively navigate the challenges of daily living experienced in the unique context of military service, to include mobility and financial readiness, mobilization and deployment readiness, and personal and family life readiness.

Although the U.S. Department of Defense (DoD) definition of family readiness could apply to children, adolescents, and the family as a whole, this study considered only adult family members who receive BH care. The Army aims to provide timely access to high-quality BH care for both soldiers and their families. Monitoring symptom improvement provides valuable information about the effectiveness of Army BH care and the proportion of patients whose symptoms improve. Yet existing metrics do not fully capture whether soldiers and adult family members achieve readiness after receiving BH care. Incorporating one or more readiness metrics into its outcome monitoring process would provide a clearer picture of readiness levels for those who receive Army BH care.

It is important to select metrics that are reliable, directly relevant to the domain of care that is being measured, and sensitive to changes in care. Choosing the wrong metrics could lead to costly implementation and results that are not useful. However, even when an appropriate outcome metric has been selected, implementation could be associated with an increased burden for staff who must administer and incorporate the metric into the care provided and for patients who may need to spend time completing self-report measures. For these reasons, metrics that are intended to drive quality improvement have associated costs that need to be balanced with the potential gains.

Project Methods

We developed a set of criteria to use as a basis for evaluating candidate readiness metrics. These criteria were adapted from those used by the National Quality Forum to evaluate quality measures submitted for its potential endorsement (National Quality Forum, 2018). Characteristics of a desirable readiness metric include the following:

  • Importance. Does the metric address a high-priority area in which performance could improve?
  • Feasibility. Do the data exist to measure metric performance? Are those data accessible without undue burden?
  • Scientific acceptability. Is the metric reliable and valid in measuring the area of interest? Is it sensitive to change?
  • Usability. Do providers view the metric as useful and informative in assessing BH care?

To inform our assessment, we conducted interviews with stakeholders, reviewed the range of existing sources of data that could support the development of a readiness metric, and conducted an instrument search to identify and evaluate instruments that have been used to measure readiness-related domains in both military and civilian populations.

Stakeholder Interviews

To obtain stakeholder perspectives on soldier and family readiness, we conducted interviews with Army BH subject-matter experts and Army BH providers. We solicited the perspectives of these experts and providers on conceptual indicators of readiness. These perspectives provided important information about which aspects of readiness should be captured in a readiness metric. We also gained insight into how BH providers currently assess readiness and solicited suggestions from both BH experts and providers on how readiness assessment could be improved.

Eligible Army BH experts included Army personnel who had participated in designing policies around readiness, assessed BH care quality or outcomes, been involved in designing systems that capture information to assess soldier or family readiness, or participated in BH quality monitoring and improvement activities. Army BH providers were eligible to participate if they delivered BH treatment to soldiers or family members at an Army military treatment facility (MTF). Providers could have worked in a variety of BH settings, such as Embedded Behavioral Health and BH specialty clinics. Eligible provider types included psychiatrists, psychologists, master's-level clinicians, and counselors. In total, we interviewed 18 BH experts and 24 BH providers. For both groups, participation was optional. The Army provided a list of potential BH experts, and installation directors of BH were tasked with identifying potential BH provider participants.

Review of Existing Data Sources

We reviewed existing data sources to determine whether data elements that are currently collected and housed in military systems could support a soldier or family member readiness metric. We considered the content of each data source and its direct applicability to measuring readiness. For each data source, we considered the appropriateness of the variables for potential use in a readiness metric.

The Army currently monitors the BH care provided to soldiers in MTFs using data from the Behavioral Health Data Portal (BHDP). Most BHDP instruments assess symptomology that may affect readiness and inform treatment, but they do not address the broader construct of readiness. However, we identified the Walter Reed Functional Impairment Scale (WRFIS) as a promising instrument to use as the basis for developing a soldier readiness metric. The WRFIS was also suggested by two of the BH experts we interviewed.

A key benefit of the instrument is that it was developed specifically for use with active-duty military personnel, a population that faces unique occupational physical demands and opportunities for exposure to stressful events (see Herrell et al., 2014). Table 1 shows a step-by-step approach to pilot testing a WRFIS-based readiness metric.

Table 1. Plan for Pilot Testing a WRFIS-Based Readiness Metric

Step Description Goal
1 Assess current extent to which the WRFIS is used for Army BH care Determine the level of effort that will be needed to reach full implementation
2 Conduct descriptive analyses of existing WRFIS data Evaluate psychometric properties of the WRFIS among soldiers who receive BH care and inform outcome metric specifications
3 Define detailed metric specifications Develop detailed outcome metric specifications to increase the likelihood that the metric meets evaluation criteria (e.g., scientific acceptability)
4 Pilot test metric Assess whether the metric provides useful data on readiness and can be used to monitor and improve BH care

A second potential source on which to base a readiness metric is eProfile, the data system that the Army uses to track soldiers who have a temporary or permanent medical condition that may make them medically unready for a deployment. There are policies and guidance on when and how to assign a soldier a temporary or permanent profile, including the recently implemented Behavioral Health Readiness Evaluation and Decisionmaking Instrument (B-REDI), a decision support tool designed to help providers determine whether to write a profile for a soldier for a BH condition. Nonetheless, our interview respondents reported a lack of consistency in how profiles are applied, something that could limit the utility of profile data as a source for a readiness metric. Potential inconsistency also points to a need for continued training and decision support for providers on when and how to place a soldier on a profile.

Instrument Search

We conducted a search of the peer-reviewed literature to identify instruments that have been used to assess the readiness of soldiers or adult family members. There are many components that affect readiness (e.g., social support, financial constraints, family issues, mental and physical health), so we aimed to identify a single instrument that would support a readiness metric capable of assessing multiple components of readiness. Our search focused on instruments that could capture the unique demands on soldiers and their families.

To remain in consideration, an instrument needed to assess readiness, be self-report, and contain no more than 30 items. Few instruments met all of these standards. We identified two potentially promising instruments that measured resilience, but both instruments were proprietary tools. Ultimately, we found that none of the instruments in the literature that had been used with military populations or their families met our final inclusion criteria for further consideration.

We also conducted a supplemental search for recommended instruments that measure functioning—a concept related to readiness—in the civilian population. Despite identifying three potentially useful instruments that assess health status and disability in both clinical and population settings, we determined that they were not as relevant to soldiers and their families, who are younger, on average, than the civilian population and face unique stressors. Therefore, none of the instruments identified in our supplemental search met final inclusion criteria for further consideration.

Findings

There are several strengths associated with the methods we employed, but our project also had some limitations. First, we focused only on the readiness of soldiers and adult family members. The readiness of children and adolescents and the family as a unit is also a key part of overall family readiness. Second, policies related to readiness vary across service branches. Our project was limited to Army policies, personnel, and families, and our interviews were limited to Army BH experts and providers. Therefore, our findings may not be applicable to other service branches. Third, we conducted a limited number of interviews, and few respondents had experience working with adult family members. Fourth, we were unable to include planned interviews with commanders, and therefore that perspective is lacking. Further, the perspectives of soldiers and adult family members were not included. Finally, because readiness represents a unique type of functioning in the military community, we applied strictly defined criteria to guide our instrument search. Therefore, we may have missed some sources that may have warranted consideration.

Stakeholders Reported That Psychiatric Symptoms, Diagnoses, Treatment, and Impaired Functioning Are Important Indicators of Lack of Readiness for Soldiers and Adult Family Members

When asked about conceptual indicators of readiness, BH experts and BH providers cited psychiatric symptoms or diagnoses and related treatment as indicators of not being ready for both soldiers and adult family members. These included psychiatric symptoms related to risk (e.g., suicidal ideation) and specific BH diagnoses (e.g., PTSD, depression, substance use disorder) with high symptom severity, need for intensive psychiatric treatment, or treatment with certain psychotropic medications. These responses reflected the Army's current policies regarding BH conditions and their potential negative impact on readiness. Findings from our stakeholder interviews highlighted the importance of ongoing symptom monitoring as a key component of monitoring readiness for soldiers and adult family members.

Input regarding conceptual indicators of readiness also highlighted the important role of multiple aspects of functioning, including occupational functioning (particularly for soldiers), social functioning, and general functioning (particularly for family members) related to readiness. Although there are numerous instruments that assess functioning in general populations, in the case of the Army, the challenge of identifying a metric of functioning is complicated by the unique demands the military places on soldiers and their adult family members.

No Existing Data Source or Patient Self-Report Instrument Met Criteria for Army-Wide Implementation of a Readiness Metric for Soldiers, but One Instrument Is Promising

We used rigorous criteria to evaluate whether any existing data sources would support a readiness metric for soldiers and/or adult family members. The WRFIS assesses important components of soldier readiness, is feasible to use, and appears valid and reliable (based on the psychometric properties of the original form). Unfortunately, there are no data about its sensitivity to change (i.e., whether scores improve over time in response to effective treatment) or its usability as a clinical tool for providers to inform care for individual patients. Therefore, some additional work is needed before it could be implemented on a large scale. Once these parameters are established, a pilot would be a useful way to test whether the WRFIS could support a soldier readiness metric.

No Existing Data Source or Patient Self-Report Instrument Met Criteria for Army-Wide Implementation of a Readiness Metric for Adult Family Members

Neither the WRFIS nor any other data source or instrument we evaluated would support a readiness metric for adult family members. The demands of military readiness on families are unique compared with the functional demands of civilian populations. Instruments to measure functioning in general populations tend to focus on self-care and activity limitations related to physical and mental health problems, making them less relevant to military adult family members who are likely to be young and in generally good health. In addition, the development of instruments to measure overall functioning, including in the context of BH care, is still evolving. Therefore, we found no existing self-report instrument to recommend for Army-wide implementation to monitor family readiness at this time.

BH Providers Reported Some Variability in Assessing Readiness, but BH Experts and Providers Offered Suggestions for Improving Readiness Assessment

BH providers reported using similar information to assess readiness, including occu­pational functioning, policy guidance, and such collateral information as consulting with the soldier's command and, to a lesser extent, friends and family. The most common information used to assess readiness, however, was patient self-report measures and clinical interviews, which providers use to assess the soldier's clinical presentation, determine diagnoses, and inform the treatment plan. Responses were more variable as to whether a soldier's specific duties informed the readiness decision and whether to place the soldier on a profile.

BH experts and BH providers had many suggestions for improving how readiness is assessed. Two specific ideas had the most support: BHDP should be improved and expanded to include additional measures and information about soldiers, and more information should be collected about family members.

Recommendations and Policy Implications

Recommendation 1. Conduct a Pilot Evaluation of a WRFIS-Based Soldier Readiness Metric

We found the WRFIS to be a viable option for further work to develop a soldier readiness metric. The WRFIS's specificity to soldier readiness and feasibility of use (brief length, currently available in BHDP, and accessible to providers and Defense Health Agency leadership) are notable strengths. Although the WRFIS is currently included in BHDP, we do not have current information about its level of use by providers or its clinical utility. We also did not identify an alternative instrument that was suitable for soldiers and had adequate psychometric properties. We recommend that the Army systematically test the use of the WRFIS. These analyses would guide the development of a potential metric to monitor soldier readiness that could be pilot tested in a defined population of those who receive Army BH care.

Recommendation 2. Increase Standardization in Applying Profiles and Continue B-REDI Training

Profiles are used to communicate that a soldier has a medical condition that limits their ability to perform job-related duties. Therefore, we initially believed that profile data could potentially be used to develop a metric that assesses and tracks soldier readiness. However, we learned during interviews that there are issues with how BH profiles are applied. We recommend that, before the Army considers using profile data to develop a readiness metric, steps should be taken to ensure that profiles are applied consistently across providers. Providers should receive additional training on when and how to place a soldier on a profile and continue provider decision support efforts. The B-REDI tool and associated training are an excellent example of the Army's efforts to standardize the application of profiles, and this effort should be continued.

Directions for Future Research

We identified several areas that could be addressed in future research, including the development of an instrument to assess readiness for adult family members. Another research direction would be to assess the utility of profile data, including the reliability and validity of these data and how the presence of a profile predicts soldier service and BH outcomes. In addition, it would be useful to identify an approach to assessing the readiness of service members across service branches who receive BH care from the MHS. Lastly, efforts should be made to capture stakeholder perspectives that were not represented in this study, specifically commanders and those who receive BH care.

References

Deployment Health Clinical Center, Mental Health Disorder Prevalence Among Active Duty Service Members in the Military Health System, Fiscal Years 2005–2016, Falls Church, Va.: Deployment Health Clinical Center, Psychological Health Center of Excellence, 2017. As of August 2, 2019:
https://www.pdhealth.mil/sites/default/files/images/mental-health-disorder-prevalence-among-active-duty-service-members-508.pdf

Eaton, Karen M., Charles W. Hoge, Stephen C. Messer, Allison A. Whitt, Oscar A. Cabrera, Dennis McGurk, Cox Anthony, and Carl A. Castro, "Prevalence of Mental Health Problems, Treatment Need, and Barriers to Care Among Primary Care–Seeking Spouses of Military Service Members Involved in Iraq and Afghanistan Deployments," Military Medicine, Vol. 173, No. 11, 2008, pp. 1051–1056.

Herrell, Richard K., Edward N. Edens, Lyndon A. Riviere, Jeffrey L. Thomas, Paul D. Bliese, and Charles W. Hoge, "Assessing Functional Impairment in a Working Military Population: The Walter Reed Functional Impairment Scale," Psychological Services, Vol. 11, No. 3, 2014, pp. 254.

Lester, Patricia, Kris Peterson, James Reeves, Larry Knauss, Dorie Glover, Catherine Mogil, Naihua Duan, William Saltzman, Robert Pynoos, and Katherine Wilt, "The Long War and Parental Combat Deployment: Effects on Military Children and At-Home Spouses," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 49, No. 4, 2010, pp. 310–320.

National Quality Forum, "The ABCs of Measurement," Washington, D.C.: National Quality Forum, 2018. As of August 5, 2019:
https://www.qualityforum.org/Measuring_Performance/ABCs_of_Measurement.aspx

Steenkamp, Maria M., Nida H. Corry, Meng Qian, Li Meng, Hope Seib McMaster, John A. Fairbank, Valerie A. Stander, Laura Hollahan, and Charles R. Marmar, "Prevalence of Psychiatric Morbidity in United States Military Spouses: The Millennium Cohort Family Study," Depression and Anxiety, Vol. 35, No. 9, 2018, pp. 815–829.

The research reported here was completed in February 2020, followed by security review by the sponsor and the Office of the Chief of Public Affairs, with final sign-off in August 2021. The research described in this article was sponsored by the United States Army and conducted by the Personnel, Training, and Health Program within RAND Arroyo Center.

RAND Health Quarterly is produced by the RAND Corporation. ISSN 2162-8254.