The RAND Online Measure Repository for Evaluating Psychological Health and Traumatic Brain Injury Programs

More than 2.2 million service members have deployed to support military operations in Iraq and Afghanistan during the past decade (Levin, 2011). Among service members who had been deployed to Iraq and Afghanistan as of October 2007, approximately one-fifth reported current symptoms consistent with posttraumatic stress disorder (PTSD) or major depression, and about the same number reported having experienced a probable traumatic brain injury (TBI) while deployed (Tanielian and Jaycox, 2008). DoD has implemented numerous programs, interventions, and policies to address the increased concerns about PTSD and TBI and their effects on service members and their families. These programs focus on reducing the incidence of mental health problems via efforts to improve readiness and resilience; providing information, connecting individuals to care, and encouraging help seeking; identifying individuals with mental health concerns or TBI; providing or improving clinical services, or offering mental health services in nontraditional locations to expand access to care; providing a wide range of training and educational activities; and supporting service members and their families during times of military transition (Weinick et al., 2011).

As these efforts have proliferated, it has become increasingly important to evaluate their effectiveness. To support the design and implementation of such program evaluations, we developed an online repository of measures (the RAND Online Measures Repository, or ROMR) that indexes and describes measures related to psychological health and TBI that have been used in both civilian and military populations. Specifically, we identified measures of primary importance to TBI including measures of cognition, executive functioning, and memory. We have also identified measures of primary importance to psychological health in the following domains: PTSD, depression, anxiety, suicidal ideation, and resiliency. We also identified measures relevant to military units such as unit cohesion and force readiness and preservation.

The ROMR is the second part of a toolkit that RAND is developing to support the assessment and evaluation of the DoD portfolio of programs. A link to the ROMR is available at the “Innovative Practices for Psychological Health and Traumatic Brain Injury” web page;* the other tools in this series are also available at this site.

How the RAND Online Measure Repository Was Developed

The ROMR was developed using a series of literature searches, journal reviews, and expert recommendations to identify measures of anxiety, depression, PTSD, resiliency, suicidal thoughts, unit cohesion, force readiness and preservation, and measures related to TBI. Relevant articles were coded using a standardized abstraction procedure guided by supporting documents (e.g., glossary of terms) and procedures (weekly discussion of coding issues). We focused on sources that described the development, validation, and/or psychometric properties of one or more measures. From each source, we abstracted information about the measure's domains, administration, scoring, length, acquisition, and psychometric properties, as well as identified the populations to which the measure had been applied. Once information on measures was abstracted and reviewed, the database used for coding was converted into a searchable online tool.

Measures Included in the RAND Online Measure Repository

We identified 174 measures including a wide array of measures of depression (71), PTSD (49), and anxiety (41). Several measures related to exposure to traumatic events (21), stress and coping (16), resiliency (15), suicidal thoughts (16), and TBI (e.g., cognition functioning and speech) (20) were also identified. Fewer measures of force readiness (4) and unit cohesion (10) were identified, suggesting that this may be a less developed field of measurement. Eighty-four percent of the measures identified had been used with adults, and 23 percent had been used with children. The majority of measures were self-administered questionnaires. It is important to note that the purpose of the ROMR is to support program evaluation, rather than clinical care, so we have not included measures related to diagnosis of mental health disorders or TBI.

The measures most commonly used with military populations were those related to depression and PTSD. However, only about half of the total measures identified had ever been used with a military population. Additional work is needed to validate many of these measures in military populations, especially measures with clinical significance, no-cost measures of anxiety, and measures for evaluation of programs related to TBI. As updates are made to the ROMR, additions may also be considered to continue building areas of the ROMR where fewer measures were identified. These areas include measures of leadership, force readiness, unit cohesion, and family support.

Potential Uses of the RAND Online Measure Repository

The ROMR has a number of potential uses across a wide variety of programs and professionals.

Select Measures for Program Evaluation or Research Related to Psychological Health and Traumatic Brain Injury

The primary purpose of the ROMR is to help program evaluators select appropriate measures for use. Program evaluators, researchers, and those responsible for program implementation can use the repository to identify specific measures across a wide variety of domains related to TBI including cognition, executive functioning, and memory, and psychological health including depression, anxiety, PTSD, stress and coping, and resiliency, among others.

Select Measures for Dual Use by Both Clinicians and Program Evaluators or Researchers

The ROMR includes information on available clinical cutoff scores used to determine when individuals require clinical services, to inform clinical case planning, or to screen individuals who may be at risk for developing a psychological disorder. Measures with clinical meaning may be useful to both clinicians providing novel interventions or other services specifically targeting clinical outcomes and to the evaluators or researchers working with these clinicians to determine the effectiveness of their services.

Identify Core Outcome Measures for Evaluating Similar Programs

Organizations or individuals responsible for a group of programs could consider using the ROMR to identify and endorse a specific set of outcome measures that are both reliable and valid for the populations served across a variety of domains. Endorsing a specific set of outcome measures could allow for consistency in tracking core outcomes or indicators of effectiveness across an array of programs.

Determine Need for Additional Reliability and Validity Testing of Measures with Military Populations

The ROMR's assessment of measures currently used by program evaluators and researchers can be helpful in determining where more work needs to be done to establish reliability and validity of measures with military populations. Additional psychometric development is particularly important for domains such as force readiness, where only a few measures exist, and domains such as anxiety, where there has been little testing with military populations.

Conclusion

Valid and reliable measures of psychological health and TBI-related constructs are needed to be able to monitor and evaluate programs that address these issues. The ROMR is a valuable tool that responds to this need by providing an online and searchable database of measures related to psychological health and TBI.

References

Levin, A., “Major Study Will Assess 'What Works' in PTSD Care,” Psychiatric News, Vol. 46, No. 7, 2011, pp. 4–5.

Tanielian T and Jaycox L, eds., Invisible Wounds of War, Santa Monica, Calif.: RAND Corporation, MG-720, 2008. As of October 19, 2011:
http://www.rand.org/pubs/monographs/MG720.html

Weinick RM, Beckjord EB, Farmer CM, Martin LT, Gillen EM, Acosta JD, Fisher MP, Garnett J, Gonzalez G, Helmus TC, Jaycox LH, Reynolds KA, Salcedo N, and Scharf DM, Programs Addressing Psychological Health and Care for Traumatic Brain Injury Among U.S. Military Servicemembers and Their Families, Santa Monica, Calif.: RAND Corporation, TR-950-OASD, 2011. As of November 22, 2011:
http://www.rand.org/pubs/technical_reports/TR950.html

Note

* http://www.rand.org/multi/military/innovative-practices.html

This research was sponsored by the the Office of the Secretary of Defense (OSD) and the Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury. It was conducted in the Forces and Resources Policy Center, a RAND National Defense Research Institute (NDRI) program. NDRI is a federally funded research and development center sponsored by the OSD, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.