Evaluating the U.S. Military's Progress Toward an Integrated Primary Prevention Workforce: Five-Year Process Evaluation Plan

Issue

Service members can harm others through sexual assault, harassment (e.g., sexual harassment, bullying, hazing, reprisal, retaliation), domestic abuse, child abuse and neglect, and intimate partner abuse and can harm themselves through attempting or dying by suicide (Department of Defense Instruction 6400.09, 2020). These harmful behaviors can have detrimental and long-lasting impacts on service members' physical and mental health, affecting their quality of life and their military careers. To specifically address this issue, the Independent Review Commission on Sexual Assault in the Military recommended that the U.S. Department of Defense (DoD) develop a dedicated prevention workforce (Independent Review Commission on Sexual Assault in the Military, 2021). In response to this recommendation, DoD is hiring more than 2,000 Integrated Primary Prevention (IPP) personnel—individuals with particular knowledge and skills in the conduct of prevention activities. IPP activities aim to stop harmful behaviors before they start by simultaneously targeting multiple risk and protective factors. Much as eating healthy and exercising are individual activities that can help mitigate individual risk for certain diseases, there are environmental and organizational prevention activities that can mitigate the risk that service members will harm themselves or others. The June 13, 2022, memorandum from the Under Secretary of Defense for Personnel and Readiness outlined DoD's new Prevention Workforce Model (Cisneros, 2022), and the model was further codified with the December 20, 2022, publication of Department of Defense Instruction 6400.11, DoD Integrated Primary Prevention Policy for Prevention Workforce and Leaders (Department of Defense Instruction 6400.11, 2022). Across strategic, operational, and tactical levels, the IPP personnel will conduct a variety of these prevention activities, including oversight, research, and direct implementation of data-informed activities to prevent various harmful behaviors that may be experienced by service members.

Approach

DoD asked the RAND National Defense Research Institute to conduct a multiyear process evaluation to assess the progress toward fully implementing an IPP workforce (IPPW). We will use nine complementary and mixed methods to address six aims:

1. Assess how the DoD components' IPP policies and plans are aligned with the requirements set forth by DoD.
2. Assess the hiring progress (e.g., number and type of personnel hired) and qualifications of the IPPW.
3. Describe the structure and functioning of prevention infrastructure and whether the IPPW perceives having sufficient infrastructure support.
4. Assess the elements of leader support for the conduct of prevention (e.g., understanding of IPP, belief that IPP is important and works well), and assess the level of leader support perceived by the IPPW.
5. Assess the quality and comprehensiveness of the Comprehensive IPP (CIPP) plans, what aspects of the plans are implemented as intended, and the facilitators and barriers to their implementation.
6. Describe how the IPPW is building and maintaining prevention teams at the strategic, operational, and tactical levels and how these prevention teams are connected to leaders at each of these levels.

Table 1 presents a matrix of the various methods that will be used to address each of these aims.

In this study, we describe our planned approach to data collection and analysis for each of these methods, a logic model that guides the evaluation, the levels of focus for the evaluation (across DoD, by DoD component, for military organizations, for IPP personnel, etc.), the five-year evaluation timeline, and limitations and next steps for the evaluation.

Limitations

This evaluation faces challenges because of the mission variability and operational tempo of the military population and data availability (particularly for evaluation methods that require response from large numbers of IPP personnel). Additionally, evaluation studies using self-reported data may face some social desirability bias (King and Bruner, 2000)—meaning that some IPP personnel, leaders, and other evaluation participants may respond to evaluation inquiries in a socially desirable manner (i.e., in a way that presents themselves favorably). These challenges can affect our ability to conduct reliable statistical analyses using these data. The evaluation plan specifies strategies to mitigate these challenges as much as possible.

Next Steps

Prior to beginning data collection, the evaluation team must revise some of the previously developed evaluation instruments and acquire all needed regulatory approvals to implement the data collection, including from RAND's institutional review board (the RAND Human Subjects Protection Committee), DoD's second-level review, the Office of Management and Budget, and the U.S. Marine Corps Human Protection Program.

DoD components are still finalizing hiring plans, so the evaluation plan as scoped is subject to change as final hiring plans are released. Similarly, the evaluation plan is also subject to change based on data availability. Despite some uncertainties, the evaluation is poised to document the level of infrastructure and quality achieved by DoD's prevention initiative. Consequently, this study will be useful for

• commanders and other senior-level military leaders and policymakers interested in improving the quality of efforts to prevent harmful behavior in the military
• professionals responsible for addressing a variety of harmful behaviors (e.g., sexual assault, suicide)
• future studies of IPPW impact.