To assist the U.S. Marine Corps in evaluating its sexual assault prevention programs, the authors of this report identify and develop measures of performance and measures of effectiveness with which to assess the programs. The research team created a logic model framework to guide evaluations and mapped program goals to measures that assess the degree to which each outcome has been achieved.
Measures of Performance and Effectiveness for the Marine Corps' Sexual Assault Prevention Programs
Download
Download eBook for Free
| Format | File Size | Notes |
|---|---|---|
| PDF file | 0.9 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Purchase
Purchase Print Copy
| Format | List Price | Price | |
|---|---|---|---|
| Add to Cart | Paperback184 pages | $43.00 | $34.40 20% Web Discount |
Research Questions
- What measures of performance (MOPs) and measures of effectiveness (MOEs) are currently being used to evaluate sexual assault prevention programs?
- What data are currently available to inform evaluation of Marine Corps sexual assault prevention programs?
- What are the process components of Marine Corps sexual assault prevention programs, and what intermediate, proximal, and distal outcomes do the programs seek to achieve?
- How can the Marine Corps best develop an evaluation of its sexual assault prevention programs?
To assist the U.S. Marine Corps in evaluating its sexual assault prevention programs, the authors of this report identify and develop measures of performance (MOPs) and measures of effectiveness (MOEs) with which to assess the programs. The research team created a logic model framework to guide evaluations and mapped program goals to measures that assess the degree to which each outcome has been achieved.
This report presents logic models for the U.S. Marine Corps' sexual assault prevention annual trainings, leadership courses, and social media efforts and identifies MOPs and MOEs for each component of the logic models. Collectively, these measures serve as an item pool from which to select measures for use in future evaluations. As background, the authors also review measures being used for evaluation of sexual assault prevention training in the military and civilian arenas and existing data sources available to the Marine Corps that include variables relevant to an evaluation of sexual assault prevention programming. The authors conclude with considerations for developing an evaluation of the Marine Corps Sexual Assault Prevention and Response office's prevention programming and recommendations to make best use of the measures.
Key Findings
- MOPs are used to evaluate the way in which a program is implemented and the quality of that implementation; MOEs are used to evaluate whether the program is achieving its goals and effectively producing the desired intermediate, proximal, or distal outcomes.
- A systemic review of published evaluations of sexual assault prevention programming identified frequently used MOPs and MOEs. Interviews with subject-matter experts identified recent developments and additional themes.
- The researchers developed logic models, and identified corresponding MOPs and MOEs, for the three main types of Marine Corps sexual assault prevention programming: annual trainings, leadership courses, and social media efforts.
- The researchers developed a tailored pool of MOPs and MOEs for each of the three categories of programs. Each item in the pool includes the item wording, response options, the source of the item, and the population to which the measure applies.
- The lack of existing items and scales that fit Marine Corps processes and desired outcomes resulted in the need for RAND experts to develop items tailored to the content of the Marine Corps' training programs. As these programs continue to evolve, future measures may need to be added to and customized for the item pool.
Recommendations
- Using the logic models in this report as a guide, review training content to determine whether each intermediate, proximal, and distal outcome in the logic model has corresponding program content that theoretically could produce the intended outcome.
- Conduct regular data collection to assess the processes and outcomes associated with Marine Corps sexual assault prevention programming.
- Adopt a continuous quality improvement framework that continually collects the information necessary to incrementally improve programming and respond to organizational, cultural, or policy changes.
- Select priority areas for evaluation.
- Whenever possible and appropriate, leverage existing data systems to reduce costs and personnel time.
- Do not use distal outcomes to evaluate the effectiveness of Marine Corps sexual assault prevention programs. Year-to-year changes in the prevalence of sexual assault cannot meaningfully or statistically support inferences about the program's success or shortcomings.
- Do not use the Defense Sexual Assault Incident Database to evaluate Marine Corps sexual assault prevention programs. The number of sexual assaults reported each year will rise or fall as a function of both negative trends (increased sexual assaults) and positive trends (increased willingness to report), and therefore any change in the number of official reports is uninterpretable as it applies to the effectiveness of Marine Corps programs.
- Evaluate the effectiveness of the annual training courses and leadership courses using process, intermediate outcome, and proximal outcome measures.
- Use the measures of performance and effectiveness identified in this report as item pools from which to select measures that are best matched to a given evaluation.
Table of Contents
Chapter One
Introduction and Methods
Part I
Evaluating Sexual Assault Prevention Programs: Background
Chapter Two
Current Practices in Measurement of Performance and Effectiveness for Sexual Assault Prevention Programs
Chapter Three
Existing Data Sources Relevant to Evaluating Sexual Assault Prevention Efforts
Part II
Identifying Measures of Performance and Effectiveness for Marine Corps Sexual Assault Prevention Programs
Chapter Four
Annual Sexual Assault Prevention Training
Chapter Five
Leadership Courses
Chapter Six
Social Media
Part III
Identifying Measures of Performance and Effectiveness for Marine Corps Sexual Assault Prevention Programs
Chapter Seven
Considerations for Measure Implementation
Chapter Eight
Recommendations and Conclusions
Appendix A
Evaluation of Military Sexual Assault Prevention Programs
Appendix B
Systematic Review Methodology and Included Studies
Appendix C
Measures of Performance and Measures of Effectiveness for Assessing Annual Marine Corps Sexual Assault Prevention Trainings
Appendix D
Measures of Performance and Measures of Effectiveness for Assessing Marine Corps Leadership Courses in Sexual Assault Prevention
Appendix E
Measures of Performance for Marine Corps Sexual Assault Prevention Social Media Posts
Research conducted by
This research was contracted by the U.S. Marine Corps Operations Analysis Directorate on behalf of the sponsor, the Marine and Family Programs Division, and conducted within the Forces and Resources Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.