Resource Process and Outcome Evaluation Measures for Sexual Assault Prevention Activities in the Military
Process Evaluation Measures
| Type of Measure | Measure | Items | Measure Name and Source |
|---|---|---|---|
| 1. Demographics | 1.1 Gender | 1.1.1 Are you a . . . Response options: Man, woman, prefer not to answer | U.S. Census Bureau |
| 1.2 Age | 1.2.1 What is your age? Response options: Number | National Health Promotion Associates, Cadet Healthy Personal Skills (CHiPS): An Adaptation of Botvin Life Skills Training Report, 2018. | |
| 1.3 Ethnicity | 1.3.1 What is your ethnicity? Response options: Hispanic or Latino, non-Hispanic or Latino, prefer not to answer | National Health Promotion Associates, Cadet Healthy Personal Skills (CHiPS): An Adaptation of Botvin Life Skills Training Report, 2018. | |
| 1.4 Race | 1.4.1 What is your race? Response options: American Indian or Alaska Native, Asian, Black or African American, Native Hawaiian or other Pacific Islander, White, other (please specify), prefer not to answer | National Health Promotion Associates, Cadet Healthy Personal Skills (CHiPS): An Adaptation of Botvin Life Skills Training Report, 2018. | |
| 1.5 Pay grade | 1.5.1 What is your pay grade? Response options: E-1–E-4, E-5–E-8, W-1–W-5, O-1–O-3, O-4 and above | ||
| 2.Implementation quality | 2.1 Presenter delivers with fidelity | Ratings on adherence to the prevention activity model or curriculum made by
|
Coreen Farris, Terry L. Schell, Margaret Tankard, Lisa H. Jaycox, Barbara Bicksler, Angela Clague, and Dionne Barnes-Proby, Measures of Performance and Effectiveness for the Marine Corps' Sexual Assault Prevention Programs, Santa Monica, Calif.: RAND Corporation, RR-2220-USMC, 2019. https://www.rand.org/pubs/research_reports/RR2220.html |
| 2.2 Leadership communicated support for the prevention activity | Participant ratings of the extent to which their direct supervisor, leader, and/or commander values the prevention activity process and encouraged them to engage in the prevention activity
|
Coreen Farris, Terry L. Schell, Margaret Tankard, Lisa H. Jaycox, Barbara Bicksler, Angela Clague, and Dionne Barnes-Proby, Measures of Performance and Effectiveness for the Marine Corps' Sexual Assault Prevention Programs, Santa Monica, Calif.: RAND Corporation, RR-2220-USMC, 2019. https://www.rand.org/pubs/research_reports/RR2220.html | |
| 2.3 Participant engagement in the intervention | External observer or trainer estimate of the percentage (or number) of participants who
|
Coreen Farris, Terry L. Schell, Margaret Tankard, Lisa H. Jaycox, Barbara Bicksler, Angela Clague, and Dionne Barnes-Proby, Measures of Performance and Effectiveness for the Marine Corps' Sexual Assault Prevention Programs, Santa Monica, Calif.: RAND Corporation, RR-2220-USMC, 2019. https://www.rand.org/pubs/research_reports/RR2220.html | |
| 2.4 Participant satisfaction | Extent to which participants rated the experience as important, useful, and likely to succeed.
Perceived importance
Perceived personal relevance
Perceived usefulness of the material
Participant satisfaction
|
Coreen Farris, Terry L. Schell, Margaret Tankard, Lisa H. Jaycox, Barbara Bicksler, Angela Clague, and Dionne Barnes-Proby, Measures of Performance and Effectiveness for the Marine Corps' Sexual Assault Prevention Programs, Santa Monica, Calif.: RAND Corporation, RR-2220-USMC, 2019. https://www.rand.org/pubs/research_reports/RR2220.html
* This item is from the 2016 Workplace and Gender Relations Survey of Active Duty Members: Lisa Davis, Amanda Grifka, Kristin Williams, and Margaret Coffey, eds., Military Workplace Climate: 2016 Workplace and Gender Relations Survey of Active Duty Members Overview Report, Alexandria, Va.: U.S. Department of Defense Office of People Analytics, 2017. http://www.sapr.mil/public/docs/reports/FY16_Annual/Annex_1_2016_WGRA_Report.pdf Tanya L. Tompkins and Jody Witt, "The Short-Term Effectiveness of a Suicide Prevention Gatekeeper Training Program in a College Setting with Residence Life Advisers," Journal of Primary Prevention, Vol. 30, No. 2, 2009, pp. 131–149. Kate Hendricks Thomas and Sarah Plummer Taylor, "Bulletproofing the Psyche: Mindfulness Interventions in the Training Environment to Improve Resilience in the Military and Veteran Communities," Advances in Social Work, Vol. 16, No. 2, 2015, pp. 312–322. |
|
| 3.Level of delivery | 3.1 Prevention activity attendance | 3.1.1 How many sessions a participant attended within a given time period
3.1.2How many minutes a participant received of the prevention activity within a given time period |
Jennifer M. Gamarra, Matthew T. Luciano, Jaimie L. Gradus, and Shannon Wiltsey Stirman, "Assessing Variability and Implementation Fidelity of Suicide Prevention Safety Planning in a Regional VA Healthcare System," Crisis, Vol. 36, No. 6, 2015, pp. 433–439. Helen Christensen, Kathy Griffiths, Chloe Groves, and Ailsa Korten, "Free Range Users and One Hit Wonders: Community Users of an Internet-Based Cognitive Behaviour Therapy Program," Australian & New Zealand Journal of Psychiatry, Vol. 40, No. 1, January 2006, pp. 59–62. |
| 3.2 Participant– prevention activity staff interactions | 3.2.1 Number and quality of documented accounts of collaboration between participant and prevention activity facilitator or provider | Jennifer M. Gamarra, Matthew T. Luciano, Jaimie L. Gradus, and Shannon Wiltsey Stirman, "Assessing Variability and Implementation Fidelity of Suicide Prevention Safety Planning in a Regional VA Healthcare System," Crisis, Vol. 36, No. 6, 2015, pp. 433–439. | |
| 3.3 Attrition rate of participants involved in prevention activity | 3.3.1 Number of participants who were still enrolled at the end of the prevention activity divided by the number enrolled at the start of the prevention activity to determine the percentage of attrition | M. Katherine Shear, Charles F. Reynolds, III, Naomi M. Simon, Sidney Zisook, Yuanjia Wang, Christine Mauro, Naihua Duan, Barry Lebowitz, and Natalia Skritskaya, "Optimizing Treatment of Complicated Grief: A Randomized Clinical Trial," JAMA Psychiatry, Vol. 73, No. 7, July 1, 2016, pp. 685–694. | |
| 3.4 Training dosage | 3.4.1 Extent to which prevention activity facilitators and providers received the training—for example, the number of training sessions attended or the number of hours of training received | Takahiro A. Kato, Yuriko Suzuki, Ryoko Sato, Daisuke Fujisawa, Kumi Uehara, Naoki Hashimoto, Yasunori Sawayama, Jun Hayashi, Shigenobu Kanba, and Kotaro Otsuka, "Development of 2-Hour Suicide Intervention Program Among Medical Residents: First Pilot Trial," Psychiatry and Clinical Neurosciences, Vol. 64, No. 5, October 2010, pp. 531–540. |
Outcome Evaluation Measures
Domain 1: Preventing Sexual Harassment and Promoting Workplace Civility
| Type of Measure | Measure | Items | Measure Name and Source |
|---|---|---|---|
| 4.Longer-term outcome measures | 4.1 Workplace civility | Over the past 6 months, how often has each of the following occurred at work:
Response options: (0) Never, (1) now and then, (2) monthly, (3) weekly, (4) daily Sum item scores. Higher score indicates more negative acts. |
Measure: Short-Form Negative Acts Questionnaire
G. Notelaers, B. Van der Heijden, H. Hoel, and S. Einarsen, "Measuring Bullying at Work with the Short-Form Negative Acts Questionnaire: Identification of Targets and Criterion Validity," Work & Stress, Vol. 33, No. 1, 2019, pp. 58–75. |
| 4.2 Unit cohesion |
Response options: Strongly disagree: 1, somewhat disagree: 2, neither agree nor disagree: 3, somewhat agree: 4, strongly agree: 5 Sum item scores. Possible range is 12 to 60; higher scores are indicative of greater perceived social support from fellow unit members and unit leaders. |
Measure: Adapted Unit Support Scale from Deployment Risk and Resilience Inventory–2
D. S. Vogt, B. N. Smith, L. A. King, D. W. King, J. A. Knight, and J. J. Vasterling, "Deployment Risk and Resilience Inventory–2 (DRRI-2): An Updated Tool for Assessing Psychosocial Risk and Resilience Factors Among Service Members and Veterans," Journal of Traumatic Stress, Vol. 26, 2013, pp. 710–717. |
|
| 4.3 Gender discrimination |
Response options: Strongly disagree: 1, strongly agree: 7 Sum item scores. Higher scores indicate higher levels of perceived discrimination experiences. Strong construct, convergent, and discriminant validity for the Workplace Prejudice/Discrimination Inventory have been demonstrated (James, Lovato, and Cropanzano, 1994). |
Adapted Workplace Prejudice/Discrimination Inventory
M. M. Foynes, J. C. Shipherd, and E. F. Harrington, "Race and Gender Discrimination in the Marines," Cultural Diversity and Ethnic Minority Psychology, Vol. 19, No. 1, 2013, p. 111. K. James, C. Lovato, and R. Cropanzano, "Correlational and Known-Group Comparison Validation of a Workplace Prejudice/Discrimination Inventory," Journal of Applied Social Psychology, Vol. 24, No. 17, 1994, pp. 1573–1592. Defense Equal Opportunity Management Institute, "Sexual Assault Prevention Climate," DEOMI Organizational Climate Survey (DEOCS) Assessment to Solutions, 2019. |
|
| 4.4 Sexual Harassment |
|
Measure: RAND Short Form Measure of Sexual Harassment Risk in the Military
Terry L. Schell, Matthew Cefalu, and Andrew R. Morral, Development of a Short Form Measure of Sexual Harassment Risk in the Military: Findings from the RAND Military Workplace Study, Santa Monica, Calif.: RAND Corporation, RR-2031-OSD, 2019. https://www.rand.org/pubs/research_reports/RR2031.html NOTE: Please consult the source for the instructions that should accompany the items, programming notes, and scoring instructions. |
Outcome Evaluation Measures
Domain 2: Preventing Sexual Assault Through Bystander Intervention, Changing Social Norms, Risk Perception, Confidence in Self Defense, and Reducing Alcohol Misuse
| Type of Measure | Measures | Items | Measure Name and Source |
|---|---|---|---|
| 5. Short-term outcomes | 5.1 Positive bystander behavior | How often did you engage in the following behaviors in the last year:
Response options range from 0 to 3 (0 = not at all, 1 = 1–2 times, 2 = 3–5 times, 3 = 6 or more times). Scores are summed for a cumulative score. Higher scores indicate greater number of positive bystander behaviors. |
Measure: Positive Bystander Behavior
Ann L. Coker, Patricia G. Cook-Craig, Corrine M. Williams, Bonnie S. Fisher, Emily R. Clear, Lisandra S. Garcia, and Lea M. Hegge, "Evaluation of Green Dot: An Active Bystander Intervention to Reduce Sexual Violence on College Campuses," Violence Against Women, Vol. 17, No. 6, 2011, pp. 777–796. |
| 5.2 Risk perception |
Response options: 1 = very unlikely, 5 = very likely Scores may be totaled for a cumulative score. Higher scores indicate greater risk perception for assault by an acquaintance.
Response options: A realistic, yet hypothetical story of a sexual assault (either by an acquaintance or a stranger) is presented in 25 statements. Respondents are to identify where in the story they would "leave the situation" (1–25). Score is calculated at the point the person determines they would leave the situation. Lower scores (leaving the situation earlier) indicate greater perception of risk. Average of population assesses overall level of risk. |
Measure: Women's perception of their risk of sexual assault by male acquaintances
Charlene Y. Senn, Misha Eliasziw, Paula C. Barata, Wilfreda E. Thurston, Ian R. Newby-Clark, H. Lorraine Radtke, and Karen L. Hobden, "Sexual Assault Resistance Education for University Women: Study Protocol for a Randomized Controlled Trial (SARE Trial)," BMC Women's Health, Vol. 13, No. 1, 2013, p. 25. Measure: Risk Perception Measure Terri L. Messman-Moore and Amy L. Brown, "Risk Perception, Rape, and Sexual Revictimization: A Prospective Study of College Women," Psychology of Women Quarterly, Vol. 30, No. 2, 2006, pp. 159–172. |
|
| 5.3 Consent beliefs |
Sum responses for score. Lower scores equate to greater awareness of appropriate consent behavior. Response options: 0 = strongly disagree, 1 = disagree, 2 = neither agree nor disagree, 3 = agree, 4 = strongly agree |
Measure: Sexual Consent Scale
National Health Promotion Associates, Cadet Healthy Personal Skills (CHiPS): An Adaptation of Botvin Life Skills Training Report, 2018. Adapted from Terry P. Humphreys and Mélanie M. Brousseau, "The Sexual Consent Scale—Revised: Development, Reliability, and Preliminary Validity," Journal of Sex Research, Vol. 47, No. 5, 2010, pp. 420–428. |
|
| 5.4 Self-defense tactics | How likely are you to use the following tactics in response to an unwanted sexual advance?
Response is a 7-point scale from (1) not at all likely to (7) completely likely. Scores are totaled for a cumulative score. Higher scores indicate greater confidence related to self-defense skills. |
Measure: Resistance Tactics Scale
C. A. Gidycz, L. M. Orchowski, D. R. Probst, K. M. Edwards, M. Murphy, and E. Tansill, "Resistance Tactics Scale," database, 2015. |
|
| 5.5 Alcohol misuse |
Scores are totaled for a cumulative score. Higher scores indicate greater alcohol misuse. |
Measure: The Alcohol Use Disorders Identification Test (AUDIT) by the World Health Organization
Thomas F. Babor, J. R. de la Fuente, J. Saunders, and M. Grant, "The Alcohol Use Disorders Identification Test," Guidelines for Use in Primary Health Care, Geneva: World Health Organization, 1992. |
|
| 5.6 Intent to help |
Response options: 1(not at all likely) to 5 (extremely likely) |
Victoria L. Banyard, Mary M. Moynihan, Alison C. Cares, and Rebecca Warner, "How Do We Know If It Works? Measuring Outcomes in Bystander-Focused Abuse Prevention on Campuses," Psychology of Violence, Vol. 4, No. 1, 2014, pp. 101–115. | |
| 5.7 Bystander Efficacy Short Form | Assess confidence in one's ability to engage in bystander helping behaviors
Response options: 0 (can't do) to 50 (moderately certain) to 100 (very certain) |
V. L. Banyard, "Measurement and Correlates of Pro-Social Bystander Behavior: The Case of Interpersonal Violence," Violence and Victims, Vol. 23, 2008, pp. 83–97. | |
| 5.8 Rape beliefs | Assesses endorsement of beliefs that justify rape or blame victims for their victimization
Subscale 1: She asked for it
Subscale 2: He didn't mean to
Subscale 3: It wasn't really rape
Subscale 4: She lied
|
Measure: Updated Illinois Rape Myth Acceptance Scale
Diana L. Payne, Kimberly A. Lonsway, and Louise F. Fitzgerald, "Rape Myth Acceptance: Exploration of Its Structure and Its Measurement Using the Illinois Rape Myth Acceptance Scale," Journal of Research in Personality, Vol. 33, No. 1, 1999, pp. 27–68. Sarah McMahon and G. Lawrence Farmer, "An Updated Measure for Assessing Subtle Rape Myths," Social Work Research, Vol. 35, No. 2, June 2011, pp. 71–81. Rachel A. Breslin, Lisa Davis, Kimberly Hylton, Ariel Hill, William Klauberg, Mark Petusky, and Ashlea Klahr, 2018 Workplace and Gender Relations of Active Duty Members: Overview Report, Alexandria, Va.: Office of People Analytics, U.S. Department of Defense, OPA Report No. 2019-027, May 2019. |
|
| 5.9 Risk perception measure | This measure presents two common social experiences that end in a sexual assault (one involving a stranger and one involving an acquaintance) and asks when the respondent would leave the situation. Each scenario is broken into 25 chronological steps and the respondent has to choose the step at which they would leave. The full measure can be found at https://journals.sagepub.com/doi/full/10.1111/j.1471-6402.2006.00279.x (Messman-Moore and Brown, 2006). | Terri L. Messman-Moore and Amy L. Brown, "Risk Perception, Rape, and Sexual Revictimization: A Prospective Study of College Women," Psychology of Women Quarterly, Vol. 30, No. 2, 2006, pp. 159–172. |
Additional Outcome Measures
Measures of Performance and Effectiveness for the Marine Corps' Sexual Assault Prevention Programs is a report that includes measures of performance and measures of effectiveness for various prevention activities. In particular, Appendix C of the report lists the measures, their items, and response items. See Coreen Farris, Terry L. Schell, Margaret Tankard, Lisa H. Jaycox, Barbara Bicksler, Angela Clague, and Dionne Barnes-Proby, Measures of Performance and Effectiveness for the Marine Corps' Sexual Assault Prevention Programs, Santa Monica, Calif.: RAND Corporation, RR-2220-USMC, 2019. https://www.rand.org/t/RR2220.html
Development of a Short Form Measure of Sexual Harassment Risk in the Military: Findings from the RAND Military Workplace Study is a brief report that presents a shortened version of a measure that can predict an individual's exposure to sexual harassment. See Terry L. Schell, Matthew Cefalu, and Andrew R. Morral, Development of a Short Form Measure of Sexual Harassment Risk in the Military: Findings from the RAND Military Workplace Study, Santa Monica, Calif.: RAND Corporation, RR-2031-OSD, 2019. https://www.rand.org/t/RR2031.html
Handbook Overview