The Department of Defense Sexual Assault Prevention and Response Office asked the RAND Corporation to independently assess rates of sexual assault, sexual harassment, and gender discrimination in the military. This volume presents the results of methodological investigations into sources of potential bias in estimates produced from the 2014 RAND Military Workplace Study for active- and reserve-component members in the U.S. military.

Sexual Assault and Sexual Harassment in the U.S. Military

Volume 4. Investigations of Potential Bias in Estimates from the 2014 RAND Military Workplace Study

Edited by Andrew R. Morral, Kristie L. Gore, Terry L. Schell

Contributors: Amy Grace Donohue, Marc N. Elliott, Caroline Batka, Coreen Farris, Bonnie Ghosh-Dastidar, Lisa H. Jaycox, Dean Kilpatrick

ResearchPublished Mar 16, 2016

Cover: Sexual Assault and Sexual Harassment in the U.S. Military
Download PDF
Additional Downloads

Includes other ebook formats

In early 2014, the Department of Defense Sexual Assault Prevention and Response Office asked the RAND National Defense Research Institute to conduct an independent assessment of the rates of sexual assault, sexual harassment, and gender discrimination in the military — an assessment last conducted in 2012 by the Department of Defense using the Workplace and Gender Relations Survey of Active Duty Members. The resulting RAND Military Workplace Study invited close to 560,000 U.S. service members to participate in a survey fielded in August and September of 2014. This volume presents the results of methodological investigations into sources of potential bias in the survey estimates for active- and reserve-component service members. It includes evaluations of follow-up studies of survey nonrespondents and the efficacy of sampling weights to correct nonresponse bias, an assessment of total survey error using an administrative records benchmark, estimates of potential under- and overcounting of service members exposed to sexual assault, comparisons of events identified by prior survey forms and the RAND forms, analysis of survey non-consent and breakoff, and evaluation of service member tolerance of the RAND forms. In the final chapter, the report draws conclusions and recommendations for future administrations of sexual assault and harassment surveys in the military.

Key Takeaways

We found no conclusive evidence of substantial bias or error in previously reported RMWS estimates

  • There was, however, a general pattern that suggests the primary RMWS estimates of sexual assault, sexual harassment, and gender discrimination are more likely to be underestimates than overestimates of true population values.

We found little evidence that the study overcounted rates of sexual assault, sexual harassment, and gender discrimination

  • For example, although we concluded that a small number of pre-service sexual assaults may be captured in our estimates, this number is almost certainly lower than the larger number of assaults that go uncounted because we excluded members with fewer than six months of service and those who left the military shortly before the survey fielded.

A substantial portion of any underestimation of sexual assault rates in the military is attributable to our use of a sample frame that excludes some high-risk service members

  • The strongest evidence for bias comes from the fact that the survey sample frame clearly excluded some individuals who served in the military in the past year and who may have experienced a sexual assault. This source of bias may explain a substantial proportion of the total survey error identified by our comparison of survey-estimated counts of reported sexual assault to official reports of sexual assault.

We report evidence that the methods we used to weight survey respondents produced less-biased estimates than the conventional approach to weighting military surveys

  • In particular, the analysis of nonresponses weights found that the conventional sample weighting methods resulted in the underrepresentation of a number of groups of service members who have a high risk for sexual assault and harassment.

Recommendations

  • Future surveys of sexual assault and sexual harassment should use the RMWS measurement approach, or comparable survey questions that use behaviorally and anatomically specific language to clearly define criminal sexual assault and violations of equal opportunity law and policy.
  • Omitting service members who recently separated from the military could lead to significant bias in estimates of past-year sexual assaults, sexual harassment, and gender discrimination. As a result, we recommend including past-year separations in the sample frame of future surveys, or developing analytic approaches for estimating the number of crimes and violations those who separated experienced in the past year.
  • The Defense Manpower Data Center (DMDC) should design future surveys to include sufficient numbers of men in the sample to ensure ongoing assessment of the nature of sexual assaults against them. In practice, this means large sample surveys that may not oversample women at rates as great as in the RMWS or previous WGRA studies. This can be done without reducing the precision of women's estimates below those of men.
  • DMDC should build on approaches developed for the RMWS to include a wider set of factors in future nonresponse weighting models than has previously been possible for military surveys.
  • The Office of the Secretary of Defense (OSD), the Defense Information Systems Agency, and the services should collaborate to improve the coverage and reliability of email contact information in the personnel systems used for survey recruitment. Also, DMDC should investigate additional modes of recruitment (phone or text message) that improve outreach to members who do not routinely use email as part of their military duties.
  • In future surveys on sexual assault, DMDC should continue to compare survey estimates with actual numbers of filed Victim Reporting Preference forms as a measure of nonresponse bias and total survey error more generally.
  • OSD should conduct the survey no more frequently than once every two to four years.

Topics

Document Details

Citation

Chicago Manual of Style

Morral, Andrew R., Kristie L. Gore, and Terry L. Schell, eds., Sexual Assault and Sexual Harassment in the U.S. Military: Volume 4. Investigations of Potential Bias in Estimates from the 2014 RAND Military Workplace Study. Santa Monica, CA: RAND Corporation, 2016. https://www.rand.org/pubs/research_reports/RR870z6.html.
BibTeX RIS

Research conducted by

This research was conducted within the Forces and Resources Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.