The process of developing and administering the 2022 Department of Defense National Survey of Employers yielded lessons learned beyond the substantive findings presented in the main research report. In this companion report, the authors catalog the survey lessons, describe the implications of the lessons, and make recommendations to inform future efforts.

Gauging What Employers Think

Lessons Learned from Fielding the 2022 Department of Defense National Survey of Employers

Brian Phillips, Susan M. Gates, Laura Werber, Jonas Kempf, Lucas Greer

ResearchPublished Apr 11, 2024

As part of a research project designed to shed light on the state of the relationship between members of the U.S. reserve component (RC) and their civilian employers, researchers at the RAND Corporation developed and administered the 2022 Department of Defense (DoD) National Survey of Employers. This survey, a modified version of the 2011 DoD National Survey of Employers, was sent to private-sector employers of varying sizes, DoD and non-DoD federal agencies, and state and local governments. The survey asked about employer experiences with RC employees—both rewards and challenges—with an emphasis on the implications of military duty–related absences.

The process of developing and administering the survey yielded lessons learned beyond the substantive findings presented in the main research report. These lessons could inform future efforts to survey employers of RC personnel. Some lessons could apply in other contexts as well. In this companion report, the authors catalog survey lessons learned related to (1) sample selection, (2) developing survey content, (3) obtaining required approvals, and (4) encouraging survey response. For each set of lessons, the authors describe the implications of the lessons and make recommendations for researchers and research sponsors.

Key Findings

  • DoD data on civilian employers of RC members have deteriorated in quality since the previous DoD National Employer Survey was fielded in 2011, limiting their usefulness as a frame from which to draw a representative sample to yield generalizable insights about this population.
  • Alternative approaches to surveying employers of RC personnel, including those used in conducting the 2022 DoD National Survey of Employers, have limitations because of the low likelihood that small employers that are selected using probabilistic methods employ an RC member and because of the challenges associated with obtaining contact information for employers.
  • Surveying employers in an era of low response rates, remote work, and data security concerns can be costly and entail additional effort to convince potential respondents of the survey's legitimacy and merit.

Recommendations

  • To support future surveys of RC employers, the RCs should better position themselves to meet the reporting requirements described in DoD Manual 7730.54. The Office of the Secretary of Defense should facilitate RC compliance with these reporting requirements or develop an alternative approach to collecting and maintaining civilian employer information.
  • DoD should consider splitting future employer surveys into multiple, shorter surveys, each targeted more narrowly to the organizational level and role that is best positioned to answer the survey questions.
  • DoD should experiment with additional approaches to encourage survey response, such as nonmonetary incentives that highlight the DoD brand, and build in mechanisms to measure whether these incentives are effective.
  • Absent an improvement in data quality, DoD should limit the use of probabilistic random sample surveys to questions that are relevant to all employers.

Topics

Document Details

Citation

RAND Style Manual
Phillips, Brian, Susan M. Gates, Laura Werber, Jonas Kempf, and Lucas Greer, Gauging What Employers Think: Lessons Learned from Fielding the 2022 Department of Defense National Survey of Employers, RAND Corporation, RR-A2361-1, 2024. As of September 18, 2024: https://www.rand.org/pubs/research_reports/RRA2361-1.html
Chicago Manual of Style
Phillips, Brian, Susan M. Gates, Laura Werber, Jonas Kempf, and Lucas Greer, Gauging What Employers Think: Lessons Learned from Fielding the 2022 Department of Defense National Survey of Employers. Santa Monica, CA: RAND Corporation, 2024. https://www.rand.org/pubs/research_reports/RRA2361-1.html.
BibTeX RIS

Research conducted by

This research was prepared for the Office of the Secretary of Defense and conducted within the Personnel, Readiness, and Health Program of the RAND National Security Research Division.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.