Cover: Evaluation Design for Department of Air Force Military and Family Readiness Programs

Evaluation Design for Department of Air Force Military and Family Readiness Programs

Considerations for Air Force Families Forever, Relocation Assistance Program, and Employment Assistance

Published Mar 5, 2024

by Thomas E. Trail, Miriam Matthews, Samantha E. DiNicola, Jenna W. Kramer, Coreen Farris, Isabelle González

Download eBook for Free

Full Document

FormatFile SizeNotes
PDF file 1.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Summary

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Questions

  1. What factors lead to the success or failure of these DAF programs, and how can that success or failure be measured?
  2. What resources do these programs need to run, how do they run, and what effects do they have?
  3. What potential sources of data exist that would allow measuring of the various program outcomes desired?

Military service can be rewarding but also challenging for airmen, guardians, and their families. Frequent relocations require navigating housing and movement of household goods and can lead to difficulties for spouses wanting employment. Another aspect of military service that presents difficulties for family and friends is coping with a service member's death. To help address these challenges, the Department of the Air Force (DAF) has several programs that are implemented via Military and Family Readiness Centers: the Relocation Assistance Program (RAP), Employment Assistance (EA), and Air Force Families Forever (AFFF). RAP provides education and referrals to assist airmen, guardians, and their families with issues surrounding permanent change of station (PCS) moves. EA supports service members, DoD civilian employees, and family members in achieving employment. AFFF provides long-term information and referrals to eligible next of kin of deceased airmen and guardians. DAF asked RAND Project AIR FORCE to assist with the design of evaluations for these programs.

Key Findings

  • The success of each of the programs is unclear because of limited feedback and evaluation.
  • A factor believed to reduce the effectiveness of each of the programs is limited personnel capacity, or limited staffing, for program administration.
  • A factor commonly believed to positively affect program effectiveness is active unit commander support and engagement.
  • The logic models provided in this report map the various assumptions regarding what resources the programs need to run, how they run, and what effects they have, and they provide a road map for the development of a program evaluation.
  • Potential sources of data to measure elements of the logic models include administrative data, surveys, and qualitative assessments.

Recommendations

  • Adopt and communicate the program logic models in ways that ensure that stakeholders know and understand them.
  • Continue to engage stakeholders in the evaluation design process.
  • Implement measures that align with the program logic model.
  • Present the results of evaluations in ways that are clear and useful for stakeholders.
  • Modify the program logic models, as needed, drawing from the results of evaluations.
  • If DAF considers creating or using additional databases to track measures, involve stakeholders in its development.

Research conducted by

This research was conducted within the Workforce, Development, and Health Program of RAND Project AIR FORCE.

This report is part of the RAND research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.