Cover: Evaluation Design for the Department of the Air Force's True North Program

Evaluation Design for the Department of the Air Force's True North Program

Development of a Logic Model and Program Measures

Published Jan 24, 2024

by Miriam Matthews, Thomas E. Trail, Samantha E. DiNicola, Coreen Farris, Jenna W. Kramer, Isabelle González

Download eBook for Free

Full Document

FormatFile SizeNotes
PDF file 0.7 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Summary

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Download Support Files

Appendix B

FormatFile SizeNotes
zip file 0.1 MB

The file(s) provided above are ZIP-formatted archives, which most modern systems can natively unpack. If your computer does not unpack the archive when you double-click it, you may need to use a separate decompression program such as UnZip.

Research Questions

  1. What are the impacts of True North's embedded behavioral health efforts?
  2. What are some of the challenges that True North faces?
  3. How can the Department of the Air Force measure the effectiveness of the True North program?

Developing strong resiliency and care solutions for airmen and guardians is key to human capital development and force readiness. The True North program is one of the Department of the Air Force's (DAF's) most significant recent investments in promoting the resiliency of its people. Assessing the program's level of success, justifying funding, and informing decisions about the program’s future will require a rigorous evaluation. The authors of this report (1) identify desired outcomes for members participating in the program, (2) define appropriate measures of effectiveness that could be used in evaluating the True North program, and (3) make recommendations for ongoing internal evaluation of the program.

The True North program encompasses selected installation welcome centers, embedded religious support teams (RSTs), and embedded mental health (EMH) teams. To determine how DAF might evaluate this program and its components, the authors reviewed relevant policies and procedures and literature relevant to the program components and conducted interviews with 17 True North program personnel and 21 group and squadron commanders. They present a program logic model to determine potential evaluation measures.

Key Findings

  • Research shows promising impacts of embedded behavioral health, including in military, education, and health care settings. These impacts include greater service use, increases in accuracy of diagnosis, and greater continuity of care.
  • True North program personnel perceived positive outcomes from the program but also described challenges with funding changes and delays in embedding personnel that might limit program effectiveness.
  • Interviewees indicated that wing chaplains often provide helpful supplemental funding to religious support teams, but disagreement exists about the level at which religious support teams should be embedded.
  • Many locations that have other elements of the True North program do not have True North welcome centers.
  • Critical elements for determining the effectiveness of the True North program will be assessing whether the program has received all requisite inputs and whether activities have been implemented as intended.

Recommendations

  • Adopt and communicate the program logic models in ways that ensure that stakeholders know and understand them.
  • Continue to engage stakeholders in the evaluation design process.
  • Implement assessment tools that are aligned with the program logic model.
  • Present the results of evaluations in ways that are clear and useful for stakeholders.
  • Modify the program logic model as needed, drawing from the results of evaluations.
  • If Air Force Resilience considers creating or using a database to track True North measures, involve stakeholders in its development.

Research conducted by

This research was prepared for the the Department of the Air Force and conducted within the Workforce, Development, and Health Program of RAND Project AIR FORCE.

This report is part of the RAND research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.