The National Guard Youth ChalleNGe program is a residential, quasi-military program for youth ages 16 to 18 who are experiencing difficulties in traditional high school. This capstone report describes some of RAND's research and analyses on the ChalleNGe program from late 2017 through mid-2020. The authors focus on identifying and recommending strategies for programs to measure participants' long-term outcomes.
Download
Download eBook for Free
| Format | File Size | Notes |
|---|---|---|
| PDF file | 1.8 MB | Use Adobe Acrobat Reader version 10 or higher for the best experience. |
Purchase
Purchase Print Copy
| Format | List Price | Price | |
|---|---|---|---|
| Add to Cart | Paperback116 pages | $23.00 | $18.40 20% Web Discount |
Research Questions
- How can ChalleNGe's long-term outcomes be measured in a way that demonstrates the extent to which the program is meeting its mission?
- Are current data collection methods sufficient to support measurement of long-term outcomes?
The National Guard Youth ChalleNGe program is a residential, quasi-military program for youth ages 16 to 18 who are experiencing difficulties in traditional high school. The program provides classroom instruction and other structured activities to young people at 40 different sites. This capstone report describes some of RAND's research and analyses on the ChalleNGe program from late 2017 through mid-2020.
Using a mixed-methods approach, the RAND team developed a program logic model, carried out a series of site visits, and planned and executed numerous analytic efforts. Many of the results of specific analytic efforts are described in other reports; here, the authors focus on identifying and recommending strategies for programs to measure participants' long-term outcomes. Consistently measuring long-term outcomes will allow the program to determine overall progress toward meeting its mission of ensuring that program participants are prepared for success as productive citizens.
Key Findings
ChalleNGe sites collect considerable data, but postgraduation placement data are lacking
- ChalleNGe sites have detailed information on the number of applicants, entrants, and graduates, as well as on other relevant outcomes, such as the amount of community service performed and standardized test scores.
- Sites gather placement data through mentors, and such data generally include no more than 12 months of minimal information; sites frequently express concern that mentor reporting drops off sharply after graduation. Sites may not be able to document industry or occupation, existence of benefits, or even total earnings.
- The data collected by ChalleNGe sites provide useful information and meet reporting requirements, but this information is not sufficient to determine how well the program is meeting its stated mission of creating long-term change and helping young people go on to become productive citizens.
ChalleNGe does not keep a single, authoritative administrative database
- Many of the sites use a similar database, the Cadet Tracker.
- RAND researchers have collected consistent data from individual sites for the past four years, and the National Guard Bureau also collects data periodically.
- Some sites have tested other methods of data collection; these include developing a customized database, surveying graduates, and focusing additional resources on tracking graduate placement.
Recommendations
- Each site should select the data collection strategy or strategies most appropriate to the site context.
- All sites should discuss the importance of data collection with cadets and parents.
- Site and program staff should budget for data collection.
- All sites should collect, at a minimum, data on graduates' educational attainment, labor force experience, and progress on other core components.
- Sites should collect data for three years postgraduation.
- The ChalleNGe program should invest in a single, consistent administrative database.
Table of Contents
Chapter One
Introduction
Chapter Two
ChalleNGe: Program Design and Logic Model
Chapter Three
Program Trends
Chapter Four
Existing Data and Approaches to Program Measurement
Chapter Five
Approaches for Measuring Long-Term Outcomes and Determining Program Effectiveness
Chapter Six
Conclusions and Recommendations
Appendix A
Contextual Factors That Influence ChalleNGe Sites
Appendix B
How Similar Youth Programs Approach Data Collection and Outcomes Measurement
Appendix C
Potential Measures of Citizenship, Leadership/Followership, and Service to Community
Appendix D
Supporting Mentors
Appendix E
Survey of Washington Youth Academy Graduates
Research conducted by
This research was sponsored by the Office of the Assistant Secretary of Defense for Manpower and Reserve Affairs and conducted within the Forces and Resources Policy Center of the RAND National Security Research Division (NSRD).
This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.