Improved Measures for Participation, Outreach, and Impact

Published Oct 30, 2018

by Jennie W. Wenger, Esther M. Friedman, Erin N. Leidy, Michael Vasseur, Kristin J. Leuschner, John D. Winkler


Download eBook for Free

FormatFile SizeNotes
PDF file 1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.


Purchase Print Copy

 Format Price
Add to Cart Paperback98 pages $26.00

Research Questions

  1. What communities are served by STARBASE, and is the program reaching the disadvantaged populations it is designed to reach?
  2. What are the direct outreach efforts employed by STARBASE program directors, and how might they build awareness of the program?
  3. What other paths could serve as indirect methods of program outreach and influence—that is, how might STARBASE build positive perceptions of the military or positive sentiments toward the military?
  4. What are some examples of potential measures that could capture the impact of outreach efforts?

The Department of Defense's STARBASE program offers an innovative curriculum focused on science, technology, engineering, and mathematics (STEM) for students in underserved areas of the United States. On the individual level, the program serves to improve STEM knowledge, particularly in the disadvantaged student population. At the community level, STARBASE's goal is to serve as an outreach program, improving community relations and building mutual loyalty and feelings of goodwill between communities and the Department of Defense. To accomplish these objectives, the STARBASE program serves fifth graders from disadvantaged schools; classrooms of students (and their teachers) participate in a week of hands-on activities as part of STARBASE's STEM-focused curriculum.

Previous research suggests that STARBASE is quite effective on a variety of short- and long-term outcome measures, including student test scores and student reports of confidence or efficacy on STEM subjects, as well as longer-term outcomes, such as encouraging interest in technology, lowering school absences, and improving scores on standardized tests. To date, however, less is known about how effective the program is at targeting disadvantaged students, and very little is known about whether STARBASE achieves its goals as a community outreach program. Indeed, there is no consensus on how to measure outreach efforts and outcomes from STARBASE or other similar programs. Because of these deficits, and because outreach is central to the program's mission, this study focuses on the students served and the program's outreach efforts.

Key Findings

  • There is strong evidence that STARBASE serves a relatively disadvantaged population, with the exception of Native American/Alaskan Native students, who are less likely to be present in the districts served. Considering the locations of these populations during any future expansion of STARBASE could result in the program reaching a wider range of disadvantaged youth.
  • To date, STARBASE programs' outreach efforts are largely local and consist primarily of specific relationships between individual STARBASE programs and relevant local organizations. These efforts have the potential to improve community relations and public perceptions of the military. Programs currently appear not to make much use of social media.
  • Social media campaigns sponsored by the Department of Defense (DoD) are likely to reach an audience that is already comfortable with the department and military service. While social media is not currently a major source of influencer impressions of the military, existing data suggest that social media is viewed as a relatively trustworthy source of information, and one that is likely to become increasingly relevant in the future.
  • STARBASE may be especially well situated to create positive outreach. Overall, DoD youth programs are located in areas with relatively high levels of trust in the military, but in the cases of STARBASE and National Guard Youth ChalleNGe, the difference in trust between areas with and without such programs is very small, suggesting that some of the programs are located in areas with lower levels of trust in the military.


  • Track levels of disadvantage. Future tracking of the levels of disadvantage among the students and communities served by STARBASE will provide a way to document the extent to which the program continues to serve students from disadvantaged schools.
  • Explore the use of increased social media outreach for Department of Defense (DoD) youth programs. Currently, STARBASE programs have very modest social media outreach efforts, mostly run by individual programs through Facebook accounts. A centralized social media campaign could have the potential to increase awareness of and support for STARBASE, and potentially for other DoD programs
  • Centrally manage DoD's youth programs. Measuring outreach efforts poses a substantial challenge, and more work is necessary to develop and test appropriate measures of outreach and their long-term impact. If the youth programs are considered as a group, placement decisions could be made to maximize both access to the programs and potential outreach.
  • Work toward a better understanding of the relationships between youth programs, military institutional presence, and the public's perceptions of DoD. Establishing improved measures of outreach could assist DoD in measuring the extent to which youth programs achieve this aspect of their missions. One method of measuring effectiveness would be to conduct a survey of a random subsample of the population, asking questions about awareness and impressions of various aspects of military outreach programs, or to adapt an existing survey for this purpose.

This research was sponsored by the Office of the Assistant Secretary of Defense for Reserve Affairs (Resources) and conducted within the Forces and Resources Policy Center of the RAND Corporation’s RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Department of the Navy, the Marine Corps, U.S. defense agencies, and the defense intelligence community.

This report is part of the RAND research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.