Cross-Agency Evaluation of DoD, VA, and HHS Mental Health Public Awareness Campaigns: Analysis of Campaign Scope, Content, and Reach
Jan 29, 2020
Photo by Capt. William Carraway/DVIDS Images
The materials designed to raise awareness of the Department of Veterans Affairs' (VA's) Make the Connection (MTC) campaign constitute one of four federally funded mental health public awareness campaigns that RAND is evaluating as part of a cross-agency evaluation (funded by the Department of Defense Psychological Health Center of Excellence) aimed at improving the mental health of service members and veterans. A cross-agency evaluation report describes the campaigns' overlapping and unique scope and content, as well as cross-campaign dissemination efforts. Based on the cross-agency evaluation report, this brief about MTC awareness materials is one in a series examining the individual campaigns' messages, the consistency of messaging in their materials, and the tools they use to deliver content to their audiences. Because the results presented here are from an evaluation of the four campaigns' collective reach and impact, they are not intended to serve as a full and comprehensive evaluation of MTC. It should also be noted that this evaluation focuses on the materials designed to raise awareness of MTC and not on the functioning or operations of the campaign itself.
The following are some key findings about MTC from the RAND research team's content analysis of campaign materials, an analysis of campaign-collected communication metrics, and a panel of experts who assessed the extent to which MTC's content and design align with best practices for mental health public awareness campaigns.
Of the 745 pieces of MTC content reviewed, 77 percent was aimed at veterans, and 11 percent was aimed at families of veterans. MTC's materials portrayed more veterans from pre-2001 eras than most other campaigns in the evaluation.
MTC's materials generally adhered to best practices for mental health public awareness campaign design and dissemination. The panel of experts also indicated that MTC used credible messengers in pictures and videos and portrayed people with mental health challenges as positive role models.
Almost 60 percent of MTC's materials supported its campaign messages, and its materials aligned closely with the messaging of the three other public awareness campaigns we reviewed—particularly with the Real Warriors Campaign, which also creates video testimonials about recovery. MTC content spoke mostly about mental health issues (79 percent) and substance use issues (28 percent). A small amount of content explicitly targeted its secondary audiences: 11 percent of materials were aimed at veterans' families, and less than 1 percent were aimed at health professionals.
None of MTC's content clearly identified the source of the information. MTC reports that omitting citations is intended to make materials more accessible and appealing, but this design choice could make it difficult for users to judge credibility or seek out more information.
Because of MTC's emphasis on tailored video testimonials, 81 percent of veteran-targeted campaign content was video.
The website played host to nearly 3 million sessions in 2014 and 2015, compared with 1.3 million for the other three campaigns combined. In addition, the number of website sessions increased 59 percent from 2012 to 2015.
MTC's materials provide links to care locator services. As a way to build trust and protect anonymity, the campaign does not collect users' information. However, only 8 percent of its materials provide a direct connection to care (e.g., phone line, live chat line, direct connection to a medical center).
likes, comments, and shares were generated by Facebook posts in 2015
MTC aired 11,725 radio and 39,135 television public service announcements in 2015, for a combined 422 million impressions (the number of audience members who might have been exposed to the announcement). That is a fraction of the 42.9 billion impressions for the four campaigns combined.
The experts recommended several approaches to improve the MTC campaign. See the full report for a complete list of recommendations based on the cross-campaign analysis.