Cross-Agency Evaluation of DoD, VA, and HHS Mental Health Public Awareness Campaigns: Analysis of Campaign Scope, Content, and Reach
Jan 29, 2020
Recovery Month, operated by the Department of Health and Human Services' Substance Abuse and Mental Health Services Administration, is one of four federally funded mental health public awareness campaigns that RAND is evaluating as part of a cross-agency evaluation (funded by the Department of Defense Psychological Health Center of Excellence) aimed at improving the mental health of service members and veterans. A cross-agency evaluation report describes the campaigns' overlapping and unique scope and content, as well as cross-campaign dissemination efforts. Based on the cross-agency evaluation report, this brief about Recovery Month awareness materials is one in a series examining the individual campaigns' messages, the consistency of messaging in their materials, and the tools they use to deliver content to their audiences. Because the results presented here are from an evaluation of the four campaigns' collective reach and impact, they are not intended to serve as a full and comprehensive evaluation of Recovery Month. It should also be noted that this evaluation focuses on the materials designed to raise awareness of Recovery Month and not on the functioning or operations of the campaign itself.
The following are some key findings about Recovery Month from the RAND research team's content analysis of campaign materials, an analysis of campaign-collected communication metrics, and a panel of experts who assessed the extent to which Recovery Month's content and design align with best practices for mental health public awareness campaigns.
Of the 209 pieces of Recovery Month content reviewed, 93 percent targeted the general population. A small proportion of materials (5 percent) targeted service members and veterans. Of the four campaigns, Recovery Month had the most material specific to substance use issues (93 percent), and 62 percent of materials were also related to mental health. Consistent with their intended audience, Recovery Month materials mostly depicted people with no obvious military association, but some materials contained images of service members or veterans.
More than 90 percent of Recovery Month materials supported its campaign messages, and materials generally aligned with the messaging of the other three campaigns that were reviewed. Recovery Month emphasizes substance use issues to a greater extent than the other campaigns do.
Recovery Month materials generally adhered to several best practices for mental health public awareness campaign design and dissemination, as identified by a panel of experts and a review of the literature. Experts also indicated that Recovery Month used appropriate communication channels for its target audience. However, the experts noted that the target audience was extremely broad, the materials sometimes used technical language and jargon, and the mix of substance use and mental health resources could make it difficult to distinguish between the two.
About 80 percent of Recovery Month materials failed to clearly cite the source of the information. It is possible that omitting citations is intended to make materials more accessible and appealing to veterans who may seek to avoid mental illness–related labels, but this design choice could make it difficult for users to judge credibility or seek out more information.
The Recovery Month website saw nearly 123,800 website sessions in 2015. However, the website is not Recovery Month's primary means of dissemination; its reach depends largely on its partner organizations hosting their own Recovery Month events and activities.
None of the other three campaigns that were evaluated referenced or linked to Recovery Month, though Recovery Month materials occasionally referenced two of the other campaigns. Our overall analysis found numerous missed opportunities for the four campaigns' websites to link to each other as resources.
is used only by Recovery Month and one other campaign
In 2015, about 37,700 people followed Recovery Month on Facebook, 16,780 followed on Twitter, and 7,790 viewed YouTube videos—the lowest numbers of the three campaigns that are active on social media. Recovery Month and one other campaign are the only active users of Twitter.
Recovery Month aired 91,414 radio and 59,835 television PSAs in 2015, for a combined 1.19 billion impressions (the number of audience members who might have been exposed to the PSA). That is a fraction of the 42.9 billion impressions for the campaigns combined in 2015.
Photo by Mass Communication Specialist 2nd Class Chad A. Bascom/U.S. Navy
The experts recommended several approaches to improve the Recovery Month campaign. See the full report for a complete list of recommendations based on the cross-campaign analysis.