What Has the Suicide Prevention Initiative Done So Far?

The California Mental Health Services Authority (CalMHSA)^[1]—a coalition of California counties designed to provide economic and administrative support to mental health service delivery—formed the statewide Prevention and Early Intervention (PEI) Implementation Program to reduce adverse outcomes for Californians who experience mental illness. CalMHSA's focus is on three strategic initiatives: (1) reduce stigma and discrimination toward those with mental illness, (2) prevent suicide, and (3) improve student mental health. Under each initiative, community agencies serve as PEI program partners, performing activities to meet the initiative's goals. In 2011, the RAND Corporation was asked to design and implement an evaluation of the three initiatives in terms of the content of targeted programs, their reach (i.e., who receives services or is exposed to resources and trainings), and the short- and long-term outcomes of those activities (e.g., attitudes and knowledge about mental illness and reduced suicide, reduced discrimination, and improved student performance, respectively).

For the Suicide Prevention (SP) Initiative, first-year evaluation efforts have focused on efforts being implemented by a number of program partners (and their subcontractors):

• AdEase (and its subcontracts with Education Development Center and Your Social Marketer, Inc.)
• Didi Hirsch Psychiatric Services (and its subcontract with Optum Health, Helpline, Inc., San Francisco Suicide Prevention, Wellspace Health, Kern County Mental Health, Community Health Improvement Partners)
• Family Service Agency of the Central Coast (FSACC)
• Family Services Agency of Marin (FSA Marin) and its subcontract with Sonoma County Indian Health Project
• Institute on Aging (IoA) (and its subcontract with Wellspace Health)
• Kings View
• LivingWorks (and its subcontracts with Didi Hirsch, Wellspace Health, and Contra Costa Crisis Center)
• San Francisco Suicide Prevention (SFSP) (and its sub-contracts with Contra Costa Crisis Center, Star Vista Crisis Center, and Santa Clara county)
• Transitions Mental Health Association (TMHA) (and its subcontracts with Kern County and Mental Wellness Center of Santa Barbara).

What Have the SP Initiative Program Partners Accomplished So Far?

The RAND evaluation of SP Initiative program partners focuses on four key activities: (1) networking and collaboration activities, (2) trainings or educational programs for a broad range of audiences, (3) social marketing, and (4) hotlines (to include web- and text-based crisis response services) and "warmlines" (a warmline is a noncrisis telephone service that provides encouragement and support to persons in need). To understand trends in suicide, RAND is also conducting an analysis of suicide rates in California.

While program partners have implemented many efforts in the past year, others are still in development, with plans for implementation over the coming year. Thus, the evaluation is ongoing, and the results are preliminary. RAND's Year 1 evaluation focused on the activities pursued, the early reach of materials and activities, and short-term outcomes from trainings and education.

Activities. In the networking and collaboration area, the Year 1 evaluation focused primarily on the California Statewide Suicide Prevention Network (CSPN), led by Didi Hirsch. Principal CSPN activities include (1) developing crisis line data metrics, such as demographics and suicide risk, that all participating crisis lines will collect in future years and (2) establishing regional task forces that will serve as best practice advisory boards. These advisory boards will help identify the most relevant suicide prevention activities for the region.

In the area of trainings and education, the evaluation focused on LivingWorks—a suicide intervention training company—and in particular on Applied Suicide Intervention Skills Training (ASIST)—an intensive two-day suicide intervention workshop for people "who want to feel more comfortable, confident and competent in helping to prevent the immediate risk of suicide."

In the social marketing area, the evaluation focus was on AdEase's contract for "Know the Signs," a statewide social marketing campaign. The evaluation looked at the campaign's "Your Voice Counts" web forum, its "Directing Change" contest, its website (www.suicideispreventable.org), and its Media Advocacy Toolkit.

In the hotlines and warmlines area, our focus was on the hotline, chat, and text services created or expanded by funded crisis centers during the evaluation period; hotline efforts to seek accreditation; and assessing call volume.

Reach. We examined the reach of the Know the Signs marketing campaign thus far, using metrics compiled via Google Analytics—the industry standard application for web analytics, which captures a wide range of metrics on use of and interaction with web properties, as well as traffic sources and additional information. The table provides some of the key metrics for the Know the Signs website.

Beyond the website, AdEase disseminated the campaign through television advertisements, billboards, magazine advertisements (in, for example, Newsweek), and advertisements using digital media. RAND measured these dissemination efforts by media impressions—the total number of people that may have been exposed to the campaign. In Los Angeles, for example, which has the highest number of media impressions, there were more than 21 million television impressions, 132 million billboard impressions, 5 million magazine impressions, and 191 million digital media impressions. After Los Angeles, the order is the same as for the website: San Francisco, Sacramento-Stockton-Modesto, San Diego, and Fresno.

In the hotlines and warmlines area, we measured reach by extracting call volume from quarterly reports submitted by program partners to the funder for the first three quarters (Q1–Q3) of Year 2, when such information was readily available, and directly from program partners. Call centers varied dramatically with respect to call volume, ranging from fewer than 200 calls received by a newsmall county warmline in Q3 to over 7,000 calls received by established crisis lines serving large urban areas. It is also notable that call volume increased over the three quarters at almost all of the crisis and warmlines for which volume was available.

Short-Term Outcomes. To start assessing short-term outcomes in the area of trainings and education, we administered post-training surveys for seven of the 29 ASIST trainings that occurred in February and March 2013. We examined changes in intervention self-efficacy and intervention behavioral intentions before and immediately after the training. Participants at the seven ASIST workshops reported significant changes in overall self-efficacy (e.g., feeling better prepared to help and having greater confidence in helping a person at risk, increased comfort discussing suicide with others) after training. They also reported significant changes in their overall behavioral intentions after completion of the training; such changes included the willingness of participants to ask clients directly about whether they were thinking about suicide and whether participants would be willing to intervene with someone they thought was at risk for suicide.

What Do We Find in Evaluating California Suicide Rates?

Beyond evaluating key SP Initiative program partner activities, we analyzed suicide fatalities in California to establish baselines against which later suicide rates may be compared. One key finding is that suicide rates vary dramatically by region in California. Although rates are highest in California's most rural areas, suicides in these areas account for a very small proportion of California's overall number of suicides (approximately 6 percent). This suggests that resources must still be allocated to the areas with the highest numbers of suicides, even though the rates may be lower. The Southern region (i.e., Kern and counties south of it excluding Los Angeles) had the highest number of suicides over the three-year period—nearly 4,000—compared with 660 in the Superior region (i.e., Mendocino county and its northern neighbors), which was the region with the lowest suicide prevalence.

What Are the Plans for Future Evaluation of the SP Initiative?

Continuing evaluation efforts are planned in all four SP core areas. SRI International, RAND's partner in the CalMHSA Statewide Evaluation, will lead the evaluation of networking and collaboration. With regard to trainings and education, more data from ASIST trainings will be forthcoming, and we will descriptively analyze data from the live observations of ASIST workshops to determine how ASIST trainings are delivered in community settings. With regard to social marketing, we will conduct an independent analysis of selected AdEase products and experiments to evaluate the efficacy of selected materials. We also plan to evaluate changes in media messages about suicide that may be attributable to dissemination of the Media Advocacy toolkit and other social marketing interventions over time. In the hotlines and warmlines area, this year's comprehensive literature review supported the development of a rigorous evaluation design in which independent, trained observers will rate call content. RAND's evaluation strategic plan includes implementing this design with all CalMHSA program partners contracted to initiate, expand, or enhance their crisis call services.

Notes

• [1] The California Mental Health Services Authority (CalMHSA) is an organization of county governments working to improve mental health outcomes for individuals, families and communities. Prevention and Early Intervention programs implemented by CalMHSA are funded by counties through the voter-approved Mental Health Services Act (Prop. 63). Prop. 63 provides the funding and framework needed to expand mental health services to previously underserved populations and all of California's diverse communities.

Topics

• California
• Health Care Program Evaluation
• Mental Health Treatment
• Suicide