There are more than 3,000 suicide deaths in California each year—roughly nine deaths for every 100,000 California residents. To prevent suicides and other mental health problems, the California Mental Health Services Authority (CalMHSA) is implementing a variety of Prevention and Early Intervention (PEI) initiatives. CalMHSA is an organization of county governments working to improve mental health outcomes for individuals, families, and communities. PEI programs implemented by CalMHSA are funded through the voter-approved Mental Health Services Act (Prop. 63). Prop. 63 provides the funding and framework to expand mental health services to previously underserved populations and all of California's diverse communities.
CalMHSA asked RAND to evaluate the PEI initiatives to prevent suicide. To help inform the design of this evaluation, we reviewed some key aspects of the suicide prevention (SP) program evaluation literature to understand the state of the art, including relevant theories of change, what is and is not known about suicide prevention program effectiveness, and what kinds of methodologies have been previously used in evaluations of suicide prevention programs.
This report summarizes this scientific literature related to suicide prevention and is organized into three sections. First, we provide an overview of the epidemiology of suicide and of non-fatal self-inflicted injuries in California, as well as the empirical support for suicide risk factors, and explain why understanding this epidemiology is a critical first step in any effort to evaluate the effectiveness of suicide prevention programs. Second, we present our framework for conceptualizing suicide prevention programs that can be used to guide evaluation; it is based on a review of the relevant scientific literature. Finally, we conclude with a discussion of the measures that have been used to evaluate suicide prevention programs in the past.
The Epidemiology of Suicide in California
Suicide trends in California, by age, sex, and race, parallel trends seen more broadly in the United States. However, California's suicide rate is the same, if not higher, than the U.S. rate until age 55, at which point it becomes consistent with or lower than the national rate. As in the United States more broadly, California suicide rates are highest among whites and males. According to data from 2009, the majority of suicides in California resulted from self-inflicted gunshot wounds or hanging, strangling, or suffocation (California Department of Public Health, 2011). To date, three risk factors for suicide have the strongest empirical support: prior suicide attempts (Harris and Barraclough, 1997), mental health disorders (Beautrais et al., 1996; Cavanagh et al., 2003; Goldsmith et al., 2002; Harris and Barraclough, 1997), and substance misuse, abuse, and dependence (Goldsmith et al., 2002; Wilcox et al., 2004). Understanding these trends in deaths by suicide and suicide behaviors and the data sources for each is important for informing evaluations that rely on these data as a key or ultimate outcome.
RAND Conceptual Model of Suicide Prevention Programs
RAND developed a conceptual model of suicide prevention programs to help guide the CalMHSA evaluation design. The model is built on nine different categories of suicide prevention programs and displays proximal program goals and ultimate outcomes of different types of suicide prevention programs.
The nine categories of suicide prevention programs are training on coping skills and self-referral, marketing campaigns, gatekeeper trainings, crisis hotlines, postvention programs that guide a community to appropriately respond to suicide to prevent possible contagion, screening programs to identify and refer individuals in distress, provider training in suicide risk assessment and management, targeted mental health interventions, and social/policy interventions that increase access to care or restrict access to lethal means through policies that create a safe environment (e.g., restricting access to firearms). To date, there has been limited research on the effectiveness of many of these types of programs, but the literature suggests that three program approaches can lead directly to reductions in suicides: social/policy interventions that reduce access to lethal means (e.g., Ajdacic-Gross et al., 2006; Florentine and Crane, 2010; Sinyor and Levitt, 2010), increased provision of high-quality mental health care through targeted mental health interventions (e.g., Blue Ribbon Work Group on Suicide Prevention in the Veteran Populations, 2008; Brown et al., 2005; Jobes et al., 2005; Leitner et al., 2008; Stanley and Brown, 2008), and effective acute crisis response (e.g., through provider or physician trainings, postvention programs, or crisis hotlines) (Doshi et al., 2005; Mann et al., 2005; van der Feltz-Cornelis et al., 2011). We propose that these strategies need to be accompanied by efforts that ensure individuals know what services are available (e.g., via marketing campaigns to increase awareness, training on self-referral skills), procedures and trainings that help identify those at risk (e.g., through screening programs), and efforts that ensure individuals have access to and feel comfortable accessing care themselves or referring to care those at risk (e.g., by training the individual on coping skills or by gatekeeper trainings).
Measures for Suicide Prevention Program Evaluation
Suicide prevention program evaluations are challenged by methodological issues including the relative rarity of suicide deaths (Goldsmith et al., 2002; Mann et al., 2005; Ramchand et al., 2011). Given the rarity of suicide, intermediate outcomes are often used to evaluate suicide prevention programs. This report contains sample evaluation measures corresponding to each of the aforementioned proximal program goals and the ultimate outcome (i.e., fewer suicides).
Our review of the literature found that very commonly, suicide attempts (either self-reported or measured in hospital records) are used as the primary outcome in suicide prevention research. Although the validity of this measure is substantiated by evidence suggesting that a prior suicide attempt is the strongest predictor of subsequent death by suicide, it should also be kept in mind that about half of those who die by suicide have no history of a prior attempt (Gibb et al., 2005; Isometsa and Lonnqvist, 1998; Suominen et al., 2004), and the majority of those who make non-fatal attempts do not go on to die by suicide (Gibb et al., 2005; Suominen et al., 2004). Other proximal program goals may be changes in knowledge, skills, and attitudes or in intervention behaviors (e.g., after exposure to a media campaign or a gatekeeper training program); changes in access to and use of behavioral health care for programs designed to reduce mental health stigma or otherwise increase access; and clinical measures (e.g., treatment adherence) for programs designed to improve care. In addition, programs that target known risk factors for suicide might also be effective at preventing suicide.
Conclusions and Next Steps
Our review identified several important methodological considerations that can inform evaluation of suicide prevention programs.
- Suicide is a rare event. Even with a well-constructed evaluation, identifying that a prevention program was effective at reducing suicide deaths is challenging because of the narrow sample and lag in the availability of suicide data.
- Suicide varies by age, race, and sex. Thus, prevention programs may have differential effects on different population subgroups.
- Suicide ideation and attempts are important indicators of suicidality, but their relationship with suicide death is complicated. Even if a program shows immediate reductions in ideations and attempts, its effects on long-term suicide deaths are uncertain.
In addition to these methodological issues related to SP program evaluation, we also identified two areas where more research is needed:
- Linking SP programs to suicide rates. Although a body of SP evaluation research explores the effects of SP programs on such outcomes as reduced access to lethal means, provision of care, and crisis response, we also need to learn more about how these programs influence suicide rates.
- Determining SP program effectiveness among population subgroups. Although some evidence suggests that SP programs targeted for specific subgroups can be effective (“Suicide Prevention Evaluation in a Western Athabaskan American Indian Tribe—New Mexico, 1988—1997,” 1998; May et al., 2005), more research must address the differential effectiveness of SP programs for population subgroups vulnerable to suicide.
Ajdacic-Gross, V., Killias, M., Hepp, U., Gadola, E., Bopp, M., Lauber, C., Schnyder, U., Gutzwiller, F., and Rossler, W., “Changing Times: A Longitudinal Analysis of International Firearm Suicide Data,” American Journal of Public Health, Vol. 96, No. 10, 2006, pp. 1752–1755.
Beautrais, A. L., Joyce, P. R., Mulder, R. T., Fergusson, D. M., Deavoll, B. J., and Nightingale, S. K., “Prevalence and Comorbidity of Mental Disorders in Persons Making Serious Suicide Attempts: A Case-Control Study,” American Journal of Psychiatry, Vol. 153, No. 8, 1996, pp. 1009–1014.
Blue Ribbon Work Group on Suicide Prevention in the Veteran Populations, “Report of the Blue Ribbon Work Group on Suicide Prevention in the Veteran Populations,” 2008.
Brown, G. K., Ten Have, T., Henriques, G. R., Xie, S. X., Hollander, J. E., and Beck, A. T., “Cognitive Therapy for the Prevention of Suicide Attempts: A Randomized Controlled Trial,” JAMA: Journal of the American Medical Association, Vol. 294, No. 5, 2005, pp. 563–570.
California Department of Public Health, “EpiCenter: California Injury Data Online.” As of October 22, 2012: http://epicenter.cdph.ca.gov/Default.aspx
Cavanagh, J. T., Carson, A. J., Sharpe, M., and Lawrie, S. M., “Psychological Autopsy Studies of Suicide: A Systematic Review,” Psychological Medicine, Vol. 33, No. 3, 2003, pp. 395–405.
Doshi, A., Boudreaux, E. D., Wang, N., Pelletier, A. J., and Camargo, C. A., Jr., “National Study of Us Emergency Department Visits for Attempted Suicide and Self-Inflicted Injury, 1997–2001,” Annals of Emergency Medicine, Vol. 46, No. 4, 2005, pp. 369–375.
Florentine, J. B., and Crane, C., “Suicide Prevention by Limiting Access to Methods: A Review of Theory and Practice,” Social Science and Medicine, Vol. 70, No. 10, 2010, pp. 1626–1632.
Gibb, S. J., Beautrais, A. L., and Fergusson, D. M., “Mortality and Further Suicidal Behaviour After an Index Suicide Attempt: A 10-Year Study,” Australian and New Zealand Journal of Psychiatry, Vol. 39, No. 1–2, 2005, pp. 95–100.
Goldsmith, S. K., Pellman, T. C., Kleinman, A. M., and Bunney, W. E., Reducing Suicide: A National Imperative, Washington, D.C.: National Academies Press, 2002.
Harris, E. C., and Barraclough, B., “Suicide as an Outcome for Mental Disorders. A Meta- Analysis,” British Journal of Psychiatry, Vol. 170, 1997, pp. 205–228.
Isometsa, E. T., and Lonnqvist, J. K., “Suicide Attempts Preceding Completed Suicide,” British Journal of Psychiatry, Vol. 173, 1998, pp. 531–535.
Jobes, D. A., Wong, S. A., Conrad, A. K., Drozd, J. F., and Neal-Walden, T., “The Collaborative Assessment and Management of Suicidality Versus Treatment as Usual: A Retrospective Study with Suicidal Outpatients,” Suicide and Life-Threatening Behavior, Vol. 35, No. 5, 2005, pp. 483–497.
Leitner, M., Barr, W., and Hobby, L., Effectiveness of Interventions to Prevent Suicide and Suicidal Behaviour: A Systematic Review, Liverpool, Scotland: Scottish Government Social Research, 2008.
Mann, J. J., Apter, A., Bertolote, J., Beautrais, A., Currier, D., Haas, A., Hegerl, U., Lonnqvist, J., Malone, K., Marusic, A., Mehlum, L., Patton, G., Phillips, M., Rutz, W., Rihmer, Z., Schmidtke, A., Shaffer, D., Silverman, M., Takahashi, Y., Varnik, A., Wasserman, D., Yip, P., and Hendin, H., “Suicide Prevention Strategies: A Systematic Review,” JAMA: Journal of the American Medical Association, Vol. 294, No. 16, 2005, pp. 2064–2074.
Ramchand, R., Acosta, J., Burns, R.M., Jaycox, L., Pernin, C.G., Center for Military Health Policy Research, RAND Health, National Defense Research Institute (U.S.). Forces and Resources Policy Center, RAND Corporation, and United States Dept. of Defense, Office of the Secretary of Defense, The War Within: Preventing Suicide in the U.S. Military, Santa Monica, Calif.: RAND, 2011.
Sinyor, M., and Levitt, A. J., “Effect of a Barrier at Bloor Street Viaduct on Suicide Rates in Toronto: Natural Experiment,” British Medical Journal, Vol. 341, 2010, p. 2884.
Stanley, B., and Brown, G. K., The Safety Plan Treatment Manual to Reduce Suicide Risk: Veteran Version, Washington, D.C.: United States Department of Veterans Affairs, 2008.
Suominen, K., Isometsa, E., Suokas, J., Haukka, J., Achte, K., and Lonnqvist, J., “Completed Suicide After a Suicide Attempt: A 37-Year Follow-Up Study,” American Journal of Psychiatry, Vol. 161, No. 3, 2004, pp. 562–563.
van der Feltz-Cornelis, C.M., Sarchiapone, M., Postuvan, V., Volker, D., Roskar, S., Grum, A.T., Carli, V., McDaid, D., O'Connor, R., and Maxwell, M., “Best Practice Elements of Multilevel Suicide Prevention Strategies: A Review of Systematic Reviews,” The Journal of Crisis Intervention and Suicide Prevention, Vol. 32, No. 6, 2011, p. 319.
Wilcox, H. C., Conner, K. R., and Caine, E. D., “Association of Alcohol and Drug Use Disorders and Completed Suicide: An Empirical Review of Cohort Studies,” Drug and Alcohol Dependence, Vol. 76, 2004, pp. 11–19.
The research described in this report was prepared for the California Mental Health Services Authority and was conducted within RAND Health, a division of the RAND Corporation.