Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Brief

The California Mental Health Services Authority* (CalMHSA) — a coalition of California counties designed to provide economic and administrative support to mental health service delivery — formed the statewide Prevention and Early Intervention Implementation Program to reduce adverse outcomes for Californians who experience mental illness. CalMHSA's focus is on three strategic initiatives: (1) reduce stigma and discrimination toward those with mental illness, (2) prevent suicide, and (3) improve student mental health. Under each initiative, community agencies serve as program partners, performing activities to meet the initiative's goals. In 2011, the RAND Corporation was asked to design and implement a three-year statewide evaluation of the three initiatives in terms of the content of targeted programs, their reach (e.g., who receives services or is exposed to resources and training), and the short- and long-term outcomes of these activities (e.g., attitudes and knowledge about mental illness and reduced suicide, reduced discrimination, and improved student performance).

For the Student Mental Health (SMH) Initiative, first-year evaluation efforts have focused on programs and efforts being implemented by the following program partners:

  • California County Superintendents Educational Services Association (CCSESA)
  • California Department of Education (CDE)
  • University of California
  • California State University
  • California Community Colleges.

What Have the SMH Initiative Program Partners Accomplished So Far?

Program partners have been developing and implementing a range of prevention and early intervention activities for student mental health issues, primarily targeting K–12 schools and higher education campuses across California. These activities focus on three core program activities: (1) networking and collaboration within and across educational institutions and/or other institutions addressing student mental health issues; (2) informational/online resources; and (3) training and educational programs for faculty, staff, and students. In addition, RAND has begun conducting surveys on higher education campuses and in K–12 schools. While many evaluation efforts have been implemented in the past year, others are still in development. Thus, the evaluation is ongoing, and the results discussed here are preliminary.

Core Activity Results. Evaluation of the networks and collaborations activity is in the early stages. So far, we have identified five strategically planned networks and collaborations and are compiling and reviewing documents and materials from them.

With regard to online and information resources, the already launched SMH program partner websites provide substantial resources for individuals seeking information about student mental health. Our evaluation is assessing the content and reach of the websites. Table 1 highlights key measures of reach based on statistics from Google Analytics for the CCSESA website.

Traffic Metrics Number of visits: 2,667
Number of page views (i.e., instances of a page being loaded by a browser): 11,479
Top five visited topic areas
  • Anger Management: 820 page views
  • Mental Health/Wellness: 416
  • Bullying: 344
  • Behavior Management: 331
  • Suicide: 321
User Engagement and Navigation 82 percent accessed site directly through URL into browser, bookmarks
10 percent accessed site through external link
8 percent accessed site through a search (e.g., Google)
Top three referrals coming from external link
  • 43 visits
  • 38
  • 36
Average time on site: 2.44 minutes
Average number of pages visited: 4.3 pages
Percent leaving from the homepage: 48 percent
User Characteristics Top sources of traffic to site in California
  • Sacramento-Stockton-Modesto: 922 visits
  • Los Angeles: 579
  • San Francisco-Oakland-San Jose: 389
  • San Diego: 107
  • Chico-Redding: 102
ISP Referrals from County or State Organizations Top ISP provider referrals
  • Sacramento County Office of Education: 292 visits
  • California State University Network: 82
  • San Bernardino County Superintendent of Schools: 75
  • Los Angeles County Office of Education: 64
  • Ventura County Office of Education: 63

Table 1. Reach Statistics for CCSESA Site from September 2012 Through March 2013

In the education and training area, all program partners have begun training activities. In terms of reach, for example, CCSESA has sponsored 168 trainings on such topics as crisis or behavioral intervention, suicide prevention, general mental health promotion, and bullying prevention. Based on completed training surveys, training participants were mostly 26 to 59 years old, white, and female. Approximately 35 per-cent identified as Hispanic, Latino, or Spanish origin. CDE has sponsored three Training Educators Through Recognition and Identification Strategies (TETRIS) training-of-trainer (TOT) events; each TETRIS participant is committed to conducting three local trainings. In higher education, between October 2012 and March and April 2013, California Community Colleges have conducted approximately 425 presentations and trainings, reaching approximately 16,000 faculty, staff, and students; the California State University system has conducted 200 trainings, presentations, and outreach events; and the University of California system has conducted approximately 1,100 trainings and informational events for faculty, staff, graduate teachers, research assistants, and students.

To begin assessing short-term outcomes, training evaluation surveys were analyzed for immediate posttraining changes in knowledge and attitudes among training participants. On average, in terms of satisfaction, participants who attended CCSESA trainings indicated the training was helpful, met the needs of diverse students, and was important to attend; participants who attended the two CDE TETRIS TOT trainings for which data were available also felt the training was helpful. Participants attending CCSESA trainings reported greater self-efficacy in their ability to access education and resources to learn more about mental health distress, and those attending two of the three CDE's TETRIS TOT trainings reported significantly greater confidence in identifying where to refer, comfort discussing mental health, and confidence helping students, as well as increased awareness of warning signs. CCSESA participants and those attending two of the three CDE TETRIS TOT trainings reported significant increases in their likelihood to encourage students to seek help from professionals, parents, or friends; provide advice and guidance; give students a phone number to call; ask students questions to assess the problem; and call security or an administrator to support the student.

Preliminary Survey Results. Preliminary results from the higher education students, faculty, and staff surveys are summarized in Table 2. Note that participants were not randomly selected, so there may be bias in who chose to participate.

Higher Education Survey: Students (6,309 participating students on select campuses, spring 2013)
  • 20 percent of students met or exceeded the cutoff for probable mental health problems in the past 30 days.
  • 25–35 percent reported academic performance was negatively affected by anxiety or depression.
  • Most indicated they knew where to go for help when they needed it and had many ways to work out a problem.
  • 25 percent reported having used mental health services or having been referred for or sought mental health services or counseling from a current campus's counseling or health service center.
  • Of the 25 percent who reported receiving services,
    • 75 percent reported having received them on campus
    • 72 percent of the time, students reported initiating the process of seeking service.
  • The majority reported receiving information from campuses on mental health and substance use issues.
  • Students generally agreed there was a positive campus climate in terms of mental health issues.
Higher Education Survey: Faculty and Staff (3,025 faculty and staff on select campuses, spring 2013)
  • Faculty and staff reported campuses provided adequate mental health counseling and support to students.
  • They felt campuses provide effective confidential support and referral services for students needing help with depression, stress, substance use, violence, or other emotional issues and that campuses emphasize helping students with social, emotional, and behavioral needs.
  • 24 percent reported having talked with a student about mental health once or twice, and 30 percent did so a few or many times, but almost one-half (46 percent) did not discuss mental health with students in past month.
  • 20 percent reported having attended training on student mental health (online or in person) over the past six months.
  • Over one-half felt they knew where to refer students who need mental health resources.
  • When asked about activities in the last six months, one-half reported being concerned about one or more students because of the student's psychological distress, and 34 percent referred at least one student for support services.

Table 2. Preliminary Results from Higher Education Student and Faculty/Staff Surveys

What Are the Plans for Future Evaluation of the SMH Initiative?

Plans for future evaluation in the networks and collaborations area include, for example, quantitatively analyzing data from community health and mental health departments and SMH program partners to track whether collaborative activities are associated with increases in referral and utilization rates. We are developing a follow-up survey to assess the helpfulness of the informational/online resources. As for education and training, we will, for example, conduct content analyses of facilitator and participant training materials from the one to three trainings that were most frequently offered. Also, we are working with the K–12 program partners (CDE and CCSESA) and WestED to develop and field the K–12 mental health campuswide surveys to staff and students in the 2013–14 academic year.

This report is part of the RAND research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.