What Has the CalMHSA Statewide Mental Health Prevention and Early Intervention Implementation Program Done So Far?

The California Mental Health Services Authority (CalMHSA)* — a coalition of California counties designed to provide economic and administrative support to mental health service delivery — formed the statewide Prevention and Early Intervention (PEI) Program to reduce adverse outcomes for Californians who experience mental illness. CalMHSA's focus is on three strategic initiatives: (1) reduce stigma and discrimination toward those with mental illness, (2) prevent suicide, and (3) improve student mental health. Twenty-three community organizations are under contract to develop and implement 25 specific components of the PEI initiatives to meet the goals of the strategic plans; these organizations are called PEI program partners. Starting in late summer of 2011, program partners began developing and implementing diverse PEI activities in accordance with their specific contract aims. In fall 2011, the RAND Corporation was asked to design and implement an evaluation of the three initiatives, and the final evaluation was approved and launched in summer 2012. The evaluation organizes program partner activities into six categories of core activities, many of which span more than one initiative area, as shown in the table.

The first-year report from the evaluation documents the capacities that were developed to implement core activities as of spring 2013 and assesses the early reach of selected activities (e.g., how many people receive services, are exposed to resources, and participate in trainings). While many PEI efforts were already being implemented at the time of the report, others were still under development, and implementation will continue over the next year or more. Future evaluation will include assessment of short-term outcomes of key initiative activities, as well as continued assessment of implementation and reach of activities. RAND also established baseline assessments of population risk factors and outcomes to provide a platform for longer-term monitoring of statewide PEI impacts over time.

Progress So Far

There is a logical, science-informed path from the statewide strategic plan to its goals of reducing mental health stigma and discrimination, preventing suicide, and improving student mental health. This path involves

1. the strategic planning of comprehensive, interrelated program components
2. developing new PEI program capacities
3. delivering new program activities to broadly reach California's diverse population and achieve significant exposure to program materials
4. measuring the impact of program activities on targeted short-term outcomes, such as knowledge and attitudes
5. achieving reductions in longer-term, adverse mental health outcomes for California's population, such as suicide rates, discrimination toward persons with mental health challenges, and mental-health-related school failure.

The first-year evaluation documents that program partners have been highly productive in developing new program capacities that relate to the components of the strategic plan. Furthermore, the launching of many program activities is well under way. This is impressive, given the relatively short time that program partners have had to develop and implement new program activities. So far, the reach of program activities has been relatively limited or cannot yet be determined, and many program activities are in the midst of rapidly expanding their reach. We do not know yet whether programs are having their intended short-term impacts on participants/audiences, but we expect to be able to answer those questions for key program activities over the next one to two years, within the timeframe of this evaluation.

An important aspect of the statewide PEI initiatives is the emphasis on evaluation. In addition to the independent evaluation being conducted by RAND, each program partner was required to plan its own evaluation activities, with a focus on developing capacities for quality improvement. Program partners were provided technical assistance, as needed, to carry out their own evaluation activities, as well as assistance to develop the data required for the RAND evaluation. This investment in developing evaluation capacity, at both the programmatic and broader initiative and population levels, represents a groundbreaking innovation. These evaluation efforts will not only help inform decisions about further investment in statewide PEI activities, they have also resulted in the development of evaluation approaches and tools that can be useful models for other county-directed PEI activities. The development of capacity to monitor population-level outcomes, risk factors, and exposure to PEI activities provides a platform for statewide assessment of the longer-term impacts of investments in PEI activities. It may be unrealistic, however, to expect observable population changes in the long-term outcomes of interest over the next year or two, given the start-up time required to build and launch new programs, the relatively brief time in which program effects will be observed, and the importance of broad population reach and exposure for prevention to reduce risk of deleterious mental health outcomes.

Topics

• California
• Health Care Program Evaluation
• Mental Health Treatment
• Survey Research Methodology