4: Using Behavioral Health Surveillance Data for Action

Sections 2 and 3 of this toolkit covered how to define BH-related questions of interest, identify possible data sources to help answer them, and apply appropriate analytic methods to those data sources. Those sections are likely most relevant to people working directly with surveillance data.

This section provides guidance on how to use information gathered through BH surveillance for planning, decisionmaking, and other PH action. It addresses the following topics:

• interpreting your data through triangulating among multiple sources
• exploring options for taking PH action
• dealing with uncertainty
• recognizing the importance of partnerships to interpret BH surveillance data, disseminate findings, and take action.

The information covered here will be most relevant to you if you make decisions that are informed by surveillance data or provide recommendations to other decisionmakers on actions they could take.

Overview of the Behavioral Health Surveillance Process

BH surveillance is a continual process involving several steps, the timing and urgency of which will depend on whether you are conducting BH surveillance before, during, or after a PHE and on the nature of the PHE. This process involves the following steps:

• defining the BH problem(s) or question(s) of interest
• compiling the data on relevant BH indicators
• processing, analyzing, and interpreting the data, along with BH partners, to understand BH risks, needs, and impacts
• reporting the findings to relevant partners and audiences, including PH and BH agencies, nonprofits that address BH needs, local government, the media, and the public
• using the findings to take action
• assessing how well the process worked and making continual improvements (Summary 4.1).

Although Summary 4.1 shows the cycle as unidirectional, in reality, you will often need to return to earlier steps as you move through this process. For instance, you may need to refine your question of interest as you begin to compile data; you may need to gather additional data after you start analyzing what you have; you may report the findings and receive additional queries from a partner that requires you to perform additional analyses; and so on.

As discussed later in this section, partnerships and a focus on equity are critical at each step of this process.

The BH surveillance process involves six steps. First, define the BH problem(s) or question(s) of interest. Second, compile the data on relevant BH indicators. Third, process, analyze, and interpret the data, along with BH partners, to understand BH risks, needs, and impacts. Fourth, report the findings to relevant partners and audiences, including public health and BH agencies, nonprofits that address BH needs, local government, the media, and the public. Fifth, use the findings to take action. Sixth, assess how well the process worked and make continual improvements.

Source: Adapted from the World Health Organization Injury Surveillance Guidelines (Holder, 2001).

Learn more about embedding equitable practices throughout the data life cycle.

Interpreting Behavioral Health Surveillance Data

As previous sections of this toolkit make clear, there is no single authoritative source of data on BH that can be used for BH surveillance, and each potential data source described in Section 2 has limitations.

To strengthen confidence in your estimates of BH risks, needs, and impacts, you can triangulate findings from analyses of multiple data sources to look for patterns (Summary 4.2). By doing so, you can determine if there are

• similar or conflicting trends in the data
• time lags between spikes in different data sources
• variations by location, type of PHE, or population subgroup.

Triangulation of findings involves comparing different types of data to one another to see if they tell a coherent story. These types of data could include ED visits, 2-1-1 calls, prescription medication fills, PCC calls, and EMS activations.

Triangulating data can be as simple as visualizing your data so that you can compare similar time frames from different data sources to see if they are changing in similar ways. For example, to examine how the COVID-19 pandemic declaration may be related to several BH indicators of interest, Analysis 4.1 plots the following indicators for Los Angeles County, California:

• COVID–19 cases (March 2019–June 2021)
• initial unemployment claims (March 2019–May 2021)
• new prescriptions filled for selective serotonin reuptake inhibitors (SSRIs), which are common antidepressants (March 2019–May 2021)
• fatal overdoses (January 2018–November 2020).

A black vertical line on each panel marks the March 2020 time frame when a stay-at-home order was issued in California and a national emergency was declared.

Note: Due to data availability, the x-axes are not identical across these four figures. The vertical line in each panel denotes the declaration of the COVID-19 pandemic.

Key Takeaways:

COVID-19 cases peaked in Los Angeles County between November 2020 and March 2021. Increases in unemployment claims (an upstream indicator of BH risk) occurred right after the stay-at-home order was issued in April 2020 and again in September 2020. SSRI fills (a midstream BH indicator) briefly increased in February and March 2020, then returned to pre-pandemic levels, and then steadily increased again beginning in June 2020, especially among young adults ages 18–39. Fatal overdoses (a downstream BH indicator) appeared to trend upward for all age groups, with a pronounced increase in June–July 2020. This information can help PH officials anticipate when different BH impacts of PHEs might emerge and, in the case of unemployment, when risk factors might increase, so that they can proactively address them and explore patterns in the time lags among upstream, midstream, and downstream BH signals in their jurisdiction.

Interpreting data visualizations presents uncertainties (e.g., whether that spike happened by chance) that can make it more difficult to identify meaningful trends and pinpoint time frames where PH action may be needed. A statistical model, such as an interrupted time series analysis (ITSA), could be used to identify places with significant increases that depart from prior trends. Using ITSA on the data shown in Analysis 4.1, we found the following:

• Significant increases in the number of unemployment claims and the number of new SSRI prescription fills occurred in the month after the pandemic was declared.
• The notable increases in unemployment claims, as well as new SSRI fills among adults ages 18–39, have continued through 2021.
• The number of fatal overdoses did not start to significantly increase until the second quarter after the pandemic was declared; by the second quarter, the number of fatal overdoses had increased for adults ages 20–39.

Taken together, these trends in Los Angeles County suggest that BH needs increased and that PH action was needed, particularly among adults ages 18–39 due to significant increases in fatal overdoses and new SSRI fills. Although we are still learning about how peaks in different indicators correspond to short-, medium-, and long-term population-level BH impacts, this type of data triangulation (and clear communication of each source’s strengths and limitations) is an important part of interpreting these imperfect sources of data, particularly when communicating your findings to others.

Triangulating data can also be as complex as using machine learning to look for correlations across data sources (see Example 4.1) or conducting statistical modeling across multiple sources.

To estimate weekly suicide fatalities in the United States in near real time, Choi and colleagues (2020) applied a machine learning algorithm to multiple data sources and found that “the machine learner produced a real-time estimation of US suicide fatalities with meaningful correlation with week-to-week epidemiological trends and a less than 1 percent error compared with actual counts.” These data sources included ED records from the National Syndromic Surveillance Platform (NSSP); calls to the National Suicide Prevention Lifeline; calls to PCCs; information from the Consumer Price Index, unemployment rate, and home price index; web searches related to suicide; and social posts on Reddit, Twitter, and Tumblr related to suicide.

For a real-world example of how different data sources are being integrated into existing PH surveillance systems to allow for triangulation among sources in North Carolina, see Example 4.2 and Analysis 4.2.

To support timely public health surveillance in North Carolina, the North Carolina Division of Public Health—in collaboration with the Carolina Center for Health Informatics and the University of North Carolina Department of Emergency Medicine—developed NC DETECT. This surveillance system allows users to view data from EDs, state PCCs, EMS, and select urgent care centers. The system also features maps showing ED visit trends for select mental health conditions (anxiety, depression, self-inflicted injury, SI, and trauma/stressors) based on International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes. The maps in Analysis 4.2 show the annual ED visit for depression crude rate per 10,000 in 2018 and again in 2021. Even a quick visual comparison shows the increasing rates.

Source: Reproduced from NC Detect, undated.

For a real-world example from Indiana, see the Indiana Department of Health’s (DOH)’s Drug Overdose Dashboard (2022), which includes opioid prescription data, hospital discharges and inpatient data, and mortality data to guide county responses (Example 4.3). Finally, the overview of the NSSP contains additional examples.

This screenshot from Indiana Department of Health’s Drug Overdose Dashboard shows opioid use by county and a tab on opioid prescriptions for Henry County. This tab displays the 5 most commonly-used opioids in the county (acetaminophen with hydrocodone; buprenorphine with naloxone, tramadol, acetaminophen with oxycodone, and oxycodone). From 2017 to 2022 the dispensation rate per 1,000 population in Henry County is consistently higher than the rates across Indiana, hovering around 300 per 1,000 while the state’s rate has been gradually decreasing from just over 200 per 1,000 to just under this level.

Source:Reproduced from the Indiana Department of Health, 2022.

Taking Public Health Action

Assume you have just received a briefing from your colleagues on results from their analyses of BH surveillance data. You now must make some decisions about what to do next. Tool 4.1 shows just a few of your options for taking action, depending on the findings from BH surveillance in your jurisdiction. Note that one potential option for each type of BH indicator (upstream, midstream, and downstream) is to request additional analysis to further explore the detected signals. This step may be particularly important if the signal is “weak,” the findings are unexpected, and/or the PH response would require a substantial investment of limited resources.

If surveillance detects concerning signals in upstream indicators (such as UI claims), you might consider developing protective interventions to mitigate BH risks; advocating for policies that mitigate BH risks; allocating resources to addressing the social determinants of BH (e.g., increasing communication to the public on how to apply for UI, food stamps, or housing assistance); applying for, requesting, and/or advocating for additional funding and other resources to respond to quantified needs; and requesting or performing additional analysis to further explore findings (e.g., triangulating with other data sources). If surveillance detects concerning signals in midstream indicators (such as OTC sleep aid sales or 2-1-1 or PCC calls) or downstream indicators (e.g. ED visits for BH), consider alerting BH and other health care providers to prepare for possible surges in demand for services; alerting EMS so they can pre-position resources to respond to a potential surge in, for instance, opioid overdoses; communicating to the public about signs and symptoms of BH conditions and how to access resources; applying for, requesting, and/or advocating for additional funding and other resources to respond to quantified needs; and requesting additional analysis to further explore findings.

Example 4.4 describes how the Washington State Department of Health approached BH surveillance during COVID-19.

To inform planning and interventions to address the BH impacts of COVID-19 on behavioral health in Washington state, the Department of Health stood up a COVID-19 Behavioral Health Strike Team. This team used syndromic data from emergency departments on SI and suicide attempts to create BH impact situation reports and share them with state leaders and BH partners. In response to situation reports, the Department of Health has issued provider alerts on suicide and overdose risk and created county-specific maps to guide local public health plans and action. Additionally, the Department of Health began issuing COVID-19 BH impact situation reports for youth and older adults that included warnings or alerts based on weeks prior. Of note are the caveats in these reports, which caution readers that changes in overall ED visits (due to the stay-at-home order in March 2020) could skew the data and suggest that increases could also be due to an increased awareness of mental health impacts of the pandemic and not solely due to increased distress.

Addressing Uncertainty

Given the limitations of available data for BH surveillance, even after triangulating your findings among different data sources and/or using statistical methods to test for significant shifts in key BH indicators, you (and/or other decisionmakers) may be left with uncertainty around the best course of action to take. Maybe you are deciding how to respond to the local news media about whether there has been a spike in suicides among youth in your jurisdiction. Or maybe you have enough funding only to do a targeted messaging campaign to one subpopulation in your jurisdiction about how to seek BH support after a PHE. Who is most in need of receiving that targeted messaging?

To determine how to proceed with certain PH interventions or advise decisionmakers about taking action, you’ll need to consider several inputs simultaneously, such as the following:

• your confidence in the “strength of the signal” in BH surveillance data (e.g., the magnitude of the shift from baseline pre-PHE levels, the degree to which the shift exceeds your predetermined anomaly detection threshold, the statistical significance of the shift, or the precision around the estimate of that shift)
• partner and leadership/organizational priorities
• available resources, including funding, time, and staff capacity
• anticipated likelihood that the action will yield the desired benefits
• anticipated risks of acting on a weak or inaccurate “signal” in your data (e.g., you choose to devote resources to a particular problem or subpopulation, but later on, you realize you should have targeted resources somewhere else)
• potential consequences of waiting to take action until more data or resources are available.

This challenge of decisionmaking under uncertainty is not unique to BH surveillance (Courtney, Kirkland, and Viguerie, 1997; Pathak, Dewagan, and Mohanty, 2021; van Dorsser et al., 2020).

The next section discusses the importance of partnerships for collecting, interpreting, and acting on BH surveillance data.

Collaborating with Behavioral Health Partners and Others in Your Jurisdiction for Maximal Impact

The BH system in your community is as diverse as the community itself. Partners who are knowledgeable about and connected to BH organizations in your jurisdiction are essential to each step of the process shown in Summary 4.1. You will collaborate with these partners when conducting BH needs assessments, which help define the question to be answered or problem to be addressed through monitoring and surveillance; when interpreting your BH surveillance data and making decisions under uncertainty; when disseminating findings to key stakeholders across the BH and PH systems (including BH agencies and nonprofit organizations addressing BH needs, the media, and the public); and when planning and implementing interventions based on the findings.

Partnerships that include key organizations from the BH and PH systems, as well as other sectors in your community (see Summary 4.3) can be organized in a variety of ways and focused on your community at large or on a specific geographic area or population. Structures vary depending on the size of your community and variety of partner types and availability. A few common partnership approaches include

• forming a strike team to respond to a specific concern or to monitor BH after a specific event
• forming an ongoing steering or coordinating committee that can convene organizations that address BH needs and oversee one or more working groups to tackle emerging issues
• organizing a community forum (or a series of meetings) to gather input, reach out to additional partners, and mobilize community members
• leveraging partnerships (e.g., community coalitions, neighborhood groups, or task forces) to build on prior collaborations.

PH agencies need to work with BH partners, academic institutions, and other sectors to address BH. BH partners include residential, inpatient, and outpatient treatment providers; BH crisis responders; primary care providers; service providers for justice-involved populations and youth in foster care; organizations providing BH education and awareness; organizations focused on early prevention and early intervention; and community-based services. Other sectors include philanthropic and community development organizations; job preparation programs; education and youth development organization; public safety agencies; recreation and arts organization; and social service organizations and charities.

Note: Green circles represent organizations, services, and settings that make up the BH system in the United States. Public health agencies and their partners in academic institutions are shown in blue circles. Peach circles show examples of community-based organizations across multiple sectors with relevance to BH. Connections between organizations are included for illustrative purposes; they do not represent the only linkages that exist across this complex network of partners.

A Note About Health Equity

The Kirwan Institute offers guidance (Holley, 2016) on ways to ensure equitable and inclusive civic engagement, which is critical for the partnerships needed to collect, interpret, and act on surveillance data.

Summary and Looking Ahead

The final part of this toolkit summarizes the key challenges for BH surveillance and the emerging technologies that may present new opportunities for BH surveillance in the future.

Challenges with Behavioral Health Surveillance

BH surveillance efforts currently face several challenges, including the following:

• lack of standardization across data sources in how BH indicators are collected
• limited spatio-temporal granularity and timeliness of existing data sources to make them useful in the PHE context
• predominance of lagging indicators (as opposed to leading ones) of BH impacts of PHEs
• limited information about structural inequities and well-being, which are key factors related to BH.

Opportunities for Behavioral Health Surveillance

Encouragingly, emerging technologies and methods for more effectively accessing, integrating, and analyzing different data sources present opportunities to improve BH surveillance in the context of PHEs (NSSP, 2021a; Georgoulas-Sherry and Franzen, 2021). Examples include the following:

• The use of cloud-based data storage and transfer and the use of artificial intelligence and machine learning methods support using larger and more complex-data sets.
• Wearables and other mobile and ecological momentary assessment technologies allow for frequent and tailored data collection opportunities.
• Wastewater surveillance shows promise for monitoring levels of certain substances relevant to BH (e.g., opioids).
• Platforms for aggregating and visualizing data are improving. For example, the National Emergency Medical Services Information System (NEMSIS) has added BH indicators to its dashboard (NEMSIS, 2019), and the National Retail Data Monitor (NRDM) managed by the University of Pittsburgh has added OTC sleep aids to its data.

Renewed attention to, and additional resources for, widescale PH data modernization initiatives also are poised to accelerate progress in BH surveillance (and PH surveillance more generally) by updating core data and surveillance infrastructure at the federal, state, and local levels (U.S. Department of Health and Human Services and CDC, undated; Droegemeier et al., 2020; CDC, 2022a; CDC, 2022b; CDC, 2022d). In addition to government-led efforts, private and foundation partners are taking a leadership role in PH data transformation (National Commission to Transform Public Health Data Systems, 2021). Finally, innovative data science approaches are also being explored to improve upstream suicide prevention efforts (Amankwah, Pool, and Nass, 2022).

Summary 4.4 provides a high-level overview of some of these current challenges; the short-, medium-, and long-term aims of PH data modernization initiatives; and examples of relevant emerging technologies that, if and when they enter more widespread use, could inform real-time, proactive, equitable, and trustworthy PH decisions and action based on BH surveillance data.

Current BH surveillance relies on a patchwork of systems and data sources lead to lack of standardization and lower quality data. Most data sources lack the precision, granularity, and timeliness needed for public health officials to take action. Leading indicators or early warning signals are lacking. Emerging technologies like cloud-based data storage and transfer, unstructured data (e.g., social media, wastewater and other sensors), wearables and other ecological momentary assessment capabilities, artificial intelligence and machine learning, and “smart” cities and open data portals could help to strengthen BH surveillance. Data modernization efforts could also help strengthen BH surveillance. In the short term data modernization efforts aim to provide an expanded list of indicators and relevant data sources for BH surveillance; increase electronic reporting and enhancements to surveillance systems that capture BH indicators; build a workforce with improved data science, analytics, modeling, and informatics skills; and produce targeted real-time communication of findings from BH surveillance. In the medium term data modernization efforts aim to link data systems for real-time detection of BH impacts of PHEs to inform a timely response; build a highly skilled workforce that applies state-of-the art analytic methods and tools; and generate high quality information and guidance to promote BH. In the long term data modernization efforts aim to enhance understanding of emerging BH impacts of PHEs; allow for earlier detection of and intervention for BH conditions; advance technology that reduces collection and reporting burden on states; and support faster and more complete reporting on BH indicators. Together these data modernization efforts and emerging technologies could result in a BH surveillance system that informs real-time, proactive, equitable, and trustworthy decisions and action.

Conclusions

Developing strategies that use existing resources to conduct timely, high-quality, and actionable BH surveillance will require multiple types and sources of data. Each data source comes with trade-offs; local data availability, data quality, and the data science capacity of your organization will be key factors in determining which sources can and should be used for BH surveillance in the PHE context.