2: Finding Existing Data for Behavioral Health Surveillance

As mentioned in Section 1, no dedicated surveillance systems exist that track BH in the context of PHEs. Thus, PH officials rely on a patchwork of existing surveys and surveillance systems that all have limitations, including long lags between data collection and release, nonrepresentative sampling, and lack of validated measures (CSTE, 2013). Despite these challenges, some existing data sources could be useful for monitoring and surveillance of selected BH indicators in PHEs. The following sections introduce those BH indicators and the data sources where they can be found.

Types of Behavioral Health Indicators

As the conceptual model in Summary 2.1 shows, assessing BH in this specific context can include monitoring and surveillance of

• upstream community strengths, vulnerabilities, and other social determinants of BH that influence the likelihood that a PHE will affect the BH of a population (e.g., unemployment claims) (Alegria et al., 2018)
• midstream indicators, or early signals that the PHE is affecting BH (e.g., over-the-counter [OTC] sleep aid sales, calls to 2-1-1 for community services or to suicide and crisis hotlines like 9-8-8)
• downstream indicators of a PHE’s later impacts on BH (e.g., emergency room visits for BH crises).

Upstream indicators will provide early signals that PH interventions may be needed to prevent downstream BH impacts; downstream indicators signal whether the BH problem or crisis is getting worse or improving.

There are three types of BH indicators. The first type is upstream indicators, defined as community strengths and vulnerabilities that influence the likelihood of BH impacts (such as unemployment claims). The second type is midstream indicators, defined as early signals of BH impacts that could be mitigated by prompt action (such as calls to 2-1-1 or poison control centers). The third type is downstream indicators, defined as later BH impacts that show BH is worsening, improving, or unchanged (such as emergency room visits for BH). Upstream indicators are useful if you want to understand community strengths or vulnerabilities that may influence BH in your community or if you want to identify indicators of the risks that may increase BH impacts if a PHE occurs. Midstream indicators are useful if you want to determine the kinds of impacts that a recent PHE had on your community or if you want to track early signals that a PHE may be negatively impacting your jurisdiction. Downstream indicators may be useful if you want to understand if BH is worsening or improving in your community or to assess the effectiveness of a public health intervention to address BH.

Answer the questions in Tool 2.1 to determine what type of BH indicators to track, then review the full conceptual model (click on the button) for a comprehensive list of potential indicators to consider. That list contains indicators that may not be available in all jurisdictions across the United States, but we provide them in the event that your PH agency can access data on these indicators from a local or regional source.

Use upstream indicators (e.g., unemployment claims) to understand community strengths or vulnerabilities that might influence BH in your community and to identify indicators of risks of BH impacts if a PHE occurs. Use midstream indicators (e.g., calls to 2-1-1 and Poison Control Centers) to determine the kinds of BH impacts a recent PHE may be having in your community. Use downstream indicators (e.g., emergency department visits for BH) to understand if BH is worsening or improving in your community or to assess the effectiveness of a public health intervention to address BH.

View the conceptual model

Once you know which types of BH indicator(s) you are interested in, the next step is to identify the source(s) of the data you will use.

Use Tool 2.2 to identify BH indicators that you might be interested in (left side of the figure) and relevant potential data sources (right side of the figure) that you might have access to. Note that the tool can be read from left to right or from right to left, depending on whether you are starting with the indicator(s) you want to track or the data source that you want to use to answer your questions about BH in your jurisdiction.

Unemployment insurance claims are one data source for upstream indicators of community strengths and vulnerabilities that influence the likelihood of BH impacts. Examples of data sources of midstream indicators or early signals of BH impacts that could be mitigated by prompt action are data from 2-1-1 calls, PCC calls, prescription medication fills, OTC sleep aid sales, and suicide and crisis hotline calls. Finally, emergency department visits and emergency medical services activations are data sources for downstream indicators of later behavioral impacts that suggest BH is worsening, improving, or unchanged.

Note: PCC = poison control center, ED = emergency department, EMS = emergency medical services.

For this toolkit, we identified data sources for further exploration by determining whether the source had three attributes:

1. The data source is available to PH agencies at the sub-state level (e.g., city or county).
2. The data are made available more frequently than once a year.
3. The data are available in most jurisdictions across the United States.

These attributes were deemed critical for the data source to be timely enough to help a PH agency understand the rapidly changing population-level BH impacts of a PHE in its jurisdiction. Ideally, data sources would also provide population characteristics (such as age, race, disability status, or income) to help identify inequitable impacts related to intergenerational and historical inequities, but few (if any) contain complete and reliable data on these characteristics. Of the 27 data sources we reviewed, we identified eight with the three attributes above. (For the rest of this toolkit, we refer to these as the most-promising data sources). To learn more about the most-promising data sources for conducting BH surveillance and how you can access them, choose from the buttons below.

About the Most-Promising Data Sources

Upstream

Unemployment insurance (UI) Claims

Midstream

2-1-1 calls PCC calls Prescription medication fills OTC sleep aid sales Suicide and crisis hotlines

Downstream

ED visits EMS activations

Data Sources We Considered but Do Not Highlight Because of Limitations

Table 2.1 shows administrative data sources, survey data sources, and others (such as web searches and social media data) that we considered but do not highlight in this toolkit, given the specific focus on the use of existing data to monitor BH impacts of PHEs. As of 2023, each of these sources has characteristics that limit their utility for the specific purpose of conducting BH monitoring and surveillance in the PHE context. For instance, they may not have sufficient spatial granularity (i.e., they are available only at the national level); they may not be sufficiently timely (i.e., they are released only annually); or they may not be feasible to obtain on a regular basis for ongoing monitoring. That said, there may be steps you can take (not covered in this toolkit) to make some of these data sources more available for your use in future PHEs.

Administrative Records

• Drug Enforcement Agency Automated Reports & Consolidated Ordering System
• Disaster Distress Helpline
• Mortality data (i.e., vital records)
• National Child Abuse and Neglect Data System
• National Domestic Violence Hotline
• National Highway Traffic Safety Administration Fatality Analysis Reporting System
• National Violent Death Reporting System
• State all-payer claims
• State Medicare/Medicaid claims
• State workers' compensation claims

Survey Data

• Behavioral Risk Factor Surveillance System (BRFSS)
• PHE-specific survey (e.g., the Census Household Pulse Survey)
• PLACES
• National Health Interview Survey
• National Survey of Children’s Health
• National Survey on Drug Use and Health
• Youth Risk Behavior Surveillance System

Digital Trace Data

• Data from web-scraping (e.g., news media websites)
• Social media data (such as Twitter mentions)
• Web searches (i.e., GoogleTrends)

Importantly, many of the data sources in Table 2.1 have been used as parts of extensive efforts across the United States to improve surveillance of BH during routine times. One notable example is the Council of State and Territorial Epidemiologists’ work to standardize indicators for BH surveillance (Hopkins, Landen, and Toe, 2018). However, the indicators in these data sources are not available frequently enough to be useful in the PHE context.

Access selected publications that have used web searches for monitoring and surveillance.

Comparing Promising Data Sources: Qualitative Ratings on Attributes

To help you compare the eight most-promising data sources, we documented their attributes (described in Table 2.2) after reviewing peer-reviewed articles; talking to PH officials, data producers, experts, and stakeholders; and performing a series of exploratory analyses.

Attributes of Data Sources with Descriptions

Attribute	Description
Spatio-temporal granularity of data source	Geographic and time bounds are needed for a statistically valid surveillance estimate. In most cases, data on an individual at a certain point in time are aggregated into a summary of data for groups of individuals over time. A rule of thumb for defining a statistically valid spatio-temporal unit is to aggregate data on individuals into groups of similar geographic area and/or population size, with at least 20 individuals in any one given unit.
Representativeness of data source	The population sampled or collected by the data producers is within the expected spatio-temporal unit.
Timeliness to acquire data source when needed	There is no lag in the time required for data processing and delivery to user.
Data science capacity requirements to analyze data source	Data science staff are available and have the expertise, the computing and data analytic infrastructure, and the data acquisition and analytics budget needed to analyze the data source.

Source: Adapted from Azofeifa et al., 2018.

Then, we rated each data source according to these attributes as most optimal (green), mixed (yellow), or least optimal (red) (Table 2.3).

Attribute Rating Criteria

Attribute	Most Optimal	Mixed	Least Optimal
Spatio-temporal granularity of data source	High spatio-temporal granularity (e.g., weekly county estimates)	Medium spatio-temporal granularity (e.g., quarterly county estimates)	Low spatio-temporal granularity (e.g., only state-level estimates)
Representativeness of data source	Likely to be representative of key subpopulations of interest (e.g., Census, registry, population-representative sample)	Somewhat representative of total population (e.g., representative by race/ethnicity and gender but not at the intersection of race/ethnicity and gender)	Not very representative of total population
Timeliness to acquire data when needed	Short (i.e., days)	Medium (i.e., months or weeks)	Long (i.e., a year or longer)
Data science capacity requirements to analyze data source	Does not require advanced data analysis or statistical expertise (i.e., limited data science capacity required)	Some need for data science capacity (e.g., to smooth data before visualizing)	Requires advanced data analysis or capacity (e.g., use of statistical software, such as STATA or R)

A Note About Health Equity

Ideally, we would have included an attribute that captures the data source's utility for assessing BH equity—for example, an attribute containing information on systemic access barriers that contribute to behavioral health inequities or one that allows tracking of BH impacts at the intersection of population characteristics. But new sources of data and surveillance methods are needed to better capture equity (Chandra et al., 2022).

Table 2.4 displays our qualitative ratings for the eight most-promising data sources. Qualitative ratings for the remaining sources, not highlighted in the toolkit, are available in Appendix C.

Attributes of the data sources

Data Source	Spatio-Temporal Granularity of Data Source	Representativeness of Data Source	Timeliness to Acquire Data Source When Needed	Data Science Capacity Requirements to Analyze Data Source
UI claims	Most optimal / promising +++	Most optimal / promising +++	Most optimal / promising +++	Most optimal / promising +++
2-1-1 calls	Most optimal / promising +++	Least optimal +	Most optimal / promising +++	Most optimal / promising +++
PCC calls	Most optimal / promising +++	Least optimal +	Most optimal / promising +++	Most optimal / promising +++
Prescription medication fills	Most optimal / promising +++	Most optimal / promising +++	Most optimal / promising +++	Mixed ++
OTC sleep aid sales	Most optimal / promising +++	Least optimal +	Most optimal / promising +++	Most optimal / promising +++
Suicide and crisis hotline calls (e.g., 9‑8‑8)	Most optimal / promising +++	Least optimal +	Most optimal / promising +++	Most optimal / promising +++
ED visits	Most optimal / promising +++	Mixed ++	Most optimal / promising +++	Mixed ++
EMS activations	Most optimal / promising +++	Most optimal / promising +++	Most optimal / promising +++	Most optimal / promising +++

Table Legend
Most optimal/promising	+++
Mixed	++
Least optimal	+

A Note About Social Media

Although this toolkit does not focus on the use of social media data for conducting BH monitoring and surveillance, certain social media platforms may have potential for use in BH surveillance, particularly if social media companies are able to make data more accessible for different types of analyses. Facebook traces have been shown to be a reliable and cost-effective source for understanding health outcomes and behaviors (Gittelman et al., 2015). Less research has focused on Facebook data-mining in relation to disasters, but such data may be useful for understanding baseline data and trends in mental health. For example, PH professionals can use Facebook status updates, Facebook friends, and linguistic features to predict trends in depression symptoms (De Choudhury, Counts, and Horvitz, 2013). A team of programmers and researchers recently developed a personality quiz for Facebook users, but to take the quiz, users had to opt in to share their “likes” and status updates (Schwartz, Eichstaedt, and Kern, 2014). Theoretically, PH officials could compare the quiz results with Facebook status updates to understand population-level depression prevalence using the results and the linguistic composition of posts.

In addition, researchers at North Carolina State University are developing a publicly available Twitter Sentiment Analysis tool (Healey and Ramaswamy, 2022). A recent study overlaid Twitter conversational topics onto state-level mental distress estimates from the BRFSS. Although the BRFSS is not a timely resource for monitoring PHE-associated distress, understanding Twitter conversational topics may help guide future BH surveillance via the platform. For example, the researchers aggregated Twitter conversation data to identify key topics indicating the presence of distress, such as death, illness, or injury. Finally, researchers have examined Twitter sentiment after incidents of mass violence and have described the challenges and benefits of working with Twitter data (Jones, Brymer, and Silver, 2019). On the whole, the required data science capacity and inconsistent data on the location of Twitter users may limit the utility of Twitter data for BH surveillance in PHEs (Bowen et al., 2020).

Finally, the popularity of social networking sites continues to evolve, with such platforms as TikTok and Reddit gaining prominence. These sites are beginning to be used as places where youth seek advice or express psychological distress and, to a lesser extent, where PH officials engage with online communities. Their utility for BH surveillance (in PHEs or in routine times) has not yet been demonstrated (McCashin and Murphy, 2022).

Access selected publications that have used social media data for monitoring and surveillance.

A Note About Monitoring BH Impacts on Children

If you are particularly interested in monitoring BH impacts among children and adolescents, you may be able to access administrative data from free-standing pediatric hospitals through the Pediatric Health Information System database (Children’s Hospital Association, undated), which releases data quarterly, or the SchoolCare (undated) platform, which collects data from school nurses in some kindergarten through 12th grade (K–12) settings. Absenteeism rates (among not only students but also school staff) also might point to potential BH impacts. The Public Health Informatics Institute (undated) has a playbook highlighting ways that PH agencies can form partnerships to assess child and adolescent mental health.

The next section of the toolkit discusses in more detail how you can use your existing sources of data to monitor BH impacts of PHEs along with the upstream strengths and vulnerabilities that protect communities from, or predispose them to, those impacts.