Download Free Electronic Document

FormatFile SizeNotes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 7.0 or higher for the best experience.

Research Brief

Key Findings

  1. Almost 1,500 families received Safe Start services (and about 1,250 received comparison group services) across ten sites.
  2. Analyses found positive change in many outcomes, with families in six of the studies showing statistically significant improvement in their primary outcomes over time.
  3. However, there was no strong evidence that the intervention groups improved more than the comparison groups on the outcomes examined.
  4. Taken as a whole, the results suggest that more information is needed about what interventions help children exposed to violence.

Although rates of children's exposure to violence have been declining in the United States, the problem remains extensive. The most recent study found that more than half of children in a national sample had been exposed to violence in the past year.[1] Children who have been abused or witnessed violence are more likely than other children to develop mental health problems and engage in risky behaviors. Some of these problems can persist into adulthood.

The need is clear for interventions to help children deal with these harmful effects. Yet the evidence base for understanding which interventions work best under different circumstances is still developing. Some interventions have proven effective for specific kinds of violence or specific symptoms; others, although they show signs of promise, have been difficult to implement in real-world settings.

Safe Start Promising Approaches is a large-scale initiative intended to test how effectively interventions in community settings can help children deal with the effects of exposure to violence. The initiative was developed by the federal Office of Juvenile Justice and Delinquency Prevention (OJJDP). At first, OJJDP selected 15 sites to implement promising interventions, and RAND evaluated both implementation processes[2] and outcomes.[3] Extending this effort to test more interventions, OJJDP selected an additional ten sites in 2010 and asked RAND to evaluate outcomes. The sites varied by size, location, age range served, and types of violence exposure, with each proposing an intervention to fit the needs of its target population (see Table 1). Each site identified a primary outcome that best matched the expected impact of the intervention. The evaluation used experimental or quasi-experimental designs to examine whether Safe Start interventions led to child and caregiver improvements in a range of outcomes, with each site selecting one outcome as primary, depending on the goal of its intervention.

Table 1. Safe Start Study Characteristics and Evaluation Designs

Study Location Intervention Component and Design Primary Outcome Age Range Served, in Years
Aurora, Colo. RCT: Enhancement to TF-CBT compared with TF-CBT alone Child PTSD 5–14
Aurora, Colo. RCT: Enhancement to TF-CBT compared with TF-CBT alone Treatment retention 5–14
Denver, Colo. RCT: Strengthening Family Coping Resources compared with usual probation services while on wait list Positive involvement 0–17
Detroit, Mich. RCT: Group therapy and case management compared with family nutrition groups and case management Family conflict 3–16
El Paso, Tex. RCT: Group therapy and case management compared with case management alone Child self-control 3–14
Honolulu, Hawaiia Quasi-experimental: Enhancement to an existing group therapy plus individualized clinical child assessment and individual and family therapy compared with usual services Child behavior problems 3–17
Kalamazoo, Mich.a Quasi-experimental: Adaptation to an existing group therapy compared with usual community services Positive involvement 8–17
Philadelphia, Pa. RCT: Individual home-based therapy and EHS services compared with EHS alone Caregiver depression 0–3
Queens, N.Y.a RCT: Intensive dyadic therapy compared with wait list Child PTSD 5–17
Spokane ARC, Wash. RCT: Individual therapy within Head Start compared with Head Start alone Child cooperation, assertion, and self-control 3–5
Spokane COS, Wash. RCT: Group and individual therapy within Head Start compared with Head Start alone Child cooperation, assertion, and self-control 3–5
Worcester, Mass. Quasi-experimental: Child assessments and service plans plus group therapy in homeless shelter compared with usual shelter services alone Child social–emotional well-being, assertion, and self-control 0–18

NOTE: RCT = randomized controlled trial. TF-CBT = Trauma-Focused Cognitive–Behavioral Therapy. PTSD = posttraumatic stress disorder. EHS = Early Head Start. ARC = Attachment, Self-Regulation, and Competency. COS = Circle of Security.

a Because of implementation problems, this site did not complete the study.

Evaluation

Enrolling and retaining families in these studies proved challenging. Not all sites were able to recruit and retain enough families for meaningful analysis of effects. The analysis therefore sorted the sites into three groups. Four sites (Aurora for treatment retention, Detroit, El Paso, and Worcester) had enough participants for the analysis to draw conclusions about whether the intervention had the expected effect. Five sites (Aurora for child outcomes, Denver, Philadelphia, Spokane ARC, and Spokane COS) did not have enough participants to do this. At three sites (Hawaii, Kalamazoo, and Queens), the studies could not be completed because of implementation problems.

Results

Program Reach

Almost 1,500 families received intervention services (and about 1,250 received comparison group services) across all sites. This large number of families served shows the feasibility of engaging families in community-based behavioral health and supportive services. Safe Start also reached vulnerable populations. Violence exposure among families served had averaged 1.4 experiences in the prior six months, and, overall, 31 percent of children had PTSD symptoms. However, in many cases, families did not take up the services fully, receiving fewer Safe Start services than planned. At some sites, families had competing demands and challenges, making it difficult to engage them in both services and a study. Nonetheless, family satisfaction with services was high.

Program Capacity

Most Safe Start sites succeeded in launching interventions intended to help children exposed to violence, adding needed service capacity to their local communities, where families might not otherwise have had access to these services. Agencies also developed procedures for identifying children exposed to violence and integrated trauma-informed approaches into their usual care. In many communities, the Safe Start programs also conducted outreach and provided awareness training on trauma-informed care to other social service agencies in their communities, improving communication and coordination among service providers and increasing knowledge about working with children and families exposed to violence. With greater connections with other providers, the agencies implementing Safe Start programs were able to establish new interagency and communitywide partnerships to address service gaps for children and their families.

Table 2. Intervention Effects for Selected Outcomes at Six Months

  • Primary outcome
  • Statistically significant
  • No effect
Study Location CR Child PTSD SR Child PTSD CR Positive Involvement CR Caregiver Depression CR Child Self-Control CR Family Conflict CR Child Total Behavior Problems
Aurora, Colo. Large Large Small Small Medium Small Large
Denver, Colo.       Small   Small Small
Detroit, Mich. Small Small Small Small Small Small Small
El Paso, Tex. Medium Medium Small Small Medium Medium Medium
Philadelphia, Pa.       Small   Small  
Spokane ARC, Wash. Small     Small Small Small Small
Spokane COS, Wash. Small     Small Small Small Small
Worcester, Mass. Small Small Medium (negative) Small Small Small Small

NOTE: CR = caregiver report. SR = child self-report.

Program and Intervention Effects

In general, families who participated in the Safe Start study (in both the intervention and comparison groups) improved over time. Positive gains followed in many of the outcomes regardless of intervention group assignment. Families in six of the studies showed statistically significant improvement in their primary outcomes over time (in the intervention or comparison group).

However, for the sites with large enough samples to allow analysis of intervention effects, the analysis did not find strong evidence that the intervention groups improved more than the comparison groups on the outcomes examined. In Detroit and El Paso, for example, there was no evidence that the intervention (the Strengthening Families Program and its cultural adaptation) was more effective than case management and supportive group counseling. Case management might have been successful in these sites because this approach to support families allowed program staff to assess and meet immediate family needs. Aurora showed large improvements in both the treatment and comparison groups and therefore showed no evidence that the strategic enhancement improved outcomes beyond the TF-CBT provided to both groups. For Philadelphia and Spokane, Safe Start was embedded within existing and robust programs for families, which likely made it difficult to see differential change between the groups that could be attributed to the Safe Start intervention.

Although changes were positive among those who received Safe Start services (intervention group), there was a range of within-group (or effect size) changes from baseline to the six-month follow-up across the outcomes examined. There were relatively few significant large or medium effect size changes, with most changes in outcomes small and nonsignificant.[4] Only Aurora produced large improvements, particularly in PTSD symptoms, within its intervention group. Aurora's strategic enhancement to a proven treatment model produced large significant effects on measures of child PTSD symptoms and on total child behavior problems. Within its intervention group, El Paso's cultural adaptation of the Strengthening Families Program and case management produced medium, significant improvements in its primary outcome of child self-control, both CR and SR of child PTSD, caregiver depression, family conflict, and child total behavior problems. All the other studies produced small effects on outcomes within the intervention group from baseline to the six-month follow-up.

Conclusion and Next Steps

One purpose of Safe Start Promising Approaches was to improve the evidence base for interventions directed toward children exposed to violence. Although the initiative added to our knowledge about how to address the problem, there was no clear case for using a particular intervention to help these children and their families. It is clear, though, that high-intensity services, such as those offered in the Aurora Safe Start program, can work for children with PTSD following violence exposure, but these services are not routinely available to families and not all children have these symptoms. In addition, intense evidence-based interventions are not necessarily easily implemented in community settings, as seen in Queens. Among the sites that offered lower-intensity community-based services, the two sites with adequate sample sizes to detect medium-sized improvements — Detroit and El Paso — showed that both intervention and comparison families improved. This finding suggests that supportive social services can be helpful to families, regardless of the intensity and type of services.

Taken as a whole, the results suggest that more information is needed about what does — and what does not — work for children exposed to violence. Funders and policymakers should consider supporting evaluations of interventions at three levels:

  • dissemination of evidence-based intensive services into community settings, so that the expected effects can be realized in real-world settings
  • supportive and mental health early interventions geared toward helping families and children with mild to moderate symptoms
  • community and agency prevention efforts geared toward improving family and child resilience in the face of violence.

In conclusion, the need for multiple levels and settings for interventions for children exposed to violence is clear, but the challenge remains finding the most-effective and feasible ways to do this.

Notes

  • [1] David Finkelhor, Anne Shattuck, Heather A. Turner, and Sherry L. Hamby, "Trends in Children's Exposure to Violence, 2003 to 2011," JAMA Pediatrics, Vol. 168, No. 6, June 2014, pp. 540–546; David Finkelhor, Heather A. Turner, Anne Shattuck, and Sherry L. Hamby, "Prevalence of Childhood Exposure to Violence, Crime, and Abuse: Results from the National Survey of Children's Exposure to Violence," JAMA Pediatrics, Vol. 169, No. 8, August 2015, pp. 746–754.
  • [2] Dana Schultz, Lisa H. Jaycox, Laura J. Hickman, Anita Chandra, Dionne Barnes-Proby, Joie Acosta, Alice Beckman, Taria Francois, and Lauren Honess-Morreale, National Evaluation of Safe Start Promising Approaches: Assessing Program Implementation, Santa Monica, Calif.: RAND Corporation, TR-750-DOJ, 2010.
  • [3] Lisa H. Jaycox, Laura J. Hickman, Dana Schultz, Dionne Barnes-Proby, Claude Messan Setodji, Aaron Kofner, Racine Harris, Joie Acosta, and Taria Francois, National Evaluation of Safe Start Promising Approaches: Assessing Program Outcomes, Santa Monica, Calif.: RAND Corporation, TR-991-1-DOJ, 2011.
  • [4] Effect size is commonly considered small if it is about 0.2 standard deviations or less, medium if it is about 0.5 standard deviations, and large if it is about 0.8 standard deviations or more.

This report is part of the RAND Corporation research brief series. RAND research briefs present policy-oriented summaries of individual published, peer-reviewed documents or of a body of published work.

Permission is given to duplicate this electronic document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND PDFs to a non-RAND Web site is prohibited. RAND PDFs are protected under copyright law. For information on reprint and linking permissions, please visit the RAND Permissions page.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.