In typical community-based settings, manuals and training common to structured evidence-based practices may be sufficient to yield low levels of performance and moderate levels of fidelity, but systematic implementation support is needed to achieve high levels of performance and fidelity.

Cover: Can Implementation Support Help Community-Based Settings Better Deliver Evidence-Based Sexual Health Promotion Programs?

Can Implementation Support Help Community-Based Settings Better Deliver Evidence-Based Sexual Health Promotion Programs?

A Randomized Trial of Getting To Outcomes®

Published in: Implementation Science, v. 11, no. 1, May 2016, p. 78-93

Posted on RAND.org on July 05, 2016

by Matthew Chinman, Joie D. Acosta, Patricia A. Ebener, Patrick S. Malone, Mary Ellen Slaughter

Research Questions

  1. How much support from Getting to Outcomes (GTO) did sites receive in this randomized controlled trial of an evidence-based teen pregnancy prevention program?
  2. After two years, how did GTO affect sites' performance (i.e., were key programming practices carried out) and the fidelity with which they implemented the program?
  3. Is there empirical support for the theory that GTO support during implementation predicts performance, and consequently program fidelity?

Background

Research is needed to evaluate the impact of implementation support interventions over and above typical efforts by community settings to deploy evidence-based prevention programs.

Methods

Enhancing Quality Interventions Promoting Healthy Sexuality is a randomized controlled trial testing Getting To Outcomes (GTO), a 2-year implementation support intervention. It compares 16 Boys and Girls Club sites implementing Making Proud Choices (MPC, control group), a structured teen pregnancy prevention evidence-based program with 16 similar sites implementing MPC augmented with GTO (intervention group). All sites received training and manuals typical for MPC. GTO has its own manuals, training, and onsite technical assistance (TA) to help practitioners complete key programming practices specified by GTO. During the first year, TA providers helped the intervention group adopt, plan, and deliver MPC. This group then received training on the evaluation and quality improvement steps of GTO, including feedback reports summarizing their data, which were used in a TA-facilitated quality improvement process that yielded revised plans for the second MPC implementation. This paper presents results regarding GTO's impact on performance of the sites (i.e., how well key programming practices were carried out), fidelity of MPC implementation, and the relationship between amount of TA support, performance, and fidelity. Performance was measured using ratings made from a standardized, structured interview conducted with participating staff at all 32 Boys and Girls Clubs sites after the first and second years of MPC implementation. Multiple elements of fidelity (adherence, classroom delivery, dosage) were assessed at all sites by observer ratings and attendance logs.

Results

After 2 years, the intervention sites had higher ratings of performance, adherence, and classroom delivery (dosage remained similar). Higher performance predicted greater adherence in both years.

Conclusions

These findings suggest that in typical community-based settings, manuals and training common to structured EBPs may be sufficient to yield low levels of performance and moderate levels of fidelity but that systematic implementation support is needed to achieve high levels of performance and fidelity.

Key Findings

  • Sites that received training in an evidence-based teen pregnancy prevention program plus GTO carried out key programming tasks (e.g., planning, evaluation) better than sites receiving the program's training without GTO.
  • GTO sites carried out the program with near perfect fidelity by Year 2.
  • Non-GTO sites adhered to the program's delivery standards only about half the time.
  • GTO sites were rated as having better program delivery (enthusiasm, control) than non-GTO sites.

Recommendations

  • In typical community-based settings, manuals and training common to structured evidence-based practices may be sufficient to yield low levels of performance and moderate levels of fidelity, but systematic implementation support is needed to achieve high levels of performance and fidelity.
  • Funders of evidence-based programs should build in funds for such supports.
Access further information on this document at Implementation Science

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.