Measuring Sustainment of an Evidence Based Treatment for Adolescent Substance Use

Published in: Journal of Substance Abuse Treatment, Volume 83, Supplement C (December 2017), Pages 55-61. doi: 10.1016/j.jsat.2017.10.005

by Wenjing Huang, Sarah B. Hunter, Bing Han, Mary Ellen Slaughter, Bryan R. Garner, Susan H. Godley

Read More

Access further information on this document at Journal of Substance Abuse Treatment

This article was published outside of RAND. The full text of the article can be found at the link above.


Measuring the extent a behavioral treatment is sustained in usual care practice settings after initial support for implementation ends is challenging. Oftentimes dichotomous self-report measures are used to measure health program sustainment, but these measures may not adequately capture the quality or extent of behavioral treatment delivery. Using data collected from community based organizations that received support to implement the Adolescent Community Reinforcement Approach (A-CRA), a measure of the extent of sustainment was derived.


Based on implementation theories and implementation support protocols, a total of 10 core treatment elements were identified to measure the extent of sustainment using information collected from key clinical staff. Item response theory (IRT) and principal component analyses (PCA) were used to further refine the 10 elements into composite measures of sustainment. The association between the 10 elements and the relationships between the comprehensive measures to a dichotomous sustainment measure were also examined.


Results from PCA identified two components from the initial 10 elements for measuring the extent of A-CRA treatment sustainment. The two components described different aspects of organizational support for A-CRA treatment implementation, one representing the quality or extent of treatment staffing and delivery and the other representing the quality of or extent of clinical supervision.


Using IRT and PCA, we were able to derive components that could be used to measure the extent of EBT sustainment and also better capture the quality of treatment delivery than the use of simple dichotomous measure. The methodological contribution of our study is that we have demonstrated a general analytic approach that may be applicable for other psychosocial treatments.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.