Assessing the Sensitivity of Treatment Effect Estimates to Differential Follow-Up Rates

Implications for Translational Research

Published In: Health Services and Outcomes Research Methodology, v. 12, nos. 2-3, June 2012, p. 84-103

Posted on RAND.org on January 01, 2012

by Beth Ann Griffin, Daniel F. McCaffrey, Rajeev Ramchand, Sarah B. Hunter, Marika Booth

Read More

Access further information on this document at Springer

This article was published outside of RAND. The full text of the article can be found at the link above.

We developed a new tool for assessing the sensitivity of findings on treatment effectiveness to differential follow-up rates in the two treatment conditions being compared. The method censors the group with the higher response rate to create a synthetic respondent group that is then compared with the observed cases in the other condition to estimate a treatment effect. Censoring is done under various assumptions about the strength of the relationship between follow-up and outcomes to determine how informative differential dropout can alter inferences relative to estimates from models that assume the data are Missing at Random. The method provides an intuitive measure for understanding the strength of the association between outcomes and dropout that would be required to alter inferences about treatment effects. Our approach is motivated by translational research in which treatments found to be effective under experimental conditions are tested in standard treatment settings. In such applications, follow-up rates in the experimental setting are likely to be substantially higher than in the standard setting, especially when observational data are used in the evaluation. We test the method on a case study evaluation of the effectiveness of an evidence-supported adolescent substance abuse treatment program (Motivational Enhancement Therapy/Cognitive Behavioral Therapy-5) delivered by community-based treatment providers relative to its performance in a controlled research trial. In this case study, follow-up rates in the community-based settings were extremely low (54 %) compared to the experimental setting (95 %) giving raise to concerns about non-ignorable drop-out.

This report is part of the RAND Corporation external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.