Cover: Intention-to-treat Analysis in Partially Nested Randomized Controlled Trials with Real-World Complexity

Intention-to-treat Analysis in Partially Nested Randomized Controlled Trials with Real-World Complexity

Published in: International Journal of Research & Method in Education, v. 39, no. 3, 2016, p. 268-286

Posted on Jun 10, 2016

by Jonathan Schweig, John F. Pane

Demands for scientific knowledge of what works in educational policy and practice has driven interest in quantitative investigations of educational outcomes, and randomized controlled trials (RCTs) have proliferated under these conditions. In educational settings, even when individuals are randomized, both experimental and control students are often grouped into particular classrooms and schools and share common learning experiences. Analyses that account for these clusters are common. A less common design involves one clustered experimental arm and one unclustered experimental arm, sometimes called a partially clustered design. Analysts do not always use methods that yield valid statistical inferences for such partially clustered designs. Additionally, published methods for handling partially clustered designs may not be flexible enough to handle real-world complications, including treatment non-compliance. In this paper, we illustrate how models that accommodate partial clustering may be used in educational research. We explore the performance of these models using a series of Monte Carlo simulations informed by data taken from a large-scale RCT studying the impacts of a programme designed to decrease summer learning loss. We find that clustering and non-compliance can have substantial impacts on statistical inferences about intent-to-treat effects, and demonstrate methods that show promise for addressing these complications.

This report is part of the RAND external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.