A practical guide to implementing rerandomisation in education trials

Red raffle game with different colored balls for bingo and lottery games, photo by Peter Ekvall/Adobe Stock

Peter Ekvall/Adobe Stock

Researchers provide practical guidance on how to implement rerandomisation as part of education randomised controlled trials, which may be needed to analyse education interventions.

What is the issue?

Randomised controlled trials (RCTs) are a rigorous design for assessing the causal impact of policies and interventions. They are often used in situations where a study’s internal validity is threatened, as they can improve covariate balance and lead to more precise estimates of the treatment effect.

In many cases, RCTs are used in situations where there is a substantial risk that the realised random assignment leads to an imbalance on covariates of interest, which threatens the internal validity of the study. In that case, rerandomisation may be appropriate, as it can help to minimise imbalance.

Rerandomisation, a method developed in 2012 by Kari Lock Morgan and Donald B. Rubin, may be particularly valuable in education research settings, since the method relies on the availability of covariate information at the experiment’s design stage.

How did we help?

RAND Europe and RAND Education Labor produced a guide to assist researchers conducting rerandomisation in RCT evaluations of education programmes.

The guide provides practical assistance on implementing rerandomisation as part of an RCT, and how to correctly analyse the subsequent data. In doing so, we highlight the benefits and drawbacks of rerandomisation relative to other methods, such as stratification.

What did we find?

The report outlines a step-by-step procedure for implementing rerandomisation and illustrates these steps using a hypothetical example. The steps are as follows:

Step 1: Assess whether rerandomisation is feasible

  • Are pre-randomisation data available?
  • Is it necessary to balance one or more key variables?
  • Is the sample size small?

Step 2: Decide which covariates need to be balanced

  • This is context-specific, but relevant characteristics are predictive of the outcomes of interest.

Step 3: Assess whether rerandomisation is necessary through simulation

  • Conduct several simple random assignments and use the results to assess how severe the potential for imbalance may be without rerandomisation.
  • If randomisations often result in imbalance, rerandomisation may be preferable to simple random assignment.

Step 4: Decide on the criteria for acceptable balance

  • Select the measure of similarity between the treatment and control group.
  • Determine the degree of dissimilarity allowed.

Step 5: Randomise subjects to treatment and control groups and rerandomise until the criteria for acceptable balance are met

Step 6: Analyse the results using a randomisation test

  • Ensure an unbiased estimate and define the estimand of interest.
  • Generate appropriate standard errors.