Challenges to Social Experiments

Evaluations of school-based drug prevention programs have been plagued by problems that limited the validity of their findings. These limitations pose clear challenges for current prevention research. This report describes how a recent multi-site experiment conducted in 30 junior high schools met several of those challenges, including evaluating the program in a variety of environments, achieving well-balanced experimental groups, implementing the program as designed, obtaining reliable outcome measures, and eliminating alternative explanations for the results. In most cases, multiple strategies were employed. Extensive analyses were conducted to assess how well the strategies worked; they indicated that each obstacle was overcome. This success implies several lessons for future experiments that are generally applicable to field studies conducted with schools and other organizations. Of particular importance are the guidelines for recruiting institutions from diverse communities and maintaining their cooperation over several years and the techniques for facilitating faithful program delivery and monitoring the implementation process. Recommended design and analysis features include using more than simple random assignment to achieve a balanced design, and employing control variables to rule out alternative explanations of the results--even under conditions of substantial pre-treatment equivalence.