RAND Statistics Seminar Series

Robust Standard Errors and Experimental Design

Presented by Larry V. Hedges, Northwestern University
Thursday December 6, 2007
4:30pm ET/1:30pm PT
Carnegie Mellon University
Hamerschlag Hall, Room B131
Pittsburgh, PA
Please contact Denise Miller if you would like to attend this seminar.


To address the problem testing coefficients in regression analysis with heteroscedastic errors, robust standard errors have been introduced in statistics and are widely used in econometrics. They give test statistics that are correct in large samples, even when error variances are heterogeneous. Robust standard errors are often employed in the analysis of data from randomized field experiments. Of course, standard analyses based on the analysis of variance (ANOVA) for various experimental designs are also available. I explore the use of tests based on robust standard errors for three commonly used designs in social experiments: the completely randomized design, the hierarchical (cluster randomized) design, and the generalized randomized blocks (matched within clusters) design. In some cases these robust tests correspond (almost) to familiar tests, but not always to the ANOVA tests. Consequently the robust statistics do not always have the expected sampling distribution. Sampling distributions of tests for treatment effects are derived and used to evaluate actual rejection rates. For example, in the generalized randomized blocks design with block fixed effects, the robust test for treatment effects may reject for more often than expected when the null hypothesis is true. These results give some insight into the robust tests and the robustness of the usual ANOVA tests. Some recommendations on choice of test statistics are given in light of these findings.

Speaker Bio

A national leader in the fields of educational statistics and evaluation, Larry V. Hedges is a Board of Trustees Professor at Northwestern. He is a fellow of several professional organizations, widely published and convener of the Campbell Collaboration's statistics group, which is part of a larger effort to produce an online database of "best practices" in the social sciences and education. Examples of some his recent studies include: understanding the costs of generating systematic reviews, differences between boys and girls in mental test scores, the black-white gap in achievement test scores, and frameworks for international comparative studies on education.

Attending a Seminar

Other Locations/Times:
Washington, D.C. Conf. Rm. 4302: 4:30pm ET
Santa Monica Conf. Rm. 3312: 1:30pm ET

Visitors to RAND's Santa Monica and Pittsburgh locations are welcome to attend and must RSVP at least one day prior to the seminar. To ensure your attendance please contact Denise Miller at dmiller@rand.org with your name, company (or university) affiliation, and national citizenship (for security purposes).

For parking and directions to RAND's Santa Monica office, please see: http://www.rand.org/about/locations/santa-monica.html.

For parking and directions to RAND's Pittsburgh office, please see: http://www.rand.org/about/locations/pittsburgh.html.

For further information and to be added to the mailing list contact Denise Miller at dmiller@rand.org.