Controlling for Individual Heterogeneity in Longitudinal Models, with Applications to Student Achievement
Sep 4, 2007
Research and policies concerning primary and secondary school education in the United States are increasingly focused on student achievement test scores, in part because of the rapidly growing availability of data tracking student scores over time. These longitudinal data are currently being used to measure school and teacher performance, as well as to study the impacts of teacher qualifications, teaching practices, school choice, school reform, charter schools and other educational interventions. Longitudinal data are highly valued because they offer analysts possible controls for unmeasured student heterogeneity in test scores that might otherwise bias results. Two approaches are widely used to control for this student heterogeneity: fixed effects models, which condition on the means of the individual students and use ordinary least squares to estimate model parameters; and random effects or mixed models, which treat student heterogeneity as part of the model error term and use generalized least squares for estimation. The usual criticism of the mixed model approach is that correlation between the unobserved student effects and other educational variables in the model can lead to biased and inconsistent parameter estimates, whereas under the same assumptions, the fixed effects approach does not suffer these shortcomings. This paper examines this criticism in the context of longitudinal student achievement data, where the complexities of standardized test scores may create conditions leading to bias in fixed effect estimators. The authors show that under a general model for student heterogeneity in observed test scores, the mixed model approach can have a certain “bias compression” property that can effectively safeguard against bias due to uncontrolled student heterogeneity, even in cases in which fixed effects models may lead to inconsistent estimates. They present several examples with simulated data to investigate the practical implications of our findings for educational research and practice.