Power of Tests for a Dichotomous Independent Variable Measured with Error

Published In: HSR, Health Services Research, v. 43, no. 3, June 2008, p. 1085-1101

Posted on RAND.org on January 01, 2008

by Daniel F. McCaffrey, Marc N. Elliott

OBJECTIVE: To examine the implications for statistical power of using predicted probabilities for a dichotomous independent variable, rather than the actual variable. DATA SOURCES/STDAY SETTING: An application uses 271,479 observations from the 2000 to 2002 CAHPS Medicare Fee-for-Service surveys. STUDY DESIGN AND DATA: A methodological study with simulation results and a substantive application to previously collected data. PRINCIPLE FINDINGS: Researchers often must employ key dichotomous predictors that are unobserved but for which predictions exist. The authors consider three approaches to such data: the classification estimator (1); the direct substitution estimator (2); the partial information maximum likelihood estimator (3, PIMLE). The efficiency of (1) (its power relative to testing with the true variable) roughly scales with the square of one less the classification error. The efficiency of (2) roughly scales with the R2 for predicting the unobserved dichotomous variable, and is usually more powerful than (1). Approach (3) is most powerful, but for testing differences in means of 0.2-0.5 standard deviations, (2) is typically more than 95 percent as efficient as (3). CONCLUSIONS: The information loss from not observing actual values of dichotomous predictors can be quite large. Direct substitution is easy to implement and interpret and nearly as efficient as the PIMLE.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit www.rand.org/about/research-integrity.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.