Use of Expert Ratings as Sampling Strata for a More Cost-Effective Probability Sample of a Rare Population

Published in: Public Opinion Quarterly, v. 73, no. 1, Spring 2009, p. 56-73

Posted on RAND.org on December 31, 2008

by Marc N. Elliott, Daniel F. McCaffrey, Judith F. Perlman, Grant N. Marshall, Katrin Hambarsoomian

Read More

Access further information on this document at Public Opinion Quarterly

This article was published outside of RAND. The full text of the article can be found at the link above.

The authors consider situations in which externally observable characteristics allow experts to quickly categorize individual households as likely or unlikely to contain a member of a rare target population. This classification can form the basis of disproportionate stratified sampling such that households classified as unlikely are sampled at a lower rate than those classified as likely, thereby reducing screening costs. Design weights account for this approach and allow unbiased estimates for the target population. We demonstrate that with sensitivity and specificity of expert classification at least 70 percent, and ideally at least 80 percent, our approach can economically increase effective sample size for a rare population. The authors develop heuristics for implementing this approach and demonstrate that sensitivity drives design effects and screening costs whereas specificity only drives the latter. The authors demonstrate that the potential gains from this approach increase as the target population becomes rarer. They further show that for most applications, unlikely strata should be sampled at 1/6 to 1/2 the rate of likely strata. This approach was applied to a survey of Cambodian immigrants in which the 82 percent of households rated unlikely were sampled at 1/4 the rate as likely households, reducing screening from 9.4 to 4.0 approaches per complete. Sensitivity and specificity were 86 percent and 91 percent, respectively. Weighted estimation had a design effect of 1.26, so screening costs per effective sample size were reduced by 47 percent. The authors also note that in this instance, expert classification appeared to be uncorrelated with survey outcomes of interest among eligibles.

This report is part of the RAND Corporation external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.