Use of Expert Ratings as Sampling Strata for a More Cost-Effective Probability Sample of a Rare Population
Published in: Public Opinion Quarterly, v. 73, no. 1, Spring 2009, p. 56-73
The authors consider situations in which externally observable characteristics allow experts to quickly categorize individual households as likely or unlikely to contain a member of a rare target population. This classification can form the basis of disproportionate stratified sampling such that households classified as unlikely are sampled at a lower rate than those classified as likely, thereby reducing screening costs. Design weights account for this approach and allow unbiased estimates for the target population. We demonstrate that with sensitivity and specificity of expert classification at least 70 percent, and ideally at least 80 percent, our approach can economically increase effective sample size for a rare population. The authors develop heuristics for implementing this approach and demonstrate that sensitivity drives design effects and screening costs whereas specificity only drives the latter. The authors demonstrate that the potential gains from this approach increase as the target population becomes rarer. They further show that for most applications, unlikely strata should be sampled at 1/6 to 1/2 the rate of likely strata. This approach was applied to a survey of Cambodian immigrants in which the 82 percent of households rated unlikely were sampled at 1/4 the rate as likely households, reducing screening from 9.4 to 4.0 approaches per complete. Sensitivity and specificity were 86 percent and 91 percent, respectively. Weighted estimation had a design effect of 1.26, so screening costs per effective sample size were reduced by 47 percent. The authors also note that in this instance, expert classification appeared to be uncorrelated with survey outcomes of interest among eligibles.