Use of Expert Ratings as Sampling Strata for a More Cost-Effective Probability Sample of a Rare Population

Published in: Public Opinion Quarterly, v. 73, no. 1, Spring 2009, p. 56-73

by Marc N. Elliott, Daniel F. McCaffrey, Judith F. Perlman, Grant N. Marshall, Katrin Hambarsoomian

Read More

Access further information on this document at Public Opinion Quarterly

This article was published outside of RAND. The full text of the article can be found at the link above.

The authors consider situations in which externally observable characteristics allow experts to quickly categorize individual households as likely or unlikely to contain a member of a rare target population. This classification can form the basis of disproportionate stratified sampling such that households classified as unlikely are sampled at a lower rate than those classified as likely, thereby reducing screening costs. Design weights account for this approach and allow unbiased estimates for the target population. We demonstrate that with sensitivity and specificity of expert classification at least 70 percent, and ideally at least 80 percent, our approach can economically increase effective sample size for a rare population. The authors develop heuristics for implementing this approach and demonstrate that sensitivity drives design effects and screening costs whereas specificity only drives the latter. The authors demonstrate that the potential gains from this approach increase as the target population becomes rarer. They further show that for most applications, unlikely strata should be sampled at 1/6 to 1/2 the rate of likely strata. This approach was applied to a survey of Cambodian immigrants in which the 82 percent of households rated unlikely were sampled at 1/4 the rate as likely households, reducing screening from 9.4 to 4.0 approaches per complete. Sensitivity and specificity were 86 percent and 91 percent, respectively. Weighted estimation had a design effect of 1.26, so screening costs per effective sample size were reduced by 47 percent. The authors also note that in this instance, expert classification appeared to be uncorrelated with survey outcomes of interest among eligibles.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.