Comparisons With Other Data Sources
Estimates of Internet access in the U.S. are on the order of 87%. So despite the fact that the ALP provides Internet access to respondents without prior Internet access (about 10% of the sample), this group is underrepresented. In and of itself, it is not unusual that different groups in the population are differentially represented in the panel. The main question is if one can correct for differential response rates by reweighting. Couper, Kapteyn, Schonlau and Winter (2007) find that conditional on Internet use both the stated willingness to participate in an Internet survey and actual participation are only weakly linked to individual characteristics. This suggests that once weights—based on individual characteristics—are applied to ALP data, the resulting sample combining both individuals with and without Internet access will be population representative. This can be further investigated by comparing weighted frequencies in the ALP with external benchmarks.
Weights
Weights can be created for each ALP dataset. Figures 1-3 below present comparisons of weighted ALP variables (this is the set of active members of the American Life Panel as of January 2016 who are considered randomly sampled) with weighted Current Population Survey (CPS) variables (CPS March 2015), for males, females, and by number of household members. The weights are calculated using a raking algorithm, as explained in the weighting section. Figure 4 presents a comparison between ALP and CPS with respect to some variables of interest that are both present in CPS and in the ALP, and which have not been used for weighting.
Comparison Charts
Figure 1. Comparison of Weighted Frequencies in ALP and CPS among Females
We compared weighted frequencies of 11 different variables among females in ALP and CPS:
- ages 18-32
- ages 33-43
- ages 44-54
- ages 55-64
- ages 65+
- education: college graduate
- education: high school or less
- education: some college
- race: Hispanic/other
- race: non-Hispanic black
- race: non-Hispanic white
The weighted variable frequencies were very similar across females in ALP and CPS. Ages 33-43, ages 55-64, and race: non-Hispanic black were all below .1. Ages 18-32, ages 44-54, ages 65+, education: college graduate, education: some college, and race: Hispanic/other were all between .1 and .2. Education: high school or less was between .2 and .3, and race: non-Hispanic white was between .3 and .4.
Figure 2. Comparison of Weighted Frequencies in ALP and CPS among Males
We compared weighted frequencies of 11 different variables among males in ALP and CPS:
- ages 18-32
- ages 33-43
- ages 44-54
- ages 55-64
- ages 65+
- education: college graduate
- education: high school or less
- education: some college
- race: Hispanic/other
- race: non-Hispanic black
- race: non-Hispanic white
The weighted variable frequencies were very similar across males in ALP and CPS. Ages 33-43, ages 44-54, ages 55-64, ages 65+, and race: non-Hispanic black were all below .1. Ages 18-32, education: college graduate, education: some college, and race: Hispanic/other were all between .1 and .2. Education: high school or less was between .2 and .3, and race: non-Hispanic white was above .3.
Figure 3. Comparison of Weighted Frequencies in ALP and CPS by Income
We compared weighted frequencies of four different income variables by number of household members in ALP and CPS:
- $75,000 or more
- less than $25,000
- $25,000-$50,000
- $50,000-$75,000
The weighted variable frequencies were very similar in ALP and CPS. For households with one member, income of $75,000 or more, $25,000-$50,000, and $50,000-$75,000 had a frequency below .05. Income of $25,000 had a frequency between .05 and .1. For households with two members, income of $75,000 or more, less than $25,000, and $50,000-$75,000 had a frequency between .05 and .1. Income of $25,000-$50,000 had a frequency between .1 and .15. For households with three or more members, of income of $75,000 or more, less than $25,000, $25,000-$50,000, and $50,000-$75,000 all had a frequency between .1 and .15.
Figure 4. Comparison of Weighted Frequencies in ALP and CPS for Variables Not Matched in the Weighting Algorithm
We compared weighted frequencies of nine different variables of interest in ALP and CPS:
- born in U.S.
- currently working
- marital: married
- marital: widowed/divorced
- region: Midwest
- region: Northeast
- region: South
- region: West
- U.S. citizen
The weighted variable frequencies were similar in ALP and CPS, but with greater differences than previous comparisons on this page. Born in US, currently working, marital: married, region: South, region: West, and U.S. citizen have higher frequencies in CPS than they do in ALP. Marital: widowed/divorced, region: Midwest, and region: Northeast have higher frequencies in ALP than they do in CPS. Marital: widowed/divorced has a frequency below .2. Region: Northeast has a frequency of approximately .2 in ALP and slightly less than .2 in CPS. Region: Midwest, region: South, and region: West all have frequencies between .2 and .4. Currently working has a frequency of just below .6 in ALP and just above .6 in CPS. Marital: married has a frequency between .5 and .6 in ALP and between .6 and .7 in CPS. Born in U.S. and U.S. citizen have frequencies between .8 and 1.