Comparisons With Other Data Sources

Estimates of Internet access in the U.S. are on the order of 87%. So despite the fact that the ALP provides Internet access to respondents without prior Internet access (about 10% of the sample), this group is underrepresented. In and of itself, it is not unusual that different groups in the population are differentially represented in the panel. The main question is if one can correct for differential response rates by reweighting. Couper, Kapteyn, Schonlau and Winter (2007) find that conditional on Internet use both the stated willingness to participate in an Internet survey and actual participation are only weakly linked to individual characteristics. This suggests that once weights—based on individual characteristics—are applied to ALP data, the resulting sample combining both individuals with and without Internet access will be population representative. This can be further investigated by comparing weighted frequencies in the ALP with external benchmarks.

Weights

Weights can be created for each ALP dataset. Figures 1-3 below present comparisons of weighted ALP variables (this is the set of active members of the American Life Panel as of January 2016 who are considered randomly sampled) with weighted Current Population Survey (CPS) variables (CPS March 2015), for males, females, and by number of household members. The weights are calculated using a raking algorithm, as explained in the weighting section. Figure 4 presents a comparison between ALP and CPS with respect to some variables of interest that are both present in CPS and in the ALP, and which have not been used for weighting.

Comparison Charts

A chart that shows a comparison of weighted frequencies in ALP and CPS among females.

Figure 1. Comparison of Weighted Frequencies in ALP and CPS among Females

We compared weighted frequencies of 11 different variables among females in ALP and CPS:

  • ages 18-32
  • ages 33-43
  • ages 44-54
  • ages 55-64
  • ages 65+
  • education: college graduate
  • education: high school or less
  • education: some college
  • race: Hispanic/other
  • race: non-Hispanic black
  • race: non-Hispanic white

The weighted variable frequencies were very similar across females in ALP and CPS. Ages 33-43, ages 55-64, and race: non-Hispanic black were all below .1. Ages 18-32, ages 44-54, ages 65+, education: college graduate, education: some college, and race: Hispanic/other were all between .1 and .2. Education: high school or less was between .2 and .3, and race: non-Hispanic white was between .3 and .4.

A chart that shows a comparison of weighted frequencies in ALP and CPS among males.

Figure 2. Comparison of Weighted Frequencies in ALP and CPS among Males

We compared weighted frequencies of 11 different variables among males in ALP and CPS:

  • ages 18-32
  • ages 33-43
  • ages 44-54
  • ages 55-64
  • ages 65+
  • education: college graduate
  • education: high school or less
  • education: some college
  • race: Hispanic/other
  • race: non-Hispanic black
  • race: non-Hispanic white

The weighted variable frequencies were very similar across males in ALP and CPS. Ages 33-43, ages 44-54, ages 55-64, ages 65+, and race: non-Hispanic black were all below .1. Ages 18-32, education: college graduate, education: some college, and race: Hispanic/other were all between .1 and .2. Education: high school or less was between .2 and .3, and race: non-Hispanic white was above .3.

A chart that shows a comparison of weighted frequencies in ALP and CPS, by number of household members.

Figure 3. Comparison of Weighted Frequencies in ALP and CPS by Income

We compared weighted frequencies of four different income variables by number of household members in ALP and CPS:

  • $75,000 or more
  • less than $25,000
  • $25,000-$50,000
  • $50,000-$75,000

The weighted variable frequencies were very similar in ALP and CPS. For households with one member, income of $75,000 or more, $25,000-$50,000, and $50,000-$75,000 had a frequency below .05. Income of $25,000 had a frequency between .05 and .1. For households with two members, income of $75,000 or more, less than $25,000, and $50,000-$75,000 had a frequency between .05 and .1. Income of $25,000-$50,000 had a frequency between .1 and .15. For households with three or more members, of income of $75,000 or more, less than $25,000, $25,000-$50,000, and $50,000-$75,000 all had a frequency between .1 and .15.

A chart that shows a comparison of weighted frequencies in ALP and CPS for variables not matched in the weighting algorithm.

Figure 4. Comparison of Weighted Frequencies in ALP and CPS for Variables Not Matched in the Weighting Algorithm

We compared weighted frequencies of nine different variables of interest in ALP and CPS:

  • born in U.S.
  • currently working
  • marital: married
  • marital: widowed/divorced
  • region: Midwest
  • region: Northeast
  • region: South
  • region: West
  • U.S. citizen

The weighted variable frequencies were similar in ALP and CPS, but with greater differences than previous comparisons on this page. Born in US, currently working, marital: married, region: South, region: West, and U.S. citizen have higher frequencies in CPS than they do in ALP. Marital: widowed/divorced, region: Midwest, and region: Northeast have higher frequencies in ALP than they do in CPS. Marital: widowed/divorced has a frequency below .2. Region: Northeast has a frequency of approximately .2 in ALP and slightly less than .2 in CPS. Region: Midwest, region: South, and region: West all have frequencies between .2 and .4. Currently working has a frequency of just below .6 in ALP and just above .6 in CPS. Marital: married has a frequency between .5 and .6 in ALP and between .6 and .7 in CPS. Born in U.S. and U.S. citizen have frequencies between .8 and 1.