Comparing telephone and mail responses to the CAHPS survey instrument
The objective of the Consumer Assessment of Health Plans (CAHPS) survey was to evaluate the comparability of answers to CAHPS questions when data are collected by mail and by telephone. Two studies comparing phone and mail responses used parallel samples of Medicaid beneficiaries in California, and adults with chronic conditions who had health insurance through the State of Washington. A third study used a revised instrument. In the first two tests, numerous significant differences were found in rates at which questions that might not apply to all respondents were answered: some ratings were more positive on the telephone. In the test of a revised instrument, nine of 58 comparisons differed significantly by mode. Although further steps to reduce the remaining mode effects are needed, data indicate that when the revised CAHPS questions are used, mode of data collection will have little effect on key results.