Equivalence of Mail and Telephone Responses to CAHPS Hospital Survey

Published In: HSR, Health Services Research, v. 40, no. 6, pt. 2, Dec. 2005, p. 2120-2139

Posted on RAND.org on January 01, 2005

by Han de Vries, Marc N. Elliott, Kimberly A. Hepner, San Keller, Ron D. Hays

Read More

Access further information on this document at Blackwell Publishing

This article was published outside of RAND. The full text of the article can be found at the link above.

OBJECTIVE: To estimate the effect of survey mode (mail versus telephone) on reports and ratings of hospital care. DATA SOURCES/STUDY SETTING: The total sample included 20,826 patients discharged from a group of 24 distinct hospitals in three states (Arizona, Maryland, New York). The authors collected CAHPS data in 2003 by mail and telephone from 9,504 patients, of whom 39 percent responded by telephone and 61 percent by mail. STUDY DESIGN: The authors estimated mode effects in an observational design, using both propensity score blocking and (ordered) logistic regression on covariates. They used variables derived from administrative data (either included as covariates in the regression function or used in estimating the propensity score) grouped in three categories: individual characteristics, characteristics of the stay and hospital, and survey administration variables. DATA COLLECTION/EXTRACTION METHODS. The authors mailed a 66-item questionnaire to everyone in the sample and followed up by telephone with those who did not respond. PRINCIPAL FINDINGS: The authors found significant (p<.01) mode effects for 13 of the 21 questions examined in this study. The maximum magnitude of the survey mode effect was an 11 percentage-point difference in the probability of a yes response to one of the survey questions. Telephone respondents were more likely to rate care positively and health status negatively, compared with mail respondents. Standard regression-based case-mix adjustment captured much of the mode effects detected by propensity score techniques in this application. CONCLUSIONS: Telephone mode increases the propensity for more favorable evaluations of care for more than half of the items examined. This suggests that mode of administration should be standardized or carefully adjusted for. Alternatively, further item development may minimize the sensitivity of items to mode of data collection.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit www.rand.org/about/principles.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.