Cover: Effects of a Web-Mail Mode on Response Rates and Responses to a Care Experience Survey

Effects of a Web-Mail Mode on Response Rates and Responses to a Care Experience Survey

Results of a Randomized Experiment

Published in: Journal of Survey Statistics and Methodology (2024). DOI: 10.1093/jssam/smae013

Posted on rand.org Apr 25, 2024

by Anagha Alka Tolpadi, Layla Parast, Marc N. Elliott, Ann C. Haas, Melissa A. Bradley, Joshua Wolf, Joan M. Teno, Maria DeYoreo, Lauren Fuentes, Rebecca Anhang Price

Patient experience surveys are vital to evaluating healthcare provider performance. However, declining response rates over time and questions about whether responses reflect the perspectives of all patients under care have raised concerns. One proposed approach to address these concerns is web-based survey administration, a mode that has not been studied in the hospice setting. We tested a sequential web-mail mode for administering a care experience survey in this unique setting, where family caregivers respond after the patient dies. Sampled caregivers of 15,515 patients who died March-August 2021 while receiving care from 56 hospices across the US were randomized to one of four survey modes: mail-only, telephone-only, mail-telephone (mail with telephone follow-up), or web-mail (email invitation to a web survey with mail follow-up). Email addresses were available for 31.3 percent of sampled eligible caregivers.

Relative to mail-only (estimated response rate = 35.1 percent), response rates were significantly higher for web-mail (39.7 percent) and mail-telephone (45.3 percent) and significantly lower for telephone-only (31.5 percent). The web-mail response rate was similar to the mail-only response rate among caregivers without email addresses (35.2 versus 34.3 percent), but substantially higher among caregivers with email addresses (49.6 versus 36.7 percent). Web-mail and mail-only respondents reported similar experiences for 26 of 27 evaluative items. Among eligible sampled caregivers, several patient/caregiver characteristics differed by caregivers' email address availability, but web-mail and mail-only respondents did not differ on any characteristic. A web-mail mode is feasible for surveying bereaved caregivers about care experiences, producing substantially higher response rates than single-mode approaches, with increasing benefits for hospices with higher proportions of caregivers with email addresses. Findings may be applicable to surveys of other sensitive topics and to populations that prefer asynchronous survey modes.

Research conducted by

This report is part of the RAND external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.