Updated Adjustment of the HCAHPS Survey for New Modes of Survey Administration and Patient Mix

Background

Web-first multimode survey protocols increase HCAHPS survey response rates and representativeness but may result in different HCAHPS scores because of survey mode effects and selective email address availability. A variable absent from many patient-mix adjustment models that may result in more positive patient experiences is whether the hospital admission was planned; adjustment for planned stays may better measure hospital performance.

Objectives

Develop adjustments for new Web-first survey protocols and planned admissions to facilitate comparisons across hospitals.

Research Design

Using 2021 survey mode experiment data, we estimate survey protocol effects in linear models predicting HCAHPS top-box outcomes from protocol indicators (which incorporate email availability for Web-first protocols), patient-mix adjustors, and hospital intercepts. We evaluate the unique effect on scores of whether a stay was planned.

Results

Phone-only and Web-Phone without email produce more positive responses than Mail-only, requiring negative adjustments. All other survey protocol effects and adjustments are mixed in direction and generally small. Planned stays are associated with more positive experiences for otherwise similar patients and make a unique contribution beyond other current patient-mix adjustment variables.

Conclusions

It is important to adjust HCAHPS scores for survey protocol effects to ensure fair comparisons across hospitals and to enable hospitals to choose the survey protocol that best represents their patients. Incomplete email address availability necessitates that HCAHPS survey protocol adjustment control for email address availability when a Web-first protocol is used. Accounting for differences associated with planned stays may improve patient-mix adjustment.