Development and Evaluation of CAHPS Questions to Assess the Impact of Health Information Technology on Patient Experiences with Ambulatory Care

Published in: Medical Care, v. 50, no. 11, suppl 3, Nov. 2012, p. S11-S19

Posted on on October 29, 2012

by K McInnes, Julie A. Brown, Ron D. Hays, Patricia Gallagher, James D. Ralston, Mildred Hugh, Michael Kanter, Carl Serrato, Carol Cosenza, John Halamka, et al.

BACKGROUND: Little is known about whether health information technology (HIT) affects patient experiences with health care. OBJECTIVE: To develop HIT questions that assess patients care experiences not evaluated by existing ambulatory Consumer Assessment of Health Plans and Systems (CAHPS) measures. RESEARCH DESIGN: We reviewed published articles and conducted focus groups and cognitive testing to develop survey questions. We collected data, using mail and the internet, from patients of 69 physicians receiving care at an academic medical center and 2 regional integrated delivery systems in late 2009 and 2010. We evaluated questions and scales about HIT using factor analysis, item-scale correlations, and reliability (internal consistency and physician-level) estimates. RESULTS: We found support for 3 HIT composites: doctor use of computer (2 items), e-mail (2 items), and helpfulness of provider's website (4 items). Corrected item-scale correlations were 0.37 for the 2 doctor use of computer items and 0.71 for the 2 e-mail items, and ranged from 0.50 to 0.60 for the provider's website items. Cronbach α was high for e-mail (0.83) and provider's website (0.75), but only 0.54 for doctor use of computer. As few as 50 responses per physician would yield reliability of 0.70 for e-mail and provider's website. Two HIT composites, doctor use of computer (P<0.001) and provider's website (P=0.02), were independent predictors of overall ratings of doctors. CONCLUSIONS: New CAHPS HIT items were identified that measure aspects of patient experiences not assessed by the CAHPS C&G 1.0 survey.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.