Measuring Success for Health Care Quality Improvement Interventions

Published in: Medical Care, v. 50, no. 12, Dec. 2012, p. 1086-1092

Posted on RAND.org on January 01, 2012

by Kristy Gonzalez Morganti, Susan L. Lovejoy, Amelia Haviland, Ann C. Haas, Donna O. Farley

BACKGROUND: The lack of a standard measure of quality improvement (QI) success and the use of subjective or self-reported measures of QI success has constrained efforts to formally evaluate QI programs and to understand how the various contextual factors impact QI success. OBJECTIVES: The objective of this study was to assess how best to measure "QI success" by comparing self-reported and externally rated measures of QI success. RESEARCH DESIGN: We performed a retrospective evaluation that analyzed data on different measures of QI success for organizations after their staff completed the QI training. SUBJECTS: The sample included 30 organizations whose staff had received QI training during 2006-2008, and who had used this training to carry out at least some subsequent QI initiative in their organizations. MEASURES: We developed 2 measures of self-reported QI success based on survey responses and 4 externally rated measures of QI success based on outcome data provided by the participating organizations in addition to qualitative data generated from the interviews. RESULTS: We found some variation in the mean scores of the different QI success measures and only moderate to small correlations between the self-report and externally rated QI measures. CONCLUSIONS: This study confirms that there are important differences between self-reported and externally rated measures of QI success and provides researchers with a methodology and criteria to externally rate measures of QI success.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.