Better-than-average and Worse-Than-Average Hospitals May Not Significantly Differ from Average Hospitals

An Analysis of Medicare Hospital Compare Ratings

Published in: BMJ Quality and Safety, v. 24, no. 2, Feb. 2015, p. 128-134

Posted on RAND.org on November 26, 2014

by Susan M. Paddock, John L. Adams, Fernando Hoces de la Guardia

Read More

Access further information on this document at BMJ Quality and Safety

This article was published outside of RAND. The full text of the article can be found at the link above.

BACKGROUND: Public report card designers aim to provide comprehensible provider performance information to consumers. Report cards often display classifications of providers into performance tiers that reflect whether performance is statistically significantly above or below average or not statistically significantly different from average. To further enhance the salience of public reporting to consumers, report card websites often allow a user to compare a subset of selected providers on tiered performance rather than direct statistical comparisons of the providers in a consumer's personal choice set. OBJECTIVE: We illustrate the differences in conclusions drawn about relative provider performance using tiers versus conducing statistical tests to assess performance differences. METHODS: Using publicly available cross-sectional data from Medicare Hospital Compare on three mortality and three readmission outcome measures, we compared each provider in the top or bottom performance tier with those in the middle tier and assessed the proportion of such comparisons that exhibited no statistically significant differences. RESULTS: Across the six outcomes, 1.3–6.1% of hospitals were classified in the top tier. Each top-tier hospital did not statistically significantly differ in performance from at least one mid-tier hospital. The percentages of mid-tier hospitals that were not statistically significantly different from a given top-tier hospital were 74.3–81.1%. The percentages of hospitals classified as bottom tier were 0.6–4.0%. Each bottom-tier hospital showed no statistically significant difference from at least one mid-tier hospital. The percentage of mid-tier hospitals that were not significantly different from a bottom-tier hospital ranged from 60.4% to 74.8%. CONCLUSIONS: Our analyses illustrate the need for further innovations in the design of public report cards to enhance their salience for consumers.

This report is part of the RAND Corporation external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.