About Our Calculations

Seven Domains of Performance

In 2016, the Centers for Medicare and Medicaid Services (CMS) released star ratings for all hospitals nationally, all publicly accessible through the Hospital Compare website. These ratings are a score (1–5 stars) designed to help consumers understand the overall performance of different hospitals.

Each hospital submits to CMS its data on up to 57 performance measures (e.g., readmission rate for patients who had a heart attack). These 57 measures are grouped into seven domains of hospital performance:

  • Mortality
  • Safety of Care
  • Readmissions
  • Patient Experience
  • Timeliness of Care
  • Effectiveness of Care
  • Efficient Use of Medical Imaging

Then group scores are calculated for each domain.

The calculator is based on the December 2016 methods report (PDF) from QualityNet, so it does not include the most-current data from Hospital Compare. At this time, we do not plan to update the tool with new data or other methodological changes because its main purpose is to illustrate a method (user-entered weights) that can be applied to any performance report.

The calculator is intended as a proof of concept, not a consumer-ready performance-rating site. Read more in our perspective piece in New England Journal of Medicine.

Our Calculations

To calculate overall star ratings for hospitals using customized weighting of the different performance domains, we modified CMS's analytic programs for calculating star ratings to allow the end users to assign their own weights. No other modifications were made to the star rating calculation methods.

To create the user interface, we allowed each of the seven domains to take the following weight values:

  • 0 points
  • 4 points
  • 22 points
  • 50 points
  • 100 points

Two of these weights (4 points and 22 points) are the default weights assigned by CMS to the seven composites; three domains have a default weight of 4 points, and four domains have a default weight of 22 points. We chose the remaining weights (0, 50, and 100 points) to provide a wider possible range of weights. Allowing users to choose from five possible weights for each of the seven domains resulted in thousands of possible combinations.

For each combination of user-entered weights, we calculated rescaled weights by dividing the points assigned to each domain by the total points assigned. This step ensured that all possible combinations of rescaled weights summed to 100%.

We then reran the CMS overall star ratings program once for each possible combination of weights, using the rescaled weights and applying the program to the December 2016 public release of the Hospital Compare database. For each user-selected combination of weights, the web tool displays the hospital star ratings that resulted from these program runs.

Because the star ratings program uses a cluster algorithm, the overall distribution of stars (including the mean and median ratings) can change as different weights are applied. This means that the star rating for a given hospital can increase or decrease between different weight settings, yet the hospital’s ranking relative to other hospitals might change in the opposite direction (or not at all). Therefore, we recommend focusing on the effects of different weight settings on the relative performance of a given hospital—comparing its star rating with other hospitals’ star ratings rather than with a fixed threshold.

NOTE: Hospitals with no CMS data or star ratings also have no data in the Personalized Hospital Performance Report Card. Further documentation of CMS methods for calculating overall star ratings is available on QualityNet.

Funding

This tool and the research that underlies it are philanthropically supported. Philanthropic contributions support RAND’s ability to take the long view, tackle tough and often-controversial topics, and share our findings in innovative and compelling ways. RAND’s research findings and recommendations are based on data and evidence, and therefore do not necessarily reflect the policy preferences or interests of its clients, donors, or supporters.

Funding for this research was provided by gifts from RAND supporters and income from operations.