Comparison Between Clinical Registry and Medicare Claims Data on the Classification of Hospital Quality of Surgical Care

Published In: Annals of Surgery, v. 261, no. 2, Feb. 2015, p. 290-296

Posted on on January 28, 2015

by Elise H. Lawson, David Zingmond, Bruce L. Hall, Rachel Louie, Robert H. Brook, Clifford Y. Ko

Read More

Access further information on this document at Wolters Kluwer Health, Inc

This article was published outside of RAND. The full text of the article can be found at the link above.

OBJECTIVE: To compare the classification of hospital statistical outlier status as better or worse performance than expected for postoperative complications using Medicare claims versus clinical registry data. Background: Controversy remains as to the most favorable data source for measuring postoperative complications for pay-for-performance and public reporting polices. METHODS: Patient-level records (2005–2008) were linked between the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) and Medicare inpatient claims. Hospital statistical outlier status for better or worse performance than expected was assessed using each data source for superficial surgical site infection (SSI), deep/organ-space SSI, any SSI, urinary tract infection, pneumonia, sepsis, deep venous thrombosis, pulmonary embolism, venous thromboembolism, and myocardial infarction by developing hierarchical multivariable logistic regression models. Kappa statistics and correlation coefficients assessed agreement between the data sources. RESULTS: A total of 192 hospitals with 110,987 surgical patients were included. Agreement on hospital rank for complication rates between Medicare claims and ACS-NSQIP was poor-to-moderate (weighted κ: 0.18–0.48). Of hospitals identified as statistical outliers for better or worse performance by Medicare claims, 26% were also identified as outliers by ACS-NSQIP. Of outliers identified by ACS-NSQIP, 16% were also identified as outliers by Medicare claims. Agreement between the data sources on hospital outlier status classification was uniformly poor (weighted κ: −0.02–0.34). CONCLUSIONS: Despite using the same statistical methodology with each data source, classification of hospital outlier status as better or worse performance than expected for postoperative complications differed substantially between ACS-NSQIP and Medicare claims.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

Our mission to help improve policy and decisionmaking through research and analysis is enabled through our core values of quality and objectivity and our unwavering commitment to the highest level of integrity and ethical behavior. To help ensure our research and analysis are rigorous, objective, and nonpartisan, we subject our research publications to a robust and exacting quality-assurance process; avoid both the appearance and reality of financial and other conflicts of interest through staff training, project screening, and a policy of mandatory disclosure; and pursue transparency in our research engagements through our commitment to the open publication of our research findings and recommendations, disclosure of the source of funding of published research, and policies to ensure intellectual independence. For more information, visit

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.