Response to ProPublica's Rebuttal of Our Critique of the Surgeon Scorecard

In the summer of 2015, ProPublica published its Surgeon Scorecard, which displays "Adjusted Complication Rates" for individual, named surgeons for eight surgical procedures performed in hospitals. Public reports of provider performance have the potential to improve the quality of health care that patients receive. A valid performance report can drive quality improvement and usefully inform patients' choices of providers. However, performance reports with poor validity and reliability are potentially damaging to all involved. In September 2015, RAND released a critique of the Scorecard authored by a group of heath policy researchers from RAND and other institutions, and on October 7, 2015 ProPublica published a rebuttal of our critique. In this follow-on Perspective, we revisit the main points in our initial critique, summarize ProPublica's rebuttal, explain why this rebuttal fails to address our methodological concerns, provide suggestions on how to validate and improve the Scorecard, and explain why we continue to advise potential users of the Scorecard, as it is currently constructed, not to consider it a valid or reliable predictor of the health outcomes any individual surgeon is likely to provide.