Methodological Considerations in Generating Provider Performance Scores for Use in Public Reporting

Public reports of health care providers' performance on measures of quality, cost and resource use, patient experience, and health outcomes have become increasingly common. These reports are often intended to help patients choose providers and may encourage providers to improve their performance. At the July 2009 National Meeting of Chartered Value Exchanges (CVEs) hosted by AHRQ, CVE stakeholders identified a dilemma: Two organizations could, by making different methodological decisions, use the exact same data to produce divergent public performance reports that send conflicting messages to patients and providers. In response to this dilemma, AHRQ commissioned RAND Corporation to develop a white paper to identify methodological decision points that precede publication of a performance report and to delineate the options for each. Our overall aim in developing this white paper is to produce a resource that is useful to CVEs and other community collaboratives as they consider the range of available methodological options for performance reporting. This white paper reviews a number of methodological decision points that CVEs and other community collaboratives may encounter when generating provider performance scores. The paper also discusses the advantages and disadvantages associated with various choices for each of these decision points. This report is intended to help CVEs understand different types of measurement error, how sources of error may enter into the construction of provider performance scores, and how to mitigate or minimize the risk of misclassifying a provider. Again, the methods decisions generally involve important tradeoffs. There are rarely clear "right answers," and value judgments underlie most decisions.