Physician Cost Profiling

Reliability and Risk of Misclassification

Published In: The New England Journal of Medicine, v. 362, no. 11, Mar. 18, 2010, p. 1014-1021

Posted on RAND.org on December 31, 2009

by John L. Adams, Ateev Mehrotra, William Thomas, Elizabeth A. McGlynn

Read More

Access further information on this document at Massachusetts Medical Society

This article was published outside of RAND. The full text of the article can be found at the link above.

BACKGROUND: Insurance products with incentives for patients to choose physicians classified as offering lower-cost care on the basis of cost-profiling tools are increasingly common. However, no rigorous evaluation has been undertaken to determine whether these tools can accurately distinguish higher-cost physicians from lower-cost physicians. METHODS: We aggregated claims data for the years 2004 and 2005 from four health plans in Massachusetts. We used commercial software to construct clinically homogeneous episodes of care (e.g., treatment of diabetes, heart attack, or urinary tract infection), assigned each episode to a physician, and created a summary profile of resource use (i.e., cost) for each physician on the basis of all assigned episodes. We estimated the reliability (signal-to-noise ratio) of each physician's cost-profile score on a scale of 0 to 1, with 0 indicating that all differences in physicians' cost profiles are due to a lack of precision in the measure (noise) and 1 indicating that all differences are due to real variation in costs of services (signal). We used the reliability results to estimate the proportion of physicians in each specialty whose cost performance would be classified inaccurately in a two-tiered insurance product in which the physicians with cost profiles in the lowest quartile were labeled as lower cost. RESULTS: Median reliabilities ranged from 0.05 for vascular surgery to 0.79 for gastroenterology and otolaryngology. Overall, 59% of physicians had cost-profile scores with reliabilities of less than 0.70, a commonly used marker of suboptimal reliability. Using our reliability results, we estimated that 22% of physicians would be misclassified in a two-tiered system. CONCLUSIONS: Current methods for profiling physicians with respect to costs of services may produce misleading results.

This report is part of the RAND Corporation external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.