Reliability of Utilization Measures for Primary Care Physician Profiling
Published in: Healthcare, v. 1, no. 1-2, June 2013, p. 22-29
Posted on RAND.org on June 01, 2013
- Do common utilization measures reflect real variation in health costs among primary care physicians?
- Can these measures be used appropriately for physician profiling?
BACKGROUND: Given rising health care costs, there has been a renewed interest in using utilization measures to profile physicians. Despite the measures' common use, few studies have examined their reliability and whether they capture true differences among physicians. METHODS: A local health improvement organization in New York State used 2008–2010 claims data to create 11 utilization measures for feedback to primary care physicians (PCP). The sample consists of 2938 PCPs in 1546 practices who serve 853,187 patients. We used these data to measure reliability of these utilization measures using two methods (hierarchical model versus test–retest). For each PCP and each practice, we estimate each utilization measure's reliability, ranging from 0 to 1, with 0 indicating that all differences in utilization are due to random noise and 1 indicating that all differences are due to real variation among physicians. RESULTS: Reliability varies significantly across the measures. For 4 utilization measures (PCP visits, specialty visits, PCP lab tests (blood and urine), and PCP radiology and other tests), reliability was high (mean>0.85) at both the physician and the practice level. For the other 7 measures (professional therapeutic visits, emergency room visits, hospital admissions, readmissions, skilled nursing facility days, skilled home care visits, and custodial home care services), there was lower reliability indicating more substantial measurement error. CONCLUSIONS: The results illustrate that some utilization measures are suitable for PCP and practice profiling while caution should be used when using other utilization measures for efforts such as public reporting or pay-for-performance incentives.
- Only four commonly used utilization measures reliably capture real differences in utilization among physicians.
- Other measures commonly used for physician profiling have lower reliability and should be used with caution.
- Given the substantial variation across utilization measures, organizations using them to profile physicians should rigorously evaluate the measures' reliability.