Cover: Percentile-based Empirical Distribution Function Estimates for Performance Evaluation of Healthcare Providers

Percentile-based Empirical Distribution Function Estimates for Performance Evaluation of Healthcare Providers

Published in: Journal of the Royal Statistical Society: Series C (Applied Statistics), v. 60, no. 4, Aug. 2011, p. 575-589

Posted on on March 18, 2011

by Susan M. Paddock, Thomas A. Louis

Hierarchical models are widely used to characterize the performance of individual healthcare providers. However, little attention has been devoted to systemwide performance evaluations, the goals of which include identifying extreme (e.g. the top 10%) provider performance and developing statistical benchmarks to define high quality care. Obtaining optimal estimates of these quantities requires estimating the empirical distribution function (EDF) of provider-specific parameters that generate the data set under consideration. However, the difficulty of obtaining uncertainty bounds for a squared error loss minimizing EDF estimate has hindered its use in systemwide performance evaluations. We therefore develop and study a percentile-based EDF estimate for univariate provider-specific parameters. We compute order statistics of samples drawn from the posterior distribution of provider-specific parameters to obtain relevant assessments of uncertainty of an EDF estimate and its features, such as thresholds and percentiles. We apply our method to data from the Medicare end stage renal disease programme, which is a health insurance programme for people with irreversible kidney failure. We highlight the risk of misclassifying providers as exceptionally good or poor performers when uncertainty in statistical benchmark estimates is ignored. Given the high stakes of performance evaluations, statistical benchmarks should be accompanied by precision estimates.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.