Estimating Reliability and Misclassification in Physician Profiling

John L. Adams, Ateev Mehrotra, Elizabeth A. McGlynn

ResearchPublished Sep 28, 2010

This technical report explains the relationship between reliability measurement and misclassification for physician quality and cost measures in health care. It provides details and a practical method to calculate reliability and misclassification from the data typically available to health plans. This report builds on other RAND work on reliability and misclassification and has two main goals. First, it can serve as a tutorial for measuring reliability and misclassification. Second, it will describe the likelihood of misclassification in a situation not addressed in our prior work in which physicians are categorized using statistical testing. For any newly proposed system, the methods presented here should enable an evaluator to calculate the reliabilities and, consequently, the misclassification probabilities. It is our hope that knowing these misclassification probabilities will increase transparency about profiling methods and stimulate an informed debate about the costs and benefits of alternative profiling systems. The appendixes provide more technical detail on how to measure reliability with related program code as well as a set of lookup tables that can be used to obtain the rate of misclassification associated with a reliability estimate under various scenarios.

Topics

Document Details

  • Copyright: RAND Corporation
  • Availability: Web-Only
  • Year: 2010
  • Pages: 37
  • Document Number: TR-863-MMS

Citation

RAND Style Manual
Adams, John L., Ateev Mehrotra, and Elizabeth A. McGlynn, Estimating Reliability and Misclassification in Physician Profiling, RAND Corporation, TR-863-MMS, 2010. As of September 11, 2024: https://www.rand.org/pubs/technical_reports/TR863.html
Chicago Manual of Style
Adams, John L., Ateev Mehrotra, and Elizabeth A. McGlynn, Estimating Reliability and Misclassification in Physician Profiling. Santa Monica, CA: RAND Corporation, 2010. https://www.rand.org/pubs/technical_reports/TR863.html.
BibTeX RIS

This work was sponsored by the American Medical Association and the Massachusetts Medical Society. The research was conducted in RAND Health, a division of the RAND Corporation.

This publication is part of the RAND technical report series. RAND technical reports, products of RAND from 2003 to 2011, presented research findings on a topic limited in scope or intended for a narrow audience; discussions of the methodology employed in research; literature reviews, survey instruments, modeling exercises, guidelines for practitioners and research professionals, and supporting documentation; and preliminary findings. All RAND technical reports were subject to rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.