Methods Commonly Used to Create 'Report Cards' May Overestimate the Quality of Health Care
Feb 23, 2006
Published In: Medical Care, v. 44, no. 2, Feb. 2006, p. 141-148
Posted on RAND.org on January 01, 2006
BACKGROUND: Administrative data are used to determine performance for publicly reported in health plan report cards, accreditation status, and reimbursement. However, it is unclear how performance based on administrative data and medical records compare. METHODS: The authors compared applicability, eligibility, and performance on 182 measures of health care quality using medical records and administrative data during a 13-month period for a random sample of 399 vulnerable older patients enrolled in managed care. RESULTS: Of 182 quality indicators (QIs) spanning 22 conditions, 145 (80%) were applicable only to medical records and 37 (20%) to either medical records or administrative data. Among 48 QIs specific to geriatric conditions, all were applicable to medical records; 2 of these also were applicable to administrative data. Eligibility for the 37 QIs that were applicable to both medical records and administrative data was similar for both data sources (94% agreement, [kappa] = 0.74). With the use of medical records, 152 of the 182 the QIs that were applicable to medical records were triggered and yielded an overall performance of 55%. Using administrative data, 30 of the 37 QIs that were applicable to administrative data were triggered and yielded overall performance of 83% (P < 0.05 vs. medical records). Restricting to QIs applicable to both data sources, overall performance was 84% and 83% (P = 0.21) for medical records and administrative data, respectively. CONCLUSIONS: The number and spectrum of QIs that can be measured for vulnerable elderly patients is far greater for medical records than for administrative data. Although summary estimates of health care quality derived from administrative data and medical records do not differ when using identical measures, summary scores from these data sources vary substantially when the totality of care that can be measured by each data source is measured.
This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.