A New Instrument to Measure Appropriateness of Services in Primary Care

Published in: International Journal For Quality in Health Care, v. 16, no. 2, Apr. 2004, p. 133-140

Posted on RAND.org on January 01, 2004

by David H. Thom, Richard L. Kravitz, Steven Kelly-Reif, Ronnie V. Sprinkle, Joseph R. Hopkins, Lisa V. Rubenstein

Read More

Access further information on this document at intqhc.oxfordjournals.org

This article was published outside of RAND. The full text of the article can be found at the link above.

OBJECTIVE: To develop a new instrument for judging the appropriateness of three key services (new prescription, diagnostic test, and referral) as delivered in primary care outpatient visits. DESIGN: Candidate items were generated by a seven-member expert panel, using a five-step nominal technique, for each of three service categories in primary care: new prescriptions, diagnostic tests, and referrals. Expert panelists and a convenience sample of 95 community-based primary care physicians ranked items for (i) importance and (ii) feasibility of ascertaining from a typical office chart record. Resulting items were used to construct a measure of appropriateness using principals of structured implicit review. Two physician reviewers used this measure to judge the appropriateness of 421 services from 160 outpatient visits. SETTING: Primary care practices in a staff model health maintenance organization and a large preferred provider network. MEASURES: Inter-rater agreement was measured using intraclass correlation coefficient (ICC) and kappa statistic. RESULTS: For overall appropriateness, the ICC and kappa were 0.52 and 0.44 for new medication, 0.35 and 0.32 for diagnostic test, and 0.40 and 0.41 for referral, respectively. Only 3% of services were judged to be inappropriate by either reviewer. The proportion of services judged to be less than definitely appropriate by one or both reviewers was 56% for new medication, 31% for diagnostic test, and 22% for referral. CONCLUSIONS: This new measure of appropriateness of primary care services has fair inter-rater agreement for new medications and referrals, similar to appropriateness measures of other general services, but poor agreement for diagnostic tests. It may be useful as a tool to assess the appropriateness of common primary care services in studies of health care quality, but is not suitable for evaluating performance of individual physicians.

This report is part of the RAND Corporation External publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.