Using Standardised Patients to Measure Physicians' Practice

Validation Study Using Audio Recordings

Published in: BMJ, British Medical Journal, v. 325, no. 7366, Sep. 28, 2002, p. 1-5

Posted on RAND.org on January 01, 2002

by Jeffrey Luck, John Peabody

Read More

Access further information on this document at www.bmj.com

This article was published outside of RAND. The full text of the article can be found at the link above.

OBJECTIVE: To assess the validity of standardised patients to measure the quality of physicians' practice. DESIGN: Validation study of standardised patients' assessments. Physicians saw unannounced standardised patients presenting with common outpatient conditions. The standardised patients covertly tape recorded their visit and completed a checklist of quality criteria immediately afterwards. Their assessments were compared against independent assessments of the recordings by a trained medical records abstractor. SETTING: Four general internal medicine primary care clinics in California. Participants: 144 randomly selected consenting physicians. MAIN OUTCOME MEASURES: Rates of agreement between the patients' assessments and independent assessment. RESULTS: 40 visits, one per standardised patient, were recorded. The overall rate of agreement between the standardised patients' checklists and the independent assessment of the audio transcripts was 91% (kappa=0.81). Disaggregating the data by medical condition, site, level of physicians' training, and domain (stage of the consultation) gave similar rates of agreement. Sensitivity of the standardised patients' assessments was 95%, and specificity was 85%. The area under the receiver operator characteristic curve was 90%. CONCLUSIONS: Standardised patients' assessments seem to be a valid measure of the quality of physicians' care for a variety of common medical conditions in actual outpatient settings. Properly trained standardised patients compare well with independent assessment of recordings of the consultations and may justify their use as a gold standard in comparing the quality of care across sites or evaluating data obtained from other sources, such as medical records and clinical vignettes.

This report is part of the RAND Corporation external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.