Cover: Accuracy of the Safer Dx Instrument to Identify Diagnostic Errors in Primary Care

Accuracy of the Safer Dx Instrument to Identify Diagnostic Errors in Primary Care

Published in: Journal of General Internal Medicine, 2016

Posted on rand.org Mar 2, 2016

by Aymer Al-Mutairi, Ashley N. Meyer, Eric J. Thomas, Jason Michel Etchegaray, Kevin M. Roy, Maria C. Davalos, Shazia Sheikh, Hardeep Singh

Research Question

  1. How well does Safer Dx, a research instrument, spot medical diagnosis errors?

IMPORTANCE: Diagnostic errors are common and harmful, but difficult to define and measure. Measurement of diagnostic errors often depends on retrospective medical record reviews, frequently resulting in reviewer disagreement. OBJECTIVES: We aimed to test the accuracy of an instrument to help detect presence or absence of diagnostic error through record reviews. DESIGN: We gathered questions from several previously used instruments for diagnostic error measurement, then developed and refined our instrument. We tested the accuracy of the instrument against a sample of patient records (n = 389), with and without previously identified diagnostic errors (n = 129 and n = 260, respectively). RESULTS: The final version of our instrument (titled Safer Dx Instrument) consisted of 11 questions assessing diagnostic processes in the patient–provider encounter and a main outcome question to determine diagnostic error. In comparison with the previous sample, the instrument yielded an overall accuracy of 84 %, sensitivity of 71 %, specificity of 90 %, negative predictive value of 86 %, and positive predictive value of 78 %. All 11 items correlated significantly with the instrument's error outcome question (all p values ≤ 0.01). Using factor analysis, the 11 questions clustered into two domains with high internal consistency (initial diagnostic assessment, and performance and interpretation of diagnostic tests) and a patient factor domain with low internal consistency (Cronbach's alpha coefficients 0.93, 0.92, and 0.38, respectively). CONCLUSIONS: The Safer Dx Instrument helps quantify the likelihood of diagnostic error in primary care visits, achieving a high degree of accuracy for measuring their presence or absence. This instrument could be useful to identify high-risk cases for further study and quality improvement.

Key Findings

  • Safer Dx Instrument assesses 11 aspects of a primary care encounter for breakdown of the diagnostic process.
  • Instrument was tested against 389 patient records containing clinically identified errors.
  • The Instrument detected diagnostic errors 84% of the time.
  • Safer Dx could help standardize detection of diagnostic error in primary care.

This report is part of the RAND external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.