Sensitivity, Specificity, Reliability, and Clinical Validity of Provider-Reported Symptoms

A Comparison with Self-Reported Symptoms

Published in: Journal of Acquired Immune Deficiency Syndromes, v. 21, no. 2, June 1, 1999, p. 126-133

Posted on RAND.org on January 01, 1999

by Amy C. Justice, Linda Rabeneck, Ron D. Hays, Albert W. Wu, Samuel A. Bozzette

Read More

Access further information on this document at www.jaids.com

This article was published outside of RAND. The full text of the article can be found at the link above.

BACKGROUND: If symptoms are to be recognized and effectively addressed in clinical research, they must be collected using sensitive, specific, reliable, and clinically meaningful methods. OBJECTIVE: To perform a comparison of self-administered symptom survey data with data from conventional provider-reports. DESIGN/METHODS: Secondary data analysis of AIDS Clinical Trials Group Study 081 (ACTG 081), a randomized trial taking place in 33 sites comparing three approaches to prophylaxis for Pneumocystis carinii-related pneumonia that found no difference among treatment arms. The study was performed on 842 subjects with advanced HIV infection. No intervention was undertaken as a result of this study. ACTG 081 included data on functional status, global quality of life and survival, and two methods of symptom measurement: an open-ended, provider-reported symptom assessment (provider-report) and a self-administered symptom survey (self-report). Agreement was measured using kappa scores. Sensitivity and specificity were calculated using self-report as the standard. Reliability was measured by intersite variation and test-retest reliability (8 weeks later). Clinical validity was evaluated by testing expected associations with functional status, global quality of life, and survival. RESULTS: Symptom data were available for 808 patients (96%). Patient and provider agreement was poor (mean kappa, 0.14; range, 0.07-0.25). Compared with self-report, providers underreported the presence and severity of symptoms (mean symptom count, 5.2 versus 1.3; mean severity score, 1.3 versus 0.74). provider-report demonstrated greater variability by site (R2 associated with site, 0.02 versus 0.16) and poorer test-retest reliability (mean kappa, 0.34 versus 0.25). Provider-report severity scores were less strongly associated than were self-report with functional status (chi2, 252 versus 80), global quality of life (R2 for model, 0.57 versus 0.15), and survival (chi2, 38 versus 24). Self-reported symptom severity was strongly correlated to patient-reported global quality of life (p, 0.75; p < .0001). CONCLUSIONS: Provider-reported symptoms as currently collected within the ACTG are less sensitive and reproducible than a self-administered symptom survey. Provider-reported severity scores are also more weakly associated with functional status, global quality of life, and survival. A self-reported symptom survey may provide a better method of symptom measurement for HIV research.

This report is part of the RAND Corporation external publication series. Many RAND studies are published in peer-reviewed scholarly journals, as chapters in commercial books, or as documents published by other organizations.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.