This Note provides an overview and detailed discussion of latent class and latent trait methods for the analysis of observer agreement. These methods make it possible to estimate (1) the accuracy of individual raters; (2) the most likely category of a case, given several opinions; and (3) the optimal number of ratings on which to base classifications, given a sample of cases, each rated by multiple, independent observers. These techniques provide promising new tools for the analysis of observer agreement, and have many important potential applications in social science and medical research.
This report is part of the RAND Corporation Note series. The note was a product of the RAND Corporation from 1979 to 1993 that reported other outputs of sponsored research for general distribution.
This research in the public interest was supported by RAND, using discretionary funds made possible by the generosity of RAND's donors, the fees earned on client-funded research, and independent research and development (IR&D) funds provided by the Department of Defense.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.