Analytic Methods for Adjusting Subjective Rating Schemes

by Richard V.L. Cooper, Gary R. Nelson

Download eBook for Free

FormatFile SizeNotes
PDF file 1.9 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Subjective evaluations of individual performances by supervisors are subject to bias. It is important to correct for biases in order to more accurately measure the effects of specific variables on individual performance. This report develops statistical and econometric techniques for correcting biases in models of individual performance using a variant of the classical linear regression model. A multiscale model is proposed to deal with two types of bias: location bias when an individual's performance is systematically overestimated or underestimated, and scale bias when differences among individuals rated are exaggerated or minimized. Several specific multiscale estimating techniques are developed, including equal total variance, equal residual variance, maximum likelihood, and least squares. Finally, the multiscale estimators are applied to the problem of estimating the cost of on-the-job training in the military. The multiscale model can be applied to a wide variety of estimating problems where observations can naturally be categorized into specific subgroups.

This report is part of the RAND Corporation Report series. The report was a product of the RAND Corporation from 1948 to 1993 that represented the principal publication documenting and transmitting RAND's major research findings and final research.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.