The Role of Judgment in Admissions
Download eBook for Free
|PDF file||6.1 MB||
Use Adobe Acrobat Reader version 10 or higher for the best experience.
This report explores issues involved in higher education admissions processes. It analyzes admissions and subsequent student performance at the RAND Graduate School. The academic literature on this problem informs the presentation of statistical relationships among admissions criteria, admissions committee ratings and performance measures. Transformation techniques provide the foundation for a discussion of the merits of alternative ways of thinking about selection, performance measurement and prediction. This research supports several conclusions: 1) Admissions committees confront conflicting objectives. Committee decisions may show inconsistency across applicants, partly because selection combines individual judgments through persuasion and consensus-building. Furthermore, quantified measures of committee members' ratings may not fully reflect their true judgments about the desirability of applicants; 2) Accepted applicants typically receive more ratings than rejected applicants. Moreover, committee members sometimes discuss applicants with each other and see other members' ratings before making their own judgments. Nevertheless, including the evaluation of an additional judge in average ratings increases the correlation between average ratings and both the admission decision and some performance measures; 3) While the RGS admissions committee implicitly gives great importance to GRE scores, different performance measures correlate most strongly with different selection criteria. This complicates the establishment of screening rules; 4) Transforming ratings and grades increases the correlation between them, and it increases the explained variance in admissions decisions. Thus, both admissions and later performance evaluation depend at least partially on a relative rating of students. Such relativism implies that admissions decisions, which depend at least in part on prediction of future performance, affect how that future performance will be measured; 5) The empirical results show the statistical significance of several selection criteria in predicting a variety of measures of student performance. They also compare the significance of these criteria to the significance of quantified committee-member ratings in predicting the same performance measures. Furthermore, this report shows several techniques for transforming measures of selection criteria, ratings and performance, in order to discuss the appropriateness of relative and absolute measures. Although this dissertation focuses on data from RGS, it presents a framework for thinking about problems associated with admissions that general audiences may find useful or enlightening.
Research conducted by
This publication is part of the RAND Corporation Dissertation series. Pardee RAND dissertations are produced by graduate fellows of the Pardee RAND Graduate School, the world's leading producer of Ph.D.'s in policy analysis. The dissertations are supervised, reviewed, and approved by a Pardee RAND faculty committee overseeing each dissertation.
This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.
The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.