Please use this identifier to cite or link to this item:
Lee, Jungmin
Year of Publication: 
Series/Report no.: 
IZA Discussion paper series 1257
The quality of subjective performance evaluation is dependent on the incentive structures faced by evaluators, in particular on how they are monitored and themselves evaluated. Figure skating competitions provide a unique opportunity to study subjective evaluation. This paper develops and tests a simple model of what I call "outlier aversion bias" in which subjective evaluators avoid submitting outlying judgments. We find significant evidence for the existence of outlier aversion. Individual judges within a game manipulate scores to achieve a targeted level of agreement with the other judges. Furthermore, a natural experiment shows that the dispersion of scores across judges depends upon the type of judge-assessment system and its implication for outlier aversion. Agreement may not be a good criterion for the validity of an evaluation system, contradicting the industrial psychology and personnel management literature.
subjective performance evaluation
outlier aversion bias
Document Type: 
Working Paper

Files in This Item:
353.78 kB

Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.