Please use this identifier to cite or link to this item: https://hdl.handle.net/10419/208709 
Year of Publication: 
2009
Series/Report no.: 
PhD Series No. 1.2009
Publisher: 
Copenhagen Business School (CBS), Frederiksberg
Abstract: 
The present maps study maps the decision-making behaviors of experienced raters in a well-established Communal Writing Assessment (CWA) context, tracing their behaviors all the way from the independent rating sessions, where the initial images and judgments are formed, to the communal rating sessions, where the final scores are assigned on the basis of collaboration between two rates. Results from think-aloud protocols, recorded discussions, retrespective reports and reported scores from 20 raters rating 15 ESL essays show that when moving from the independent ratings to the communal ratings, there is little, if any, increase in rater agreement levels and the raters' attention to the textual features corresponding to the official criteria become more evenly distributed. However, rather than consulting the scale descriptors directly in resolving insecurities about score assignment, the raters seemed to rely heavily on each others' expertise, thereby reducing the importance of the scale and emphasizing the value of the community of raters. In validating their scores in the communal rating discussions the raters appeared to be critically and equally engaged in the discussions, and through deliberating and refining their assessments the raters believed that CWA practices produce more accurate scores than in independent ratings and lead to professional development. These interpretations support a hermeneutic rather than a psychometric approach to establishing the validity of the present CWA practices.
Subjects: 
sprogundervisning
eksamination
danmark
evaluering
Persistent Identifier of the first edition: 
ISBN: 
9788759383773
Creative Commons License: 
cc-by-nc-nd Logo
Document Type: 
Doctoral Thesis

Files in This Item:
File
Size





Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.