Please use this identifier to cite or link to this item: https://hdl.handle.net/10419/307623 
Year of Publication: 
2023
Citation: 
[Journal:] The Review of International Organizations [ISSN:] 1559-744X [Volume:] 18 [Issue:] 4 [Publisher:] Springer US [Place:] New York, NY [Year:] 2023 [Pages:] 753-776
Publisher: 
Springer US, New York, NY
Abstract: 
International organizations (IOs) of the United Nations (UN) system publish around 750 evaluation reports per year, offering insights on their performance across project, program, institutional, and thematic activities. So far, it was not feasible to extract quantitative performance measures from these text-based reports. Using deep learning, this article presents a novel text-based performance metric: We classify individual sentences as containing a negative, positive, or neutral assessment of the evaluated IO activity and then compute the share of positive sentences per report. Content validation yields that the measure adequately reflects the underlying concept of performance; convergent validation finds high correlation with human-provided performance scores by the World Bank; and construct validation shows that our measure has theoretically expected results. Based on this, we present a novel dataset with performance measures for 1,082 evaluated activities implemented by nine UN system IOs and discuss avenues for further research.
Subjects: 
Performance
Evaluation
International organizations
Natural language processing
Machine learning
Persistent Identifier of the first edition: 
Creative Commons License: 
cc-by Logo
Document Type: 
Article
Document Version: 
Published Version

Files in This Item:
File
Size





Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.