Please use this identifier to cite or link to this item: https://hdl.handle.net/10419/325494 
Year of Publication: 
2025
Series/Report no.: 
Ruhr Economic Papers No. 1163
Publisher: 
RWI - Leibniz-Institut für Wirtschaftsforschung, Essen
Abstract: 
As interest in economic narratives has grown in recent years, so has the number of pipelines dedicated to extracting such narratives from texts. Pipelines often employ a mix of state-of-the-art natural language processing techniques, such as BERT, to tackle this task. While effective on foundational linguistic operations essential for narrative extraction, such models lack the deeper semantic understanding required to distinguish extracting economic narratives from merely conducting classic tasks like Semantic Role Labeling. Instead of relying on complex model pipelines, we evaluate the benefits of Large Language Models (LLMs) by analyzing a corpus of Wall Street Journal and New York Times newspaper articles about inflation. We apply a rigorous narrative definition and compare GPT 4o outputs to gold-standard narratives produced by expert annotators. Our results suggests that GPT-4o is capable of extracting valid economic narratives in a structured format, but still falls short of expert-level performance when handling complex documents and narratives. Given the novelty of LLMs in economic research, we also provide guidance for future work in economics and the social sciences that employs LLMs to pursue similar objectives.
Subjects: 
Economic narratives
natural language processing
large language models
JEL: 
C18
C55
C87
E70
Persistent Identifier of the first edition: 
ISBN: 
978-3-96973-348-6
Document Type: 
Working Paper

Files in This Item:
File
Size





Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.