Please use this identifier to cite or link to this item: https://hdl.handle.net/10419/308618 
Year of Publication: 
2023
Citation: 
[Journal:] International Journal on Digital Libraries [ISSN:] 1432-1300 [Volume:] 24 [Issue:] 4 [Publisher:] Springer [Place:] Berlin, Heidelberg [Year:] 2023 [Pages:] 243-261
Publisher: 
Springer, Berlin, Heidelberg
Abstract: 
Data science deals with the discovery of information from large volumes of data. The data studied by scientists in the humanities include large textual corpora. An important objective is to study the ideas and expectations of a society regarding specific concepts, like "freedom" or "democracy," both for today's society and even more for societies of the past. Studying the meaning of words using large corpora requires efficient systems for text analysis, so-called distant reading systems. Making such systems efficient calls for a specification of the necessary functionality and clear expectations regarding typical work loads. But this currently is unclear, and there is no benchmark to evaluate distant reading systems. In this article, we propose such a benchmark, with the following innovations: As a first step, we collect and structure various information needs of the target users. We then formalize the notion of word context to facilitate the analysis of specific concepts. Using this notion, we formulate queries in line with the information needs of users. Finally, based on this, we propose concrete benchmark queries. To demonstrate the benefit of our benchmark, we conduct an evaluation, with two objectives. First, we aim at insights regarding the content of different corpora, i.e., whether and how their size and nature (e.g., popular and broad literature or specific expert literature) affect results. Second, we benchmark different data management technologies. This has allowed us to identify performance bottlenecks.
Subjects: 
Benchmark design
Text corpus
Distant reading
Query performance
Corpus insights
Persistent Identifier of the first edition: 
Creative Commons License: 
cc-by Logo
Document Type: 
Article
Document Version: 
Published Version

Files in This Item:
File
Size





Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.