The ordinary meaning bot: Simulating human surveys with LLMs - Johannes Kruse (Max Planck Institute for Research on Collective Goods, 2025)
This Comment shows how large language models (LLMs) can help courts discern the "ordinary meaning" of statutory terms. Instead of relying on expert-heavy corpus-linguistic techniques (Gries 2025), the author simulates a human survey with GPT-4o. Demographically realistic AI agents replicate the 2,835 participants in Tobia's 2020 study on vehicle and yield response distributions with no statistically significant difference from the human data (Kolmogorov-Smirnov p = 0.915). The paper addresses concerns about hallucinations, reproducibility, data leakage, and explainability, and introduces the locked-prompt "Ordinary Meaning Bot," arguing that LLM-based survey simulation is a practical, accurate alternative to dictionaries, intuition, or complex corpus analysis.
hdl:21.11116/0000-0011-94AD-3
Keywords: K1; Z0; 330; Ordinary meaning; large language models; prompt engineering; human survey simulation; alignment