LLM as a law professor: Having a large language model write a commentary on freedom of assembly - Christoph Engel; Johannes Kruse (Max Planck Institute for Research on Collective Goods, 2025)
In many jurisdictions, academia is at the service of legal practice. Law professors write commentaries that summarize the state of the art of doctrine, chiefly of jurisprudence. In the spirit of a proof of concept, using the guarantee of freedom of assembly in the European Convention on Human Rights, we show that this task can be completely outsourced to large language models. Using standard NLP metrics and an LLM as a judge approach, we develop an evaluation pipeline that works without costly human annotation. The commentaries fully written by GPT 4o, Gemini 2.5 flash or Kimi K2 Instruct are on par with their best human written competitor, the Guide provided by the Court itself.
hdl:21.11116/0000-0011-B2F6-E
Keywords: 330; Generative künstliche Intelligenz; Large Language Model; Rechtswissenschaft