The Behavioral Signature of GenAI in Scientific Communication - Nikos Askitas (Munich Society for the Promotion of Economic Research - CESifo GmbH, 2025)
We examine the uptake and measurable effects of GPT-assisted writing in economics working paper abstracts. Focusing on the IZA discussion paper series, we detect a significant stylistic shift following the public release of ChatGPT-3.5 in March 2023. This shift appears in core textual metrics–including mean word length, type-token ratio, and readability–and reflects growing alignment with machine-generated writing. While the release of ChatGPT constitutes an exogenous technological shock, adoption is endogenous: authors choose whether to incorporate AI assistance. To capture and estimate the magnitude of this behavioral response, we combine stylometric analysis, machine learning classification, and prompt-based similarity testing. Event-study regressions with fixed effects and placebo checks confirm that the observed shift is abrupt, persistent, and not attributable to pre-existing trends. A similarity experiment using OpenAI's API shows that post-ChatGPT abstracts more closely resemble their GPT-optimised counterparts than do pre-ChatGPT texts. A classifier trained on these variants achieves 97% accuracy and increasingly flags post-March 2023 abstracts as GPT-like. Rather than indicating wholesale substitution, our findings suggest selective human–AI augmentation in professional writing. The framework introduced here generalises to other settings where writing plays a central role–including resumes, job descriptions, legal briefs, research proposals, and software documentation.
Keywords: C55; C88; O33; C81; L86; J24; 330; GPT adoption; academic writing; text analysis; natural language processing (NLP); machine learning; event study; linguistic metrics; AI-assisted writing; diffusion of technology