Bitte verwenden Sie diesen Link, um diese Publikation zu zitieren, oder auf sie als Internetquelle zu verweisen: https://hdl.handle.net/10419/308508 
Autor:innen: 
Brodeur, Abel
Valenta, David
Marcoci, Alexandru
Aparicio, Juan P.
Mikola, Derek
Barbarioli, Bruno
Alexander, Rohan
Deer, Lachlan
Stafford, Tom
Vilhuber, Lars
Bensch, Gunther
Goldschmitt, Dirk
Gourdon-Kanhukamwe, Amélie
de Varda, Andrea Gregor
Grigoryeva, Idaliya
Gugushvili, Alexi
Fletcher, Aaron H.A.
Habermann, Florian
Hablicsek, Márton
Haddad, Joanne
Hall, Jonathan D.
Abdelhady, Mohamed
Hammar, Olle
Hassouneh, Malek
Hausladen, Carina I.
Hendrikse, Sophie C. F.
Hepplewhite, Matthew
Ho, Anson T. Y.
Hogan-Hennessy, Senan
Howley, Elliot
Huang, Gaoyang
Hulstaert, Héloïse
Abdelmoula, Yousra
Ilchovska, Zlatomira G.
Jakobsson, Niklas
Jansson, Joakim
Jarosz, Ewa
Jebeli, Hossein
Jiang, Yanchen
Junaid, Hiba
Kalluraya, Rohan
Kelly, Edmund
Kimel, Eva
Abdul Baki, Ghina
Kingsuwankul, Sorravich
Klotzbücher, Valentin
Krähmer, Daniel
Krūminas, Pijus
Kruus, Nicholas
Kujansuu, Essi
Kurz, Christoph F.
Küster, Stephan
Lee-Whiting, Blake
Lewandowski, Felix
Aguirre, Tomás
Li, Tongzhe
Li, Ruoxi
Liu, Dan
Liu, Jiacheng
Lo, Helix
Loter, Katharina
Dias, Felipe Macedo
Madan, Christopher R.
Mäder, Nicolas
Mandas, Marco
Aiyer, Sriraj
Marcus, Jan
Fages, Diego Marino
Martin, Xavier
McWay, Ryan
Medina-Gaspar, Daniel
Meng, Sisi
Meng, Lingyu
Miller, Alex P.
Mirabel, Thibault
Mishra, Dibya Deepta
Akhtar, Shumi
Mishra, Sumit
Moges, Belay W.
Mojarrad, Morteza Mohandes
Mohnen, Myra
Morin, Louis-Philippe
Motoki, Fabio
Muehlenbachs, Lucija
Mullin, Gastón
Musulan, Andreea
Muzzì, Sara
Akhtar, Farida
Myers, James A. C.
Neubauer, Florian
Nguyen, Tuan
Niazi, Ali
Nordstrom, Ardyn
Nowak, Bartłomiej
O’Habib, Daneal
Ölkers, Tim
Ong, Justin
Orozco Castiblanco, Valeria
Albada, Melle R.
Özak, Ömer
Ozkes, Ali I.
Paaso, Mikael
Pandey, Shubham
Papazoglou, Varvara
Penheiro, Romeo
Pham, Linh
Phieler, Ulrike
Pütz, Peter
Qi, Quan
Altman, Micah
Qiu, Jingyi
Reinstein, David A.
Repo, Juuso
Rudolf, Nicolas
Saha, Shree
Saka, Orkun
Saponaro, Chiara
Sator, Georg
Schoenmakers, Martijn
Seri, Raffaello
Angenendt, David
Shah, Meet
Sibille, Paul
Siemroth, Christoph
Skavysh, Vladimir
Slater, Ben
Song, Wenting
Staubli, Stefan
Steindl, Tobias
Waongo, Nomwendé Steven
Stott, Paul
Lari, Zahra Arjmandi
Strobel, Stephenson
Sudhaharan, Roshini
Sun, Pu
Swain, Scott D.
Talavera, Oleksandr
Tantiangco, Hanz M.
Tarasenko, Georgy
Tarlinton, Boyd
Tarraf, Mariam
Teoh, Ken
De León Tejada, Jorge Armando
Thériault, Rémi
Thompson, Bethan
Tian, Tonghui
Tian, Wenjie
Rein, Manuel Tobias
Tolani, Emmanuel
Topstad Borgen, Nicolai
Topstad Borgen, Solveig
Torralba, Javier
Velez-Ospina, Carolina
Asanov, Igor
Mak, Man Wai
Wallrich, Lukas
Wang, Zeyang
Ward, Leah
Webb, Matthew D.
Webb, Duncan
Weber, Bryan S.
Weber, Christoph
Weng, Wei-Chien
Westheide, Christian
Asanov Noha, Anastasiya-Mariya
Wilkinson, Tom
Wong, Kwong-Yu
Wroński, Marcin
Wu, Zhuangchen
Wu, Qixia
Wu, Victor Y.
Xiao, Bohan
Xu, Feihong
Xu, Cong
Yadav, Pranav
Ashong, Rebecca
Chou, Yu Yang
Yap, Luther
Yazbeck, Myra
Zagrodzka, Zuzanna
Zahra, Tahreen
Zaneva, Mirela
Zhang, Xiaomeng
Zhao, Ziwei
Zhong, Han
Zirgulis, Aras
Auer, Tobias
Zou, Jiacheng
Zoutman, Floris
Zozoungbo, Christelle
Bahamonde-Birke, Francisco J.
Baker, Bradley J.
Bartram, Söhnke M.
Bao, Dongqi
Batinovic, Lucija
Batistoni, Tommaso
Beeder, Monica
Beland, Louis-Philippe
Bienz, Carsten
Aryanto, Christ Billy
Bolibaugh, Cylcia
Bonander, Carl
Bravo, Ramiro
Brennan, Katherine
Bronnikov, Egor
Bruns, Stephan
Buliskeria, Nino
Caicedo-Silva, Sara
Calef, Andrea
Caulker, Solomon
Cepenas, Simonas
Chatton, Arthur
Chen, Zirou
Ewurum, Ngozi Chioma
Ciocîrlan, Anda-Bianca
Clouth, Felix J.
Collins, Jason
Cook, Nikolai
Cornejo, Cesar
Craveiro, João
Cui, Jing
Vayalabron, Niveditha Chalil
Czymara, Christian
Bermúdez Jaramillo, Carlos Daniel
Datta, Hannes
Denoo, Lien
Dhaliwal, Arshia
Dhameja, Nency
Djemai, Elodie
Dujeancourt, Erwan
Dündar, Uğurcan
Duprey, Thibaut
Eissa, Yasmine
El Fassi, Youssef
El Fassi, Ismail
Ellis, Keaton
Elminejad, Ali
Elsherif, Mahmoud
Emirmahmutoglu, Aysil
Etingin-Frati, Giulian
Eze, Emeka
Dollbaum, Jan Fabian
Victoria, Jan Feld
Rengifo Jaramillo, Andres Felipe
Fenig, Guidon
Fernandes, Victoria
Fiala, Lenka
Fink, Lukas
Fish, Sara
Fitzgerald, Jack
Forshaw, Rachel
Fortier-Chouinard, Alexandre
Fréget, Louis
Frese, Joris
Gabani, Jacopo
Gallegos, Sebastian
Gamill, Max C.
Gáspár, Attila
Gauriot, Romain
Gavrilova, Evelina
Geraldes, Diogo
Cantone, Giulio Giacomo
Gibson, Grant
Erscheinungsjahr: 
2025
Schriftenreihe/Nr.: 
I4R Discussion Paper Series No. 195
Verlag: 
Institute for Replication (I4R), s.l.
Zusammenfassung: 
This study evaluates the effectiveness of varying levels of human and artificial intelligence (AI) integration in reproducibility assessments of quantitative social science research. We computationally reproduced quantitative results from published articles in the social sciences with 288 researchers, randomly assigned to 103 teams across three groups - human-only teams, AI-assisted teams and teams whose task was to minimally guide an AI to conduct reproducibility checks (the "AI-led" approach). Findings reveal that when working independently, human teams matched the reproducibility success rates of teams using AI assistance, while both groups substantially outperformed AI-led approaches (with human teams achieving 57 percentage points higher success rates than AI-led teams, 𝒑 ﹤ 0.001). Human teams were particularly effective at identifying serious problems in the analysis: they found significantly more major errors compared to both AI-assisted teams (0.7 more errors per team, 𝒑 = 0.017) and AI-led teams (1.1 more errors per team, 𝒑 ﹤ 0.001). AI-assisted teams demonstrated an advantage over more automated approaches, detecting 0.4 more major errors per team than AI-led teams ( 𝒑 = 0.029), though still significantly fewer than human-only teams. Finally, both human and AI-assisted teams significantly outperformed AIled approaches in both proposing (25 percentage points difference, 𝒑 = 0.017) and implementing (33 percentage points difference, 𝒑 = 0.005) comprehensive robustness checks. These results underscore both the strengths and limitations of AI assistance in research reproduction and suggest that despite impressive advancements in AI capability, key aspects of the research publication process still require human substantial human involvement.
Dokumentart: 
Working Paper

Datei(en):
Datei
Größe
1.15 MB





Publikationen in EconStor sind urheberrechtlich geschützt.