Please use this identifier to cite or link to this item: https://hdl.handle.net/10419/289437 
Authors: 
Brodeur, Abel
Mikola, Derek
Cook, Nikolai
Brailey, Thomas
Briggs, Ryan
de Gendre, Alexandra
Dupraz, Yannick
Fiala, Lenka
Gabani, Jacopo
Gauriot, Romain
Haddad, Joanne
Kjellsson, Gustav
Kjelsrud, Anders
Kotsadam, Andreas
Korpershoek, Jori
Krashinsky, Lewis
Kundu, Suranjana
Kustov, Alexander
Lalayev, Nurlan
Langlois, Audrée
Wright, Taylor
Duncan, Jamie
Laufer, Jill
Lee-Whiting, Blake
Leibing, Andreas
Lenz, Gabriel
Levin, Joel
Li, Peng
Li, Tongzhe
Lin, Yuchen
Listo, Ariel
Liu, Dan
Duong, Kiet Tuan
Connolly, Marie
Lu, Xuewen
Lukmanova, Elvina
Luscombe, Alex
Lusher, Lester R.
Lyu, Ke
Ma, Hai
Mäder, Nicolas
Makate, Clifton
Malmberg, Alice
Duprey, Thibaut
Maitra, Adit
Hoces de la Guardia, Fernando
Mandas, Marco
Marcus, Jan
Margaryan, Shushanik
Márk, Lili
Martignano, Andres
Marsh, Abigail
Masetto, Isabella
McCanny, Anthony
Dworschak, Christoph
McManus, Emma
McWay, Ryan
Johannesson, Magnus
Metson, Lennard
Kinge, Jonas Minet
Mishra, Sumit
Mohnen, Myra
Möller, Jakob
Montambeault, Rosalie
Montpetit, Sébastien
Ellingsrud, Sigmund
Morin, Louis-Philippe
Morris, Todd
Moser, Scott
Miguel, Edward
Motoki, Fabio
Muehlenbachs, Lucija
Musulan, Andreea
Musumeci, Marco
Nabin, Munirul
Nchare, Karim
Elminejad, Ali
Neubauer, Florian
Nguyen, Quan M. P.
Nguyen, Tuan
Nguyen-Tien, Viet
Vilhuber, Lars
Niazi, Ali
Nikolaishvili, Giorgi
Nordstrom, Ardyn
Nüß, Patrick
Odermatt, Angela
Eissa, Yasmine
Olson, Matt
Øien, Henning
Ölkers, Tim
Oliver i Vert, Miquel
Oral, Emre
Abarca, Alejandro
Oswald, Christian
Ousman, Ali
Özak, Ömer
Pandey, Shubham
Erhart, Andrea
Pavlov, Alexandre
Pelli, Martino
Penheiro, Romeo
Park, RyuGyung
Pérez Martel, Eva
Petrovičová, Tereza
Acharya, Mahesh
Phan, Linh
Prettyman, Alexa
Procházka, Jakub
Etingin-Frati, Giulian
Putri, Aqila
Quandt, Julian
Qiu, Kangyu
Nguyen, Loan Quynh Thi
Rahman, Andaleeb
Rea, Carson H.
Reiremo, Adam
Adjisse, Sossou Simplice
Renée, Laëtitia
Richardson, Joseph
Fatemi-Pour, Elaheh
Rivers, Nicholas
Rodrigues, Bruno
Roelofs, William
Roemer, Tobias
Rogeberg, Ole
Rose, Julian
Roskos-Ewoldsen, Andrew
Rosmer, Paul
Akhtar, Ahwaz
Sabada, Barbara
Lima, Goncalo
Saberian, Soodeh
Salamanca, Nicolas
Sator, Georg
Sawyer, Antoine
Scates, Daniel
Schlüter, Elmar
Sells, Cameron
Sen, Sharmi
Sethi, Ritika
Ramirez Lizardi, Eduardo Alberto
Federice, Alexa
Shcherbiak, Anna
Sogaolu, Moyosore
Soosalu, Matt
Sørensen, Erik Ø.
Sovani, Manali
Spencer, Noah
Staubli, Stefan
Stans, Renske
Stewart, Anya
Stips, Felix
Feld, Jan
Albrecht, Sabina
Stockley, Kieran
Strobel, Stephenson
Struby, Ethan
Tang, John
Tanrisever, Idil
Yang, Thomas Tao
Tastan, Ipek
Tatić, Dejan
Tatlow, Benjamin
Fenig, Guidon
Tchuisseu Seuyong, Féraud
Andersen, Synøve Nygaard
Thériault, Rémi
Thivierge, Vincent
Tian, Wenjie
Toma, Filip-Mihai
Totarelli, Maddalena
Tran, Van-Anh
Truong, Hung
Tsoy, Nikita
Firouzjaeiangalougah, Mojtaba
Tuzcuoglu, Kerem
Ubfal, Diego
Andlib, Zubaria
Villalobos, Laura
Walterskirchen, Julian
Wang, Joseph Taoyi
Wattal, Vasudha
Webb, Matthew D.
Weber, Bryan
Weisser, Reinhard
Fleisje, Erlend
Weng, Wei-Chien
Westheide, Christian
White, Kimberly
Arrora, Falak
Winter, Jacob
Wochner, Timo
Woerman, Matt
Wong, Jared
Woodard, Ritchie
Wroński, Marcin
Fortier-Chouinard, Alexandre
Yazbeck, Myra
Yang, Gustav Chung
Yap, Luther
Yassin, Kareman
Ash, Thomas
Ye, Hao
Yoon, Jin Young
Yurris, Chris
Zahra, Tahreen
Zaneva, Mirela
Engel, Julia Francesca
Zayat, Aline
Zhang, Jonathan
Zhao, Ziwei
Zhong Yaolang
Bacher, Etienne
Bachler, Sebastian
Bacon, Félix
Bagues, Manuel
Balogh, Timea
Batmanov, Alisher
Fries, Tilman
Barschkett, Mara
Basdil, B. Kaan
Baxa, Jaromír
Becker, Sascha
Beeder, Monica
Beland, Louis-Philippe
Bello, Abdel Hamid
Markovits, Daniel Benenson
Benjamin, Grant
Bergeron, Thomas
Fortier, Reid
Blimpo, Moussa P.
Binetti, Marco
Bonander, Carl
Bonneau, Joseph
Borbáth, Endre
Topstad Borgen, Nicolai
Topstad Borgen, Solveig
Borowsky, Jonathan
Brini, Elisa
Brown, Myriam
Fréchet, Nadjim
Brun, Martín
Bruns, Stephan
Buliskeria, Nino
Calef, Andrea
Cameron, Alistair
Campa, Pamela
Campos-Rodríguez, Santiago
Cantone, Giulio Giacomo
Carpena, Fenella
Carter, Perry
Ankel-Peters, Jörg
Castañeda Dower, Paul
Castek, Ondrej
Caviglia-Harris, Jill
Chauca Strand, Gabriella
Chen, Shi
Chzhen, Asya
Chung, Jong
Collins, Jason
Coppock, Alexander
Cordeau, Hugo
Galipeau, Thomas
Couillard, Ben
Crechet, Jonathan
Crippa, Lorenzo
Cui, Jeanne
Czymara, Christian
Daarstad, Haley
Dao, Danh Chi
Dao, Dong
Schmandt, Marco David
de Linde, Astrid
Gallegos, Sebastián
De Melo, Lucas
Deer, Lachlan
De Vera, Micole
Dimitrova, Velichka
Dollbaum, Jan Fabian
Dollbaum, Jan Matti
Donnelly, Michael
Huynh, Luu Duc Toan
Dumbalska, Tsvetomira
Gangji, Areez
Gao, Xiaoying
Garnache, Cloé
Gáspár, Attila
Gavrilova, Evelina
Ghosh, Arijit
Gibney, Garreth
Gibson, Grant
Dreber, Anna
Godager, Geir
Goff, Leonard
Gong, Da
González, Javier
Gretton, Jeremy
Griffa, Cristina
Grigoryeva, Idaliya
Grøtting, Maja
Guntermann, Eric
Guo, Jiaqi
Campbell, Douglas
Gugushvili, Alexi
Habibnia, Hooman
Häffner, Sonja
Hall, Jonathan D.
Hammar, Olle
Kordt, Amund Hanson
Hashimoto, Barry
Hartley, Jonathan S.
Hausladen, Carina I.
Havránek, Tomáš
Kattan, Lamis
Hazen, Jacob
He, Harry
Hepplewhite, Matthew
Herrera-Rodriguez, Mario
Heuer, Felix
Heyes, Anthony
Ho, Anson T. Y.
Holmes, Jonathan
Holzknecht, Armando
Hsu, Yu-Hsiang Dexter
Marino Fages, Diego
Hu, Shiang-Hung
Huang, Yu-Shiuan
Huebener, Mathias
Huber, Christoph
Huynh, Kim P.
Irsova, Zuzana
Isler, Ozan
Jakobsson, Niklas
Frith, Michael James
Jananji, Raphaël
Mierisch, Fabian
Jayalath, Tharaka A.
Jetter, Michael
John, Jenny
Forshaw, Rachel Joy
Juan, Felipe
Kadriu, Valon
Karim, Sunny
Kelly, Edmund
Dang, Duy Khanh Hoang
Khushboo, Tazia
Sun, Pu
Kim, Jin
Year of Publication: 
2024
Series/Report no.: 
I4R Discussion Paper Series No. 107
Publisher: 
Institute for Replication (I4R), s.l.
Abstract: 
This study pushes our understanding of research reliability by reproducing and replicating claims from 110 papers in leading economic and political science journals. The analysis involves computational reproducibility checks and robustness assessments. It reveals several patterns. First, we uncover a high rate of fully computationally reproducible results (over 85%). Second, excluding minor issues like missing packages or broken pathways, we uncover coding errors for about 25% of studies, with some studies containing multiple errors. Third, we test the robustness of the results to 5,511 re-analyses. We find a robustness reproducibility of about 70%. Robustness reproducibility rates are relatively higher for re-analyses that introduce new data and lower for re-analyses that change the sample or the definition of the dependent variable. Fourth, 52% of re-analysis effect size estimates are smaller than the original published estimates and the average statistical significance of a re-analysis is 77% of the original. Lastly, we rely on six teams of researchers working independently to answer eight additional research questions on the determinants of robustness reproducibility. Most teams find a negative relationship between replicators' experience and reproducibility, while finding no relationship between reproducibility and the provision of intermediate or even raw data combined with the necessary cleaning codes.
Subjects: 
Reproduction
Replication
Research Transparency
Open Science
Economics
Political Science
JEL: 
B41
C10
C81
Document Type: 
Working Paper

Files in This Item:
File
Size





Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.