Please use this identifier to cite or link to this item: https://hdl.handle.net/10419/315353 
Authors: 
Breznau, Nate
Rinke, Eike Mark
Wuttke, Alexander
Adem, Muna
Adriaans, Jule
Akdeniz, Esra
Alvarez-Benjumea, Amalia
Andersen, Henrik K.
Auer, Daniel
Azevedo, Flavio
Bahnsen, Oke
Bai, Ling
Balzer, Dave
Bauer, Paul C.
Bauer, Gerrit
Baumann, Markus
Baute, Sharon
Benoit, Verena
Bernauer, Julian
Berning, Carl
Berthold, Anna
Bethke, Felix S.
Biegert, Thomas
Blinzler, Katharina
Blumenberg, Johannes N.
Bobzien, Licia
Bohman, Andrea
Bol, Thijs
Bostic, Amie
Brzozowska, Zuzanna
Burgdorf, Katharina
Burger, Kaspar
Busch, Kathrin
Castillo, Juan-Carlos
Chan, Nathan
Christmann, Pablo
Connelly, Roxanne
Czymara, Christian S.
Damian, Elena
de Rooij, Eline A.
Ecker, Alejandro
Edelmann, Achim
Eder, Christina
Eger, Maureen A.
Ellerbrock, Simon
Forke, Anna
Forster, Andrea
Freire, Danilo
Gaasendam, Chris
Gavras, Konstantin
Gayle, Vernon
Gessler, Theresa
Gnambs, Timo
Godefroidt, Amélie
Grömping, Max
Groß, Martin
Gruber, Stefan
Gummer, Tobias
Hadjar, Andreas
Halbherr, Verena
Heisig, Jan Paul
Hellmeier, Sebastian
Heyne, Stefanie
Hirsch, Magdalena
Hjerm, Mikael
Hochman, Oshrat
Höffler, Jan H.
Hövermann, Andreas
Hunger, Sophia
Hunkler, Christian
Huth-Stöckle, Nora
Ignácz, Zsófia S.
Israel, Sabine
Jacobs, Laura
Jacobsen, Jannes
Jaeger, Bastian
Jungkunz, Sebastian
Jungmann, Nils
Kanjana, Jennifer
Kauff, Mathias
Khan, Salman
Khatua, Sayak
Kleinert, Manuel
Klinger, Julia
Kolb, Jan-Philipp
Kołczyńska, Marta
Kuk, John
Kunißen, Katharina
Kurti Sinatra, Dafina
Langenkamp, Alexander
Lee, Robin C.
Lersch, Philipp M.
Liu, David
Löbel, Lea-Maria
Lutscher, Philipp
Mader, Matthias
Madia, Joan E.
Malancu, Natalia
Maldonado, Luis
Marahrens, Helge
Martin, Nicole
Martinez, Paul
Mayerl, Jochen
Mayorga, Oscar J.
McDonnell, Robert
McManus, Patricia
McWagner, Kyle
Meeusen, Cecil
Meierrieks, Daniel
Mellon, Jonathan
Merhout, Friedolin
Merk, Samuel
Meyer, Daniel
Micheli, Leticia
Mijs, Jonathan
Moya, Cristóbal
Neunhoeffer, Marcel
Nüst, Daniel
Nygård, Olav
Ochsenfeld, Fabian
Otte, Gunnar
Pechenkina, Anna
Pickup, Mark
Prosser, Christopher
Raes, Louis
Ralston, Kevin
Ramos, Miguel
Reichert, Frank
Roets, Arne
Rogers, Jonathan
Ropers, Guido
Samuel, Robin
Sand, Gregor
Sanhueza Petrarca, Constanza
Schachter, Ariela
Schaeffer, Merlin
Schieferdecker, David
Schlueter, Elmar
Schmidt, Katja
Schmidt, Regine
Schmidt-Catran, Alexander
Schmiedeberg, Claudia
Schneider, Jürgen
Schoonvelde, Martijn
Schulte-Cloos, Julia
Schumann, Sandy
Schunck, Reinhard
Seuring, Julian
Silber, Henning
Sleegers, Willem
Sonntag, Nico
Staudt, Alexander
Steiber, Nadia
Steiner, Nils D.
Sternberg, Sebastian
Stiers, Dieter
Stojmenovska, Dragana
Storz, Nora
Striessnig, Erich
Stroppe, Anne-Kathrin
Suchow, Jordan W.
Teltemann, Janna
Tibajev, Andrey
Tung, Brian
Vagni, Giacomo
Van Assche, Jasper
van der Linden, Meta
van der Noll, Jolanda
Van Hootegem, Arno
Vogtenhuber, Stefan
Voicu, Bogdan
Wagemans, Fieke
Wehl, Nadja
Werner, Hannah
Wiernik, Brenton M.
Winter, Fabian
Wolf, Christof
Wu, Cary
Yamada, Yuki
Zakula, Björn
Zhang, Nan
Ziller, Conrad
Zins, Stefan
Żółtak, Tomasz
Nguyen, Hung H.V.
Year of Publication: 
2025
Citation: 
[Journal:] Royal Society Open Science [ISSN:] 2054-5703 [Volume:] 12 [Issue:] 3 [Article No.:] 241038 [Publisher:] The Royal Society Publishing [Place:] London [Year:] 2025 [Pages:] 1-23
Publisher: 
The Royal Society Publishing, London
Abstract: 
This study investigates researcher variability in computational reproduction, an activity for which it is least expected. Eighty-five independent teams attempted numerical replication of results from an original study of policy preferences and immigration. Reproduction teams were randomly grouped into a ‘transparent group’ receiving original study and code or ‘opaque group’ receiving only a method and results description and no code. The transparent group mostly verified original results (95.7% same sign and p -value cutoff), while the opaque group had less success (89.3%). Second-decimal place exact numerical reproductions were less common (76.9 and 48.1%). Qualitative investigation of the workflows revealed many causes of error, including mistakes and procedural variations. When curating mistakes, we still find that only the transparent group was reliably successful. Our findings imply a need for transparency, but also more. Institutional checks and less subjective difficulty for researchers ‘doing reproduction’ would help, implying a need for better training. We also urge increased awareness of complexity in the research process and in ‘push button’ replications.
Persistent Identifier of the first edition: 
Creative Commons License: 
cc-by Logo
Document Type: 
Article
Document Version: 
Published Version

Files in This Item:





Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.