Please use this identifier to cite or link to this item: http://hdl.handle.net/10419/216897
Authors: 
Huntington-Klein, Nick
Arenas, Andreu
Beam, Emily
Bertoni, Marco
Bloem, Jeffrey R.
Burli, Pralhad
Chen, Naibin
Greico, Paul
Ekpe, Godwin
Pugatch, Todd
Saavedra, Martin
Stopnitzky, Yaniv
Year of Publication: 
2020
Series/Report no.: 
GLO Discussion Paper No. 537
Abstract: 
Researchers make hundreds of decisions about data collection, preparation, and analysis in their research. We use a many-analysts approach to measure the extent and impact of these decisions. Two published causal empirical results are replicated by seven replicators each. We find large differences in data preparation and analysis decisions, many of which would not likely be reported in a publication. No two replicators reported the same sample size. Statistical significance varied across replications, and for one of the studies the effect’s sign varied as well. The standard deviation of estimates across replications was 3-4 times the typical reported standard error.
Subjects: 
Replication
Metascience
Research
JEL: 
C81
C10
B41
Document Type: 
Working Paper

Files in This Item:
File
Size





Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.