P-hacking and significance stars - Costanza Naguib (University of Bern, Department of Economics, 2024)
In mid-2016, all journals of the American Economic Association (AEA) stopped including significance stars in their regression tables. This policy aimed to reduce the emphasis on statistical significance and shift focus toward the broader economic importance of research findings. This study examines the impact of this change on p-hacking and publication bias. The findings indicate some reduction in the bunching of the reported test statistics just beyond the 5%-significance threshold in the treated journals after 2016. However, the effect is modest.
Keywords: A11; A14; C13; 330; p-hacking; significance stars; publication bias; difference-in-difference