To discuss experimental results without discussing how they came about makes sense when the results are robust to the way experiments are conducted. Experimental results, however, are – arguably more often than not – sensitive to numerous design and implementation characteristics such as the use of financial incentives, deception, and the way information is presented. To the extent that economists and psychologists have different experimental practices, this claim is of obvious practical and interpretative relevance. In light of the empirical results summarized below, it seems warranted to say that it does not make sense to report experimental results without reporting the design and implementation choices that were made.
Duhem-Quine problem experimental design experimental implementation financial incentives deception