Please use this identifier to cite or link to this item: http://hdl.handle.net/10419/222345
Authors: 
Riener, Gerhard
Schneider, Sebastian
Wagner, Valentin
Year of Publication: 
2020
Series/Report no.: 
DICE Discussion Paper 345
Abstract: 
In this paper, we systematically analyze the empirical importance of standard conditions for the validity and generalizability of field experiments: the internal and external overlap and unconfoundedness conditions. We experimentally varied the degree of overlap in disjoint sub-samples from a recruitment experiment with more than 3,000 public schools, mimicking small scale field experiments. This was achieved by using different techniques for treatment assignment. We applied standard methods, such as pure randomization, and the novel minMSE treatment assignment method. This new technique should achieve improved overlap by balancing covariate dependencies and variances instead of focusing on individual mean values. We assess the relevance of the overlap condition by linking the estimation precision in the disjoint sub-samples to measures of overlap and balance in general. Unconfoundedness is addressed by using a rich set of administrative data on institution and municipality characteristics to study potential self-selection. We find no evidence for the violation of unconfoundedness and establish that improved overlap, and balancedness, as achieved by the minMSE method, reduce the bias of the treatment effect estimation by more than 35% compared to pure randomization, illustrating the importance of, and suggesting a solution to, addressing overlap also in (field) experiments.
Subjects: 
external validity
field experiments
generalizability
treatment effect
overlap
balance
precision
treatment assignment
unconfoundedness
self-selection bias
site-selection bias
JEL: 
C9
C90
C93
D04
ISBN: 
978-3-86304-344-5
Document Type: 
Working Paper

Files in This Item:
File
Size
916.63 kB





Items in EconStor are protected by copyright, with all rights reserved, unless otherwise indicated.