A Comment on "Delivering Remote Learning Using a Low-Tech Solution: Evidence from a Randomized Controlled Trial in Bangladesh" - Lenka Fiala; Jack Fitzgerald; Essi Kujansuu; Derek Mikola; David Valenta; Juan P. Aparicio; Michael Wiebe; Matthew D. Webb; Abel Brodeur (Institute for Replication (I4R), 2025)
Wang et al. (2024) report that Bangladeshi students randomly given access to lessons on a phone server saw significant learning gains during COVID- 19 school closures. We identify three sets of anomalies. First, this experiment shares participants with another experiment conducted simultaneously in the same region, but test scores for the same children systematically differ between the two experiments. Second, test scores for treated participants exhibit a uniform upward shift that is completely insensitive to the number of lessons children complete. Third, numerous documentation inconsistencies (e.g., concerning survey materials, randomization procedures, etc.) cast doubt on the study's data.
Keywords: B41; C12; C93; I21; I24; 330; Reproduction; school closures; remote education; COVID-19; randomized controlled trial; Bangladesh