Survey vs scraped data: Comparing time series properties of web and survey vacancy data - Pablo de Pedraza; Stefano Visintin; Kea Gartje Tijdens; Gábor Kismihók (IZA Journal of Labor Economics, 2019)
This paper studies the relationship between a vacancy population obtained from web crawling and vacancies in the economy inferred by a National Statistics Office (NSO) using a traditional method. We compare the time series properties of samples obtained between 2007 and 2014 by Statistics Netherlands and by a web scraping company. We find that the web and NSO vacancy data present similar time series properties, suggesting that both time series are generated by the same underlying phenomenon: the real number of new vacancies in the economy. We conclude that, in our case study, web-sourced data are able to capture aggregate economic activity in the labor market.
Keywords: J23; J63; C22; C80; 330; web crawling; statistical inference; time series; vacancies; Labor demand; data collection