Q3-D3-LSA - Lukas Borke; Wolfgang Karl Härdle (Humboldt University of Berlin, Collaborative Research Center 649 - Economic Risk, 2016)
QuantNet 1 is an integrated web-based environment consisting of different types of statistics-related documents and program codes. Its goal is creating reproducibility and offering a platform for sharing validated knowledge native to the social web. To increase the information retrieval (IR) efficiency there is a need for incorporating semantic information. Three text mining models will be examined: vector space model (VSM), generalized VSM (GVSM) and latent semantic analysis (LSA). The LSA has been successfully used for IR purposes as a technique for capturing semantic relations between terms and inserting them into the similarity measure between documents. Our results show that different model configurations allow adapted similarity-based document clustering and knowledge discovery. In particular, different LSA configurations together with hierarchical clustering reveal good results under M3 evaluation. QuantNet and the corresponding Data-Driven Documents (D3) based visualization can be found and applied under http://quantlet.de. The driving technology behind it is Q3-D3-LSA, which is the combination of 'GitHub API based QuantNet Mining infrastructure in R', LSA and D3 implementation.
Keywords: 330; QuantNet; D3; GitHub API; text mining; document clustering; similarity; semantic web; generalized vector space model; LSA; visualization