Generalized stochastic gradient learning - George W. Evans; Seppo Honkapohja; Noah Williams (Center for Economic Studies and ifo Institute (CESifo), 2005)
We study the properties of generalized stochastic gradient (GSG) learning in forward-looking models. We examine how the conditions for stability of standard stochastic gradient (SG) learning both differ from and are related to E-stability, which governs stability under least squares learning. SG algorithms are sensitive to units of measurement and we show that there is a transformation of variables for which E-stability governs SG stability. GSG algorithms with constant gain have a deeper justification in terms of parameter drift, robustness and risk sensitivity.
Keywords: C65; C62; E17; E10; D83; 330; adaptive learning; E-stability; recursive least squares; robust estimation; Rationale Erwartung; Lernprozess; Prognoseverfahren; Gleichgewichtsstabilität; Theorie