Generalized optimization algorithms for complex models - Mario Martinoli; Raffaello Seri; Fulvio Corsi (Scuola Superiore Sant'Anna, Laboratory of Economics and Management (LEM), 2024)
Linking the statistic and the machine learning literature, we provide new general results on the convergence of stochastic approximation schemes and inexact Newton methods. Building on these results, we put forward a new optimization scheme that we call generalized inexact Newton method (GINM). We extensively discuss the theoretical and the computational aspects of the GINM. The results apply to both deterministic and stochastic approximation schemes, and are particular effective in the case in which the objective function to be optimized is highly irregular and/or the stochastic equicontinuity hypothesis is violated. Examples are common in dynamic discrete choice models and complex simulation models characterized by nonlinearities and high levels of heterogeneity. The theory is supported by extensive Monte Carlo experiments.
Keywords: C61; C15; C44; 330; Optimization; stochastic approximation; Newton-Raphson methods; asymptotic convergence; M-estimation; stochastic equicontinuity