Information-Theoretic Time-Varying Density Modeling - Bram van Os (Tinbergen Institute, 2023)
We present a comprehensive framework for constructing dynamic density models by combining optimization with concepts from information theory. Specifically, we propose to recursively update a time-varying conditional density by maximizing the log-likelihood contribution of the latest observation subject to a Kullback-Leibler divergence (KLD) regularization centered at the one-step ahead predicted density. The resulting Relative Entropy Adaptive Density (READY) update has attractive optimality properties, is reparametrization invariant and can be viewed as an intuitive regularized estimator of the pseudo-true density. Popular existing models, such as the ARMA(1,1) and GARCH(1,1), can be retrieved as special cases. Furthermore, we show that standard score-driven models with inverse Fisher scaling can be derived as convenient local approximations of the READY update. Empirical usefulness is illustrated by the modeling of employment growth and asset volatility.
Keywords: 330