Models based on factors such as size, value, or momentum are ubiquitous in asset pricing. Therefore, portfolio allocation and risk management require estimates of the volatility of these factors. While realized volatility has become a standard tool for liquid individual assets, this measure is not available for factor models, due to their construction from the CRSP data base that does not provide high frequency data and contains a large number of less liquid stocks. Here, we provide a statistical approach to estimate the volatility of these factors. The efficacy of this approach relative to the use of models based on squared returns is demonstrated for forecasts of the market volatility and a portfolio allocation strategy that is based on volatility timing.