Engle and Manganelli (2004) propose CAViaR, a class of models suitable for estimating conditional quantiles in dynamic settings. Engle and Manganelli apply their approach to the estimation of Value at Risk, but this is only one of many possible applications. Here we extend CAViaR models to permit joint modeling of multiple quantiles, Multi-Quantile (MQ) CAViaR. We apply our new methods to estimate measures of conditional skewness and kurtosis defined in terms of conditional quantiles, analogous to the unconditional quantile-based measures of skewness and kurtosis studied by Kim and White (2004). We investigate the performance of our methods by simulation, and we apply MQ-CAViaR to study conditional skewness and kurtosis of S&P 500 daily returns.