Much of the literature on the effect of housing wealth on consumption has been embedded in a simple life-cycle model in which housing price changes work as a "wealth effect". In such models, windfall gains in housing always lead to positive changes in consumption. However, this might be a fallacy of composition. Such models ignore that changes in housing wealth have distributional consequences between those planning to sell their house and those planning to buy a house. Further, since most housing is not simply financed out of current cash holdings but by mortgages, the institutions on mortgage markets have to be considered when looking at the "wealth effect" of housing. In this paper, a model is presented from which the classic Ando-Modigliani consumption function augmented by housing wealth can be deduced. It is shown that the deeper structural model from which this equation is deduced implies that changes in housing wealth are not necessarily positively correlated with consumption. It will be argued that changes both in demographics (the composition of the age groups in the population) as well in mortgage markets have led to a structural break in the effect of housing wealth on consumption in the mid-1980s in the US. In the empirical part of the paper, a vector auto-regressive model is estimated and impulse-response functions are computed that show that housing wealth changes did negatively affect consumption before the mid-1980s and positively only afterward.