While carbon taxes are generally seen as a rational policy response to climate change, knowledge about their performance from an expost perspective is still limited. This paper analyzes the emissions and cost impacts of the UK CPS, a carbon tax levied on all fossil-fired power plants. To overcome the problem of a missing control group, we propose a policy evaluation approach which leverages economic theory and machine learning for counterfactual prediction. Our results indicate that in the period 2013-2016 the CPS lowered emissions by 6.2 percent at an average cost of €18 per ton. We find substantial temporal heterogeneity in tax-induced impacts which stems from variation in relative fuel prices. An important implication for climate policy is that in the short run a higher carbon tax does not necessarily lead to higher emissions reductions or higher costs.