We study how punishment influences conditional cooperation. We ask two questions: 1) how does conditional cooperation change if a subject can be punished and 2) how does conditional cooperation change if a subject has the power to punish others. In particular, we disentangle the decision to be a conditional cooperator at all from the strength of conditional cooperation. We find that the possibility of being punished increases the strength of conditional cooperation. At the same time the possibility of being punished increases the number of free riders. In our study the net effect on cooperation still is positive. The possibility of punishing others has two effects: Substitution and responsibility. Players substitute conditional cooperation with punishment which leads to a decrease in conditional cooperation. The power to punish means more responsibility which leads to an increase in conditional cooperation. In our design the overall effect of responsibility is stronger than the effect of substitution. We conclude that the threat of being punished and the power to punish changes conditional cooperation behavior in several, unexpected, ways.