We study a class of two-player continuous time stochastic games in which agents can make (costly) discrete or discontinuous changes in the variables that affect their payoffs. It is shown that in these games there are Markov perfect equilibria of the two-sided (s,S) rule type. In such equilibria at a critical low state (resp. high state) player 1 (resp. 2) effects a discrete change in the environment. In some of these equilbria either or both players may be passive. On account of the presence of fixed costs (to discrete changes) the payoffs are non-convex and hence standard existence arguments fail. We prove that the best response map satisfies a surprisingly strong monotonicity condition and use this to establish the existence of Markov perfect equilibria. The first-best solution is also a two-sided (s,S) rule but the symmetric first-best solution has a wider s-S band than the symmetric Markovian equilibria. A further contribution of this paper is the development of a framework for continuous time games, which allows players to react instantaneously to their opponent's moves. We mention various applications of the theory and discuss in detail an application to product innovations.