The banking system is highly interconnected and these connections can be conveniently represented as an interbank network. This survey presents a systematic overview of the recent advances in the theoretical literature on interbank networks. We assess our current understanding of the structure of interbank networks, of how network characteristics affect contagion in the banking system and of how banks form connections when faced with the possibility of contagion and systemic risk. In particular, we highlight how the theoretical literature on interbank networks offers a coherent way of studying interconnections, contagion processes and systemic risk, while emphasizing at the same time the challenges that must be addressed before general results on the link between the structure of the interbank network and financial stability can be established. The survey concludes with a discussion of the policy relevance of interbank network models with a special focus on macro-prudential policies and monetary policy.