The correct definition of the product market and of the geographic market is a prerequisite for assessing market structures in antitrust cases. For hospital markets, both dimensions are controversially discussed in the literature. Using data for the German hospital market we aim at elaborating the need for differentiating the product market and at investigating the effects of different thresholds for the delineation of the geographic market based on patient flows. Thereby we contribute to the scarce empirical evidence on the structure of the German hospital market. We find that the German hospital sector is highly concentrated, confirming the results of a singular prior study. Furthermore, using a very general product market definition such as “acute in-patient care” averages out severe discrepancies that become visible when concentration is considered on the level of individual diagnoses. In contrast, varying thresholds for the definition of the geographic market has only impact on the level of concentration, while the correlation remains high. Our results underline the need for more empirical research concerning the definition of the product market for hospital services.