In model-based small area estimation an explicit statistical model is used to enhance efficiency of estimation in case of small subsamples. This model assumes a fixed relationship between the statistic of interest and a set of covariates, which is valid for all areas under consideration and can, thus, be used to stabilize estimation. In some applications, there might, however, be different subgroups of areas with specific data-generating processes, i.e. specific relationships between response variable and auxiliary information. In this case, estimation of a distinct model for each subgroup would be more appropriate than one model for all observations. If so, the definition of subgroups becomes a crucial task in the estimation process. We propose a Finite Mixture Fay Herriot-type model to account for unobserved heterogeneity in the data. More specifically, we assume that the statistic of interest stems from a mixture distribution with K components. The estimation of mixing proportions, area-specific probabilities of subgroup identity and the K sets of model parameters is then performed simultaneously. Eventually, the Finite Mixture Fay Herriot-type estimator is formulated as a weighted mean of predicts from model 1 to K, with weights given by the area-specific probabilities of subgroup identity. The suggested method is tested in a model-based simulation study. It is then applied to the problem of estimating regional rental prices on district level in Germany.