Abstract: The background model of traditional mixture of Gaussians is less robust to non-stationary scenes. This paper presents an adaptive spatiotemporal background model, combining the temporal information of per-pixel and the spatial information in the local region. Based on the temporal distribution model learned by mixture of Gaussians, the spatial background model of per-pixel is utilized to construct the spatial distribution of background in the local region by non-parametric density estimation. The robust detection is achieved by fusing the subtraction results separately based on the temporal and spatial background models. Additionally, to improve the computation efficiency, an adaptive selection strategy of the number of components of mixture of Gaussians model is proposed and integral image method is applied to calculate the spatial background model. Experimental comparisons demonstrate the effectiveness of the proposed method.