Abstract: Mean shift is a robust and real-time pattern matching algorithm. At present, the total model update strategy of the mean shift algorithm still has shortage under changed scenes, e.g., target appearance changes, non-target occlusion. Therefore, the paper presents a selective sub-model update strategy for the mean shift algorithm. The proposed method treats each sub-model of target model as singleton, selects and updates sub-model and its weight based on match contributing degree of each sub-model in the current frame. The experiment result shows the proposed method is more robust and effective than the total model update strategy.