Abstract: A novel multiple kernels based object tracking algorithm using histograms of oriented gradients is proposed in this paper, which is robust to illumination change and partial occlusion. The algorithm divides the object into blocks and extracts kernel weighted histograms of oriented gradients for each block. The similarity between target model and candidate model is measured by the sum of Bhattacharyya coefficients of all the corresponding histograms. The object is tracked by maximizing the similarity measure using the mean shift algorithm. Experiments on the tracking of vehicle and human demonstrate the effectiveness of the proposed algorithm.