Abstract: Most of the algorithms for object detection are sensitive to background clutter and occlusion, and cannot localize the edge of the object. In this paper, the authors present an approach based on the local shape fragments. Firstly, the model of the object is learnt from the training set. The model is composed of shape fragments and the model of the object is in multi-scales. In this way, the method is invariant to scale changes. Then, shape fragments are extracted from the test image, which is in the same way at the model shape fragments extracting method. According to the similarity of the shape fragments, candidate shape fragments are got from the test image. After that, according to the rotation relationship between the fragments on the model and the test image, rotation angle between the fragments are estimated. Finally, the object detection problem is casted to the peak detection problem in Hough space by combining the probability Hough transform. After peak positions are achieved in the Hough space, each candidate shape fragment is traced back to verify whether it belongs to the object. Test results show that the proposed method is valid.