Abstract: This paper introduces a gait expression and recognition algorithm based on the feature matrix constructed from the primary motion contours of gait. Three segments are extracted from silhouette contours to represent primary motions of gait. Three feature matrices are constructed based on the horizontal distances from the segmented curves to the silhouette centroid. Principal component analysis (PCA) and multiple discriminant analysis (MDA) are utilized to reduce redundant data and separate different classes, respectively. The proposed algorithm is evaluated on USF dataset, and its performance is compared with three typical algorithms. Experimental results show that this algorithm has a higher mean recognition rate and achieves better performance in robustness than the other algorithms.