Abstract: This paper proposes a novel clustering segmentation algorithm. Based on the criterion of the minimization of average dispersion within segments and the maximization of average dispersion between segments, the optimum speech segmentation has been approached by this clustering method. Compared with former classical methods, this algorithm has improved the performance in the phoneme segmentation of Chinese syllables significantly. The statistical results of some preliminary experiments for isolated Chinese words are reported, which can be used as reference in the phonemes recognition of Chinese speech.