Abstract: An automatic visual--thermal image sequence registration method based on co-motion was proposed. Different from other methods, co-motion (concurrent motions) statistics feature was adopted to regist heterogeneous image sequences. Compared with registration based on co-motion between homogeneous image sequences, our method faced many difficulties. By analyzing the difficulties, we proposed the corresponding point pairs extraction method and outliers removal method, which are suitable to visual--thermal image sequence. We demonstrated the performance of the method on eight groups of visual--thermal synchronous video sequences, and the results showed that the proposed algorithm carried out precise image registration under the change of image rotation, translation, scaling and viewing angle. Experiments demonstrated the accuracy and robustness of the proposed method.