Abstract: This paper deals with the relationship between BP algorithm for neural networks and differential dynamic systems. It is proposed that the iteration formula of BP algorithm is equivalent to Euler method of differential dynamic system under certain conditions, and the asymptotic solutions of the two formulas are consistent. For neural networks' BP algorithm and their corresponding differential dynamics, the solution existence theorem, exclusiveness theorem and zero solution stability theorem of dynamic system are presented. It is also theoretically proved that asymptotic solutions given by neural networks' BP algorithm are equivalent to that computed by any numerical method for differential dynamic systems under certain conditions. Therefore, training a neural network can be converted to computing numerical solution of differential dynamic systems. Also, an example to train the BP network by modified Euler method is presented.