Abstract: Reinforcement learning gets optimal policy through trial-and-error and interaction with dynamic environment. Its properties of self-improving and online learning make reinforcement learning become one of most important machine learning methods. In this paper, we firstly survey the foundation, structure and algorithms of reinforcement learning. We also discuss the exploration oriented algorithms and the exploitation oriented algorithms in Markov and non-Markov surroundings. Then we deeply discuss some key concepts of reinforcement learning, including partially observable environment, function approximation, multi-agent reinforcement learning and rule extraction from reinforcement learning. Finally, we briefly introduce some applications of reinforcement leaning and point out some directions of reinforcement learning.