Abstract: This paper proposes a competitive Takagi-Sugeno fuzzy reinforcement learning network (CTSFRLN) for solving complicated learning tasks of continuous domains. The proposed CTSFRLN is constructed by combining Takagi-Sugeno type fuzzy inference systems with action-value-based reinforcement learning methods. Two competitive learning algorithms are derived, including the competitive Takagi-Sugeno fuzzy Q-learning and the competitive Takagi-Sugeno fuzzy advantage learning. These learning methods lead to so called Takagi-Sugeno fuzzy variable structure controllers. Simulation experiments on the double inverted pendulum system demonstrate the superiority of these learning methods.