-
摘要: 自适应动态规划（Adaptive dynamic programming，ADP）作为最优控制领域的近似优化方法，是求解复杂非线性系统最优控制问题的有力工具.近年来，已成为控制理论与计算智能领域的研究热点.本文着重介绍ADP算法的理论研究进展及其在航空航天领域的应用.分析了几种典型的制导律优化设计方法，以及ADP方法在导弹制导律设计中的应用现状和前景.Abstract: Adaptive dynamic programming (ADP) is a powerful tool for optimal control of complicated nonlinear system, which is a novel approximate optimal control method. Recently, it has become a hot topic in the field of control theory and computational intelligence. This paper focuses on giving a review of ADP on the development of ADP algorithms and its aerospace applications. The design methods of classic missile guidance law are introduced, as well as the present and potential applications of ADP in the guidance law design of missiles.
-
Key words:
- Adaptive dynamic programming (ADP) /
- optimal control /
- missile /
- guidance law
-
1. Introduction
In recent years, the control methods for systems affected by uncertainties and disturbances have been focused by researchers [1]-[7]. Compared to other control methods, sliding mode control (SMC) has attracted a significant interest due to its conceptual simplicity, easy implementation, and robustness to external disturbances and model uncertainties [8]-[10]. SMC is a nonlinear control strategy that forces the closed loop trajectories to the switching manifold in finite-time using a discontinuous feedback control action. Therefore, SMC has been widely used in many applications, such as motion control, process control, etc. However, the conventional sliding mode control has a violent chattering phenomenon in the process, which can degrade the system performance. Moreover, it can guarantee invariance only if the uncertainties and disturbances satisfy the matching conditions, and cannot attenuate mismatched uncertainties and disturbances effectively.
Note that the matched and mismatched uncertainties and disturbances widely exist in practical engineering, such as power systems [11], electronic systems [12], [13] and motor systems [14]. The sliding motion of the traditional SMC is severely affected by the mismatched uncertainties and disturbances, and the well-known robustness of SMC does not hold any more. Algorithms like LMI-based control [15], [16], adaptive control [17], [18], back stepping based control [19], and integral sliding-mode control [20], [21] are proposed to handle mismatched uncertainties in a robust way, but the price is that the nominal control performance is compromised.
As a practical alternative approach, disturbance observer-based control has been proven to be promising and effective in compensating the effects of unknown external disturbances and model uncertainties in control systems as well as it will not deteriorate the existing controller [22], [23]. It could completely remove the non-vanishing disturbances from system as long as they can be estimated accurately [24]. Recently, several authors introduced a disturbance observer (DOB) for SMC to alleviate the chattering problem and retain its nominal control performance [25]-[29]. The idea is to construct the control law by combining the SMC feedback with the disturbance estimation based-feed forward compensation straightforwardly. However, these methods given in [25]-[29] are only available for the matched uncertain systems. A nonlinear extension of DO has been proposed by W. H. Chen, which estimates matched as well as mismatched disturbances [30], [31]. Reference [1] extends a recent result on sliding mode control for general nth order systems with a larger class of mismatched uncertainties by proposing an extended disturbance observer. Reference [13] investigates an extended state observer (ESO)-based sliding mode control (SMC) approach for pulse width modulation-based DC-DC buck converter systems subject to mismatched disturbances, the proposed method obtains a better disturbance rejection ability even the disturbances do not satisfy the so-called matching condition. A novel sliding-mode control based on the disturbance estimation by a nonlinear disturbance observer (NDOB) based SMC in [14] is only proposed to deal with mismatched uncertainties, it can ensure the system performance and reduce the chattering.
In this paper, aiming to improve the performance of the system affected by mismatched/matched uncertainties and disturbances, a novel nonlinear control scheme is proposed, where the SMC scheme is integrated with NDOB. By fully taking into account the estimation value of disturbances, a new sliding-mode surface is firstly designed which is insensitive to not only matched disturbances but also mismatched ones. In this paper, the contributions are listed as follows:
1) The control is proposed for a general system of $n$ order, having mismatched/matched uncertainties and disturbances.
2) The novel sliding surface is extended for a general system of n order and modified to enable improvement in the performance of the system without causing a large increase in the control.
3) The proposed method exhibits the properties of nominal performance recovery and chattering reduction as well as excellent dynamic and static performance as compared with the traditional SMC.
The paper is organized as follows: the problem of nominal sliding mode controller design with mismatched and matched uncertainties and disturbances for a class of nonlinear system is stated in Section 2. Generalization of NDOB, novel sliding surface design and the stability analysis are derived in Section 3. The simulation results are presented in Section 4, followed by some concluding remarks in Section 5.
2. Problem Statement
Consider a class of single-input single-output dynamic systems with matched and mismatched uncertainties, depicted by
$ \begin{equation} \left\{\begin{array}{llllll} & \!\!\!\!\!\!\!{{\dot x}_i} ={x_{i + 1}} + {d_i}(t)\\ & ~\vdots\\ & \!\!\!\!\!\!\!{{\dot x}_n} =a(x) + b(x)u + {d_n}(t)\\ & \!\!\!\!\!\!\!y ={x_1} \end{array} \right. \ i= 1, \ldots, n - 1\ \end{equation} $
(1) where $x = {\left[{{x_1} \ldots {x_n}} \right]^T}$ is the state vector, $u$ is the control input, $y$ is the controlled output, ${d_i}(t)$ and ${d_n}(t)$ are the mismatched and matched uncertainties and disturbances, respectively. $a(x)$ and $b(x)$ denote smooth nominal functions.
Taking a second-order system as an illustration, we can get the control model of the system:
$ \begin{equation} \left\{ \begin{array}{lllll} & \!\!\!\!\!\!\!{{\dot x}_1} ={x_2} + {d_1}(t)\\ & \!\!\!\!\!\!\!{{\dot x}_2} = a(x) + b(x)u + {d_2}(t)\\ & \!\!\!\!\!\!\!y ={x_1} \end{array} \right.. \end{equation} $
(2) Assumption 1: The lumped disturbances ${d_1}(t)$ , ${d_2}(t)$ in (2) are bounded and defined by .
The sliding mode surface and control law of the traditional SMC is usually designed as follows:
$ \begin{equation} {s_1} = {x_2} + {k_1}{x_1} \end{equation} $
(3) $ \begin{equation} u = - {b^{ - 1}}(x)\left[{{k_1}{x_2} + {\eta _1}{\rm sgn}(s_1) + a(x)} \right] \end{equation} $
(4) where ${k_1} > 0$ is a design constant, ${\eta _1} > 0$ is the switching gain to be designed. Taking the derivative of (3), and combining (2) and (4) gives
$ \begin{equation} {\dot s_1} = - {\eta _1}{\rm sgn}(s_1) + {k_1}{d_1}(t) + {d_2}(t). \end{equation} $
(5) The states of (2) will reach the sliding mode surface ${s_1} = 0$ in finite time if the switching gain ${\eta _1}$ in the control law (4) is devised such that ${\eta _1} > {d^*}$ . Once the nominal sliding surface ${s_1} = 0$ is reached, the sliding motion is obtained and given by
$ \begin{equation} {\dot x_1} + {k_1}{x_1} = {d_1}(t). \end{equation} $
(6) Remark 1: Equation (6) implies that if the ${d_1}(t) = 0$ , the system states can be driven to the desired equilibrium point, which implies that the conventional SMC is insensitive to matched disturbance. However, in the existence of mismatched disturbance ${d_1}(t) \ne 0$ , the system state ${x_1}$ is affected by the mismatched disturbance and does not converge to zero in finite time although the control law can force the system states to reach the sliding-mode surface in a finite time. To this end, it is an essential reason why the nominal SMC design is only insensitive to matched disturbance but sensitive to mismatched disturbance.
3. Main Results
In this paper, the matched and mismatched disturbance rejection problem is considered for (1). A novel sliding mode controller based on NDOB is proposed by the following two steps. First, an NDOB is employed to estimate the matched and mismatched disturbances. A novel sliding mode controller is then designed for the system based on the disturbance observation. The control structure of the second-order system is designed as Fig. 1.
图 1 Control structure of second-order system.Fig. 1 Control structure of second-order system.3.1 Nonlinear Disturbance Observer (NDOB) Design
A nonlinear disturbance observer (NDOB), which is adopted to estimate the disturbance in [14]. Consider a class of nonlinear systems with uncertainties and external disturbance:
$ \begin{equation} \left\{ \begin{array}{ccccc} \dot x =f(x) + {g_1}(x)u + {g_2}(x)d(t)\\ y =h(x)~~~~~~~~~~~~~~~~~~~~~~~~~~ \end{array} \right.. \end{equation} $
(7) A NDOB is used to estimate the compound disturbance of the system and compensate accordingly, in order to improve the robustness of the controller. The NDOB is introduced and depicted by
$ \begin{equation} \left\{ \begin{array}{lllllll} & \!\!\!\!\!\!\!\dot z = - l{g_2}(x)z - l\left[{{g_2}(x)lx + f(x) + {g_1}(x)u} \right]\\ & \!\!\!\!\!\!\!\hat d(t) = z + lx \end{array} \right. \end{equation} $
(8) where $\hat d(t)$ is the estimation vector of the disturbance vector $d(t)$ , and $z$ is the internal state vector of the nonlinear observer. $l$ is the disturbance observer gain matrix to be designed.
Assumption 2: The derivative of the disturbance in system (1) is bounded and satisfies
$ \begin{equation*}\mathop {\lim }\limits_{t \to \infty } d_i^{(j)}(t) = 0, \quad \quad i = 1, \ldots, n; j = 1, \ldots, n - 1.\end{equation*} $
Remark 2: This assumption satisfies the requirements of the simulation sample. An extended disturbance observer is proposed in [1], which can observe a class of systems with mismatched uncertainties and get for the sliding mode surface and $u$ . A nonlinear disturbance observer is designed to observe and get for the sliding mode surface and $u$ in our paper.
The disturbance estimation error of the NDOB is defined:
$ \begin{equation} {e_d}(t) = d(t) - \hat d(t). \end{equation} $
(9) The error dynamics can be derived as
$ \begin{align} & {{{\dot{e}}}_{d}}(t)=\dot{d}(t)-\dot{\hat{d}}(t) \\ & \ \ \ \ \ \ \ \ \ =\dot{d}(t)-\dot{z}-l\left[f(x)+{{g}_{1}}(x)u+{{g}_{2}}(x)d(t) \right]. \\ \end{align} $
(10) Substituting the value of $\dot z$ , error dynamics reduce to
$ \begin{equation} {\dot e_d}(t) = \dot d(t) - {lg_2}(x){e_d}(t). \end{equation} $
(11) Lemma 1 [7]: Suppose that Assumptions 1 and 2 are satisfied for (7). The disturbance estimation $\hat d(t)$ of NDOB (8) can track the disturbance $d$ of (7) asymptotically as long as the observer gain $l$ is chosen such that $lg_2(x) > 0$ holds, which implies that ${\dot e_d}(t) + {lg_2}(x){e_d}(t) = 0$ , the estimation error will converge to zero asymptotically.
3.2 Novel SMC Control With NDOB of Second-order System
Consider the system
$ \begin{equation} \dot x = f(t, x, u) \end{equation} $
(12) where is piecewise continuous in $t$ and locally Lipschitz in $x$ and $u$ . The input $u(t)$ is a piecewise continuous, bounded function of $t$ for all $t \ge {0}$ .
Definition 1: The system (12) is said to be input-to-state stable (ISS) if there exist a class $\kappa \iota $ function and a class $\kappa$ function $\gamma $ such that for any initial state $x({t_0})$ and any bounded input $u(t)$ , the solution $x(t)$ exists for all $t \ge {t_0}$ and satisfies
$ \begin{equation} \left\| {x(t)} \right\| \le \beta (\left\| {x({t_0})} \right\|, t- {t_0}) + \gamma (\mathop {\sup }\limits_{{t_0} \le \tau \le t} \left\| {u(\tau )} \right\|). \end{equation} $
(13) Such a function $\gamma$ in (13) is referred to as an ISS-gain for (12). ISS implies that (12) is bounded-input bounded-state stable when $u \ne 0$ and its zero solution (with $u = 0$ ) is globally asymptotically stable.
Lemma 2 [14]: Consider a nonlinear system $\dot x = F(x, w)$ , which is input-to-state stable (ISS). If the input satisfies $\mathop {\lim }_{t \to \infty } w(t) = 0$ , then the state .
A novel sliding mode manifold for (2) under matched and mismatched disturbances is designed as
$ \begin{equation} {s_2} = {x_2} + {k_2}{x_1} + {\hat d_1}(t). \end{equation} $
(14) Theorem 1: Considering the above (2) with matched and mismatched disturbances, we proposed sliding-mode surface (14), if the control law is designed as follows:
$ \begin{align} u = & - {b^{ - 1}}(x)\Big\{ {k_2}\left[{{x_2} + {{\hat d}_1}(t)} \right] + {\eta _2}{\rm sgn}({s_2})\nonumber\\ & + a(x) + {{\hat d}_2}(t) + {{\dot {\hat d}_1}}(t) \Big\}. \end{align} $
(15) Suppose the second-order system satisfies Assumptions 1 and 2, the observer gain $l$ is chosen such that ${\lg}{_2}(x) > 0$ holds, if the switching gain is chosen such that and ${k_2} > 0$ , then the closed-loop system is asymptotically stable.
Proof: Consider a candidate Lyapunov function as
$ \begin{equation} {V_1} = \frac{1}{2}s_2^T{s_2}. \end{equation} $
(16) Taking derivative of $V$ in (16), we obtain that
$ \begin{equation} {\dot V_1} = {s_2}{\dot s_2} = {s_2}({\dot x_2} + {k_2}{\dot x_1} + \dot{\hat{d}}_1 (t). \end{equation} $
(17) Substituting (15) into (2) yields
$ \begin{align} {{\dot x}_2} = & a(x) + b(x)u + {d_2}(t)\nonumber\\ = & a(x) - b(x) \times {b^{ - 1}}(x)\Big\{ {k_2}\left[{{x_2} + {{\hat d}_1}(t)} \right] + {\eta _2}{\rm sgn}({s_2})\nonumber\\ & + a(x) + {{\hat d}_2}(t) + {{\dot {\hat d}_1}(t)}\Big\} + {d_2}(t). \end{align} $
(18) Substituting (18) into (17) yields
$ \begin{align} {\dot V}_1= & {s_2} \Big\{ - {k_2}\left[{{x_2} + {{\hat d}_1}(t)} \right] - {\eta _2}{\rm sgn}(s) - {{\hat d}_2}(t)\nonumber\\ & - {{\dot {\hat d}_1}(t)} + {d_2}(t) + {k_2}{x_2} + {k_2}{d_1}(t) + {{\dot {\hat d}_1}(t)} \Big\}\nonumber\\[1mm] = & {s_2} \Big\{ - {k_2}{{\hat d}_1}(t) - {\eta _2}{\rm sgn}(s) - {{\hat d}_2}(t) + {d_2}(t) + {k_2}{d_1}(t) \Big\}\nonumber\\ = & {s_2}\left( { - {\eta _2}{\rm sgn}({s_2}) + {k_2}{e_{d1}} + {e_{d2}}} \right)\nonumber\\[1mm] \le & \left[{-{\eta _2} + {k_2}{e_{d1}} + {e_{d2}}} \right]\left| {{s_2}} \right|\nonumber\\[1mm] = & - \sqrt 2 \left[{{\eta _2}-({k_2}{e_{d1}} + {e_{d2}})} \right]V_1^{\frac{1}{2}} \end{align} $
(19) where ${e_{d1}} = {d_1} - {\hat d_1}$ , .
It can be derived from (19) that the system states will reach the defined sliding surface ${s_2} = 0$ in finite time when . The condition implies
$ \begin{equation} {\dot x_1} = - {k_2}{x_1} + {e_{d1}}. \end{equation} $
(20) With this result, it can be derived that (20) is ISS. According to Lemma 2, we know that the system states satisfy and . This implies that the system states can converge to the desired equilibrium point along the sliding surface asymptotically under the proposed control law.
Remark 3: Since the matched and mismatched disturbances have been precisely estimated by the NDOB, the switching gain of the proposed method can be designed much smaller than those of the traditional SMC, because the magnitude of the estimation error ${e_d}$ is much smaller than the magnitude of the disturbance $d$ . The chattering problem can be alleviated to some extent in the case the nominal performance of the sliding mode control is retained.
3.3 A Third-order System Case
Consider the following third-order system, depicted by
$ \begin{equation} \left\{ \begin{array}{llll} {{\dot x}_1} = {x_2} + {d_1}(t)\\ {{\dot x}_2} = {x_3} + {d_2}(t)\\ {{\dot x}_3} = a(x) + b(x)u + {d_3}(t)\\ y = {x_1}. \end{array} \right. \end{equation} $
(21) A sliding mode manifold for (21) is designed as
$ \begin{equation} {s_3} = {x_3} + {x_2} + {k_3}{x_1} + {\hat d_1}(t) + {\hat d_2}(t) + \dot{\hat{d}}_1(t). \end{equation} $
(22) Theorem 2: Considering the above (21) with matched and mismatched disturbances, we proposed sliding-mode surface (22), if the control law is designed as follows:
$ \begin{align} u= & - {b^{ - 1}}(x)\Big\{ {k_3}\left[{{x_2}+ {{\hat d}_1}(t)} \right] + {\eta _3}{\rm sgn}({s_3}) + {x_3} + a(x)\nonumber\\ & + {{\hat d}_2}(t)+ {{\hat d}_3}(t) + {{\dot {\hat d}_1}(t)}+{{\dot {\hat d}_2}(t)}+{{\ddot {\hat d}_1}(t)} \Big\}. \end{align} $
(23) Suppose the third-order system satisfies Assumptions 1 and 2, the observer gain $l$ is chosen such that ${\lg}{_2}(x) > 0$ holds, if the switching gain is chosen such that ${k_3} > 0$ and , then the closed-loop system is asymptotically stable.
3.4 A General High-order System Case
A sliding mode manifold for (1) in the case $n > 2$ is designed as
$\begin{equation} {s_n} = \sum\limits_{i = 2}^n {{x_i}} + {k_n}{x_1} + \sum\limits_{j = 1}^{n - 1} {\sum\limits_{i = 1}^{n - j} {\hat d_i^{(j - 1)}(t)} }. \end{equation} $
(24) Theorem 3: Considering the above system (1) with matched and mismatched disturbances, we proposed sliding-mode surface (24), if the control law is designed as follows:
$ \begin{align} u = & - {b^{ - 1}}(x)\Big\{ {k_n}\left[{{x_2} + {{\hat d}_1}(t)} \right] + {\eta _n}{\rm sgn}({s_n}) + \sum\limits_{i = 3}^n {{x_i}} \nonumber\\ & + a(x) + \sum\limits_{i = 2}^n {{{\hat d}_i}(t)} + \sum\limits_{j = 1}^{n - 1} {\sum\limits_{i = 1}^{n - j} {\hat d_i^{(j)}(t)} } \Big\}. \end{align} $
(25) Suppose the general high-order system satisfies Assumptions 1 and 2, the observer gain $l$ is chosen such that ${\lg}{_2}(x)> 0$ holds, if the switching gain is chosen such that and ${k_n} > 0$ , then the closed-loop system is asymptotically stable.
Proof: Consider a candidate Lyapunov function as
$ \begin{equation} {V_n} = \frac{1}{2}s_n^T{s_n}. \end{equation} $
(26) Taking derivative of $V$ in (26), we obtain that
$ \begin{align} {{\dot V}_n} = & {s_n}{{\dot s}_n} = {s_n}\big(\sum\limits_{i = 2}^n {{{\dot x}_i}} + {k_n}{{\dot x}_1} + \sum\limits_{j = 1}^{n - 1} {\sum\limits_{i = 1}^{n - j} {\hat d_i^{(j)}(t)} } \big) \nonumber\\ = & {s_n}\Big[ {x_3} + {d_2} + {x_4} + {d_3} + \cdots + {x_n} + {d_{n-1}} + a(x)\nonumber\\ & + b(x)u + {d_n}(t) + {k_n}{x_2} + {k_n}{d_1} + \sum\limits_{j = 1}^{n-1} {\sum\limits_{i = 1}^{n-j} {\hat d_i^{(j)}(t)} } \Big]. \end{align} $
(27) Substituting (25) into (27) yields
$\begin{align}{{\dot V}_n}= & {s_n}\Bigg\{ {x_3} + {d_2} + {x_4} + {d_3} + \cdots + {x_n} + {d_{n - 1}} + a(x)\nonumber\\ & - b(x)\!\! \times\!\! {b^{ - 1}}(x)\bigg\{ {k_n}\left[{{x_2} + {{\hat d}_1}(t)} \right] + {\eta _n}{\rm sgn}({s_n})\nonumber\\ & + \sum\limits_{i = 3}^n {{x_i}} + a(x) + \sum\limits_{i = 2}^n {{{\hat d}_i}(t)} + \sum\limits_{j = 1}^{n - 1} {\sum\limits_{i = 1}^{n - j} {\hat d_i^{(j)}(t)} } \bigg\}\nonumber\\ & + {d_n}(t) + {k_n}{x_2} + {k_n}{d_1} + \sum\limits_{j = 1}^{n - 1} {\sum\limits_{i = 1}^{n - j} {\hat d_i^{(j)}(t)} } \Bigg\}\nonumber\\ = & {s_n}\Bigg[{-{\eta _n}{\rm sgn}({s_n}) + {k_n}{e_{d1}} + \sum\limits_{i = 2}^n {{e_{di}}} } \Bigg]\nonumber\\ \le & \left| {{s_n}} \right|( - {\eta _n} + {k_n}{e_{d1}} + \sum\limits_{i = 2}^n {{e_{di}}}). \end{align} $
(28) It can be derived from ${\dot V_n}$ (28) that the system states will reach the defined sliding surface ${s_n} = 0$ in finite time when . The condition ${s_n} = 0$ implies:
$ \begin{equation} x_1^{(n - 1)} + x_1^{(n - 2)} + \ldots + {\dot x_1} + {k_n}{x_1} - \sum\limits_{j = 1}^{n - 1} {\sum\limits_{i = 1}^{n - j} {e_{di}^{(j - 1)}}} = 0. \end{equation} $
(29) We can define
$ \begin{equation} \left\{ {\begin{array}{*{20}{lll}} {{Y_1} = {x_1}}\\ {{Y_2} = {{\dot x}_1}}\\ ~~~~\vdots \\ {{Y_{n - 1}} = x_1^{(n - 2)}} \end{array}} \right.. \end{equation} $
(30) Equation (29) is given by
$ \begin{equation} \dot Y\! =\! \left[\! {\begin{array}{*{20}{c}} 0 & 1 & 0 & \cdots & 0\\ 0 & 0 & 1 & 0 & 0\\ \vdots & \vdots & \ddots & \vdots & \\ 0 & 0 & \cdots & 0 & 1\\ {-{k_n}} & {-1} & {-1} & \cdots & { - 1} \end{array}} \right]Y + \left[{\begin{array}{*{20}{c}} 0\\ 0\\ \vdots \\ 0\\ {\sum\limits_{j = 1}^{n-1} {\sum\limits_{i = 1}^{n-j} {e_{di}^{(j-1)}} } } \end{array}} \right]. \end{equation} $
(31) Equation (31) can also be expressed as
$ \begin{equation} \dot Y = AY + B\sum\limits_{j = 1}^{n - 1} {\sum\limits_{i = 1}^{n - j} {e_{di}^{(j - 1)}} } \end{equation} $
(32) where ,
In view of Assumption 1 and (11), we can conclude:
$ \begin{equation} \begin{array}{llllllll} \sum\limits_{j = 1}^{n - 1} {\sum\limits_{i = 1}^{n - j} {e_{di}^{(j - 1)}} } = \sum\limits_{i = 1}^{n - 1} {{e_{di}}} + \sum\limits_{i = 1}^{n - 2} {({{\dot d}_i} - {{\lg}_2}{e_{di}})} \\+ \sum\limits_{i = 1}^{n - 3} {({{\ddot d}_i} - {{\lg}_2}{{\dot d}_i} + {l^2}g_2^2{e_{di}})} + \cdots \\+ \sum\limits_{i = 1}^2 {(d_i^{n - 3} - {{\rm lg}_2}d_i^{n - 4} + {l^2}g_2^2d_i^{n - 5} - \cdots - {l^{n - 3}}g_2^{n - 3}{e_{di}})} \\ + (d_1^{n - 2} - {{\rm lg}_2}d_1^{n - 3} + {l^2}g_2^2d_1^{n - 4} - \cdots - {l^{n - 2}}g_2^{n - 2}{e_{d1}}). \end{array} \end{equation} $
(33) Combining Lemma 1, (11) and (33) can get:
$ \begin{equation} \mathop {\lim }\limits_{t \to \infty } \sum\limits_{j = 1}^{n - 1} {\sum\limits_{i = 1}^{n - j} {e_{di}^{(j - 1)}} } = 0. \end{equation} $
(34) According to the principle of Hurwitz matrix, it can be designed $A$ as a Hurwitz matrix by selecting a proper ${k_n}$ . Taking as an input of (32), it follows from Lemma 2 that (32) is ISS. It is derived from the definition of ISS that the state of (32) converges to zero as $t \to \infty $ , that is, . This implies that the system states can converge to zero along the sliding surface asymptotically under the proposed control law.
Remark 4: The above proof implies that states of system can be driven to the desired equilibrium point and the control law can force the system states to reach the sliding-mode surface in finite time. This is the main reason why the proposed NDOB-based SMC method is insensitive to matched uncertainties and disturbances as well as mismatched uncertainties and disturbances.
4. Simulations
To evaluate the effectiveness of the proposed method, two examples are given below.
4.1 Numerical Example
$ \begin{equation} \left\{ \begin{array}{l} {{\dot x}_1} = {x_2} + {d_1}(t)\\ {{\dot x}_2} = {x_3} + {d_2}(t)\\ {{\dot x}_3} = - 2{x_2} - {x_3} + {e^{{x_1}}} + u + {d_3}(t)\\ y = {x_1}. \end{array} \right. \end{equation} $
(35) In order to show the advantages of the NDOB-based SMC method proposed in this paper compared with the nominal sliding control, we will use simulations to compare the performance between them for (35). The control parameters of the two control methods are listed in Table Ⅰ. Consider the initial states of (35) as . Mismatched/matched uncertainties and disturbances ${d_1} = 1.5, {d_2} = 0.5, {d_3} = 1$ are imposed on the system at 25, 30, 15 sec respectively. The simulation results are shown in Figs. 2-4.
表 Ⅰ Control Parameters for the Numerical Example in Case 1Table Ⅰ Control Parameters for the Numerical Example in Case 1Controllers Parameters SMC1 k = 8, η= 16 SMC2 k = 8, η= 10 NDOB-SMC k = 8, η = 10, l = diag{6, 6, 6} 
图 4 Reference/estimation uncertainties and disturbances.Fig. 4 Reference/estimation uncertainties and disturbances.The reference/estimation uncertainties and disturbances of the system are shown in Fig. 4. It can be seen from Fig. 4 that the proposed control gives a better estimation of the disturbance and a better performance.
4.2 Application to PMSM Speed Control System
In $d-q$ coordinates the permanent magnet synchronous motor (PMSM) dynamic equation can be written as
$ \begin{equation} \dot \omega = - a\omega + b{i_q} - d \end{equation} $
(36) where $a = B/J$ , $b = 3p{\psi _f}/{(2J)}$ , $d = T_L/J$ . $\omega$ is the rotor angular velocity, $p$ is the number of pole pairs, ${\psi _f}$ is the flux linkage, ${T_L}$ is the load torque, $B$ is the viscous friction coefficient, $J$ is the moment of inertia.
The state variable of speed error is defined as and its derivative as follows:
$ \begin{equation} {\dot x_1} = {x_2} = {\dot \omega _{\rm ref}} - \dot \omega = {\dot \omega _{\rm ref}} + a\omega - b{i_q} + d \end{equation} $
(37) where ${\omega _{\rm ref}}$ is the reference speed signal.
The speed error derivative dynamic equation of the motor can be expressed as follows, with the parameters variations taken into account:
$ \begin{eqnarray} {{\dot x}_1} & = & {{\dot \omega }_{\rm ref}} + a\omega + \Delta a\omega - b{i_q} - \Delta b{i_q} + d + \Delta d\nonumber\\ & = & {x_2} + \Delta a\omega - \Delta b{i_q} + \Delta d\nonumber\\ & = & {x_2} + {d_1} \end{eqnarray} $
(38) where $\Delta a$ , $\Delta b$ , and $\Delta d$ are parameter variations of $a$ , $b$ , $d$ , and is considered as mismatched uncertainties.
The second-order model of speed error derivative dynamic equation of PMSM system is described by
$ \begin{equation} {\dot x_2} = {\ddot \omega _{\rm ref}} - \ddot \omega = {\ddot \omega _{\rm ref}} + a\dot \omega - b{\dot i_q} + \dot d. \end{equation} $
(39) ${\dot x_2}$ can be expressed as follows, with the parameters variations taken into account:
$ \begin{eqnarray} {{\dot x}_2} & = & {{\ddot \omega }_{\rm ref}} + a\dot \omega + \Delta a\dot \omega - b{{\dot i}_q} - \Delta b{{\dot i}_q} + \dot d + \Delta \dot d\nonumber\\ & = & - a{x_2} - b{{\dot i}_q} + {d_2}\nonumber\\ & = & - a{x_2} - bu + {d_2} \end{eqnarray} $
(40) where $u = {\dot i_q}$ , .
The second-order model of the PMSM speed regulation system can be represented in the following state-space form:
$ \begin{equation} \left\{ \begin{array}{l} {{\dot x}_1} = {x_2} + {d_1}\\ {{\dot x}_2} = - a{x_2} - bu + {d_2} \end{array} \right.. \end{equation} $
(41) To demonstrate the efficiency of the proposed method, simulation studies are carried out in this section. The parameters of the PMSM are given as follows: ${R_s} = 1.62 \Omega $ , ${L_d} = {L_q}$ = 0.005 H, B = 7.403 $\times {10^{ - 5}}$ N $\cdot$ m $\cdot$ s/rad, $J = 1.74 \times {10^{ - 4}}$ kg $\cdot$ m $^2$ , ${\psi _f}$ = 1.608 wb, $p =2$ . The control parameters of the two control methods are listed in Table Ⅱ. The simulation results are shown in Figs. 5-8.
表 Ⅱ Control Parameters for the Numerical Example in Case 2Table Ⅱ Control Parameters for the Numerical Example in Case 2Controllers Parameters SMC1 k = 8000, η= 950 NDOB-SMC k = 8000, η = 950, l = diag{6, 6} 
图 8 Speed curve under variable parameter.Fig. 8 Speed curve under variable parameter.Fig. 5 depicts the variable curve, reference speed changes from 20 rad/s to 40 rad/s at 5 s, and 40 rad/s to -20 rad/s at 10 s. It can be observed that the proposed method exhibits a faster speed and smooth transition than the nominal SMC method.
The unknown load torque ${T_L}$ = 3 N $\cdot$ m is supposed to add to the PMSM at 10 s, and be removed from the system at 15 s. Response curves of the rotor speed, q-axis currents are shown in Figs. 6 and 7. It can be seen that the proposed control method obtains fine tracking performance in the presence of unknown external load torque variations.
The response curves of the PMSM under the proposed method in the presence of mechanical parameter variations are shown in Fig. 8. The moment of inertia is supposed to have variations in its nominal operation values, 2 J at 10 s, and 1.5 J at 15 s, respectively. It can be observed from Fig. 8 that the proposed method is insensitive to mismatched uncertainty, and has fine robustness performance, while the nominal SMC method is sensitive to mismatched uncertainty.
5. Conclusion
In this paper, the mismatched/matched uncertainties and disturbances rejection control problem have been studied for the second-, third-, and higher-order systems. A novel NDOB-based SMC approach has been proposed. The controller not only make the states of closed-loop system obtain better tracking performance, but also enhance the disturbance attenuation and system robustness. The proposed method has exhibited nominal performance recovery and chattering reduction as compared with the nominal SMC. Simulation results reveal the effectiveness of the proposed method.
-
[1] Zhang H G, Zhang X, Luo Y H, Yang J. An overview of research on adaptive dynamic programming. Acta Automatica Sinica, 2013, 39(4):303-311 doi: 10.1016/S1874-1029(13)60031-2 [2] Liu D R, Li H L, Wang D. Data-based self-learning optimal control:research progress and prospects. Acta Automatica Sinica, 2013, 39(11):1858-1870 doi: 10.3724/SP.J.1004.2013.01858 [3] Werbos P. Beyond Regression:New Tools for Prediction and Analysis in the Behavioral Sciences[Ph., D. dissertation], Harvard University, USA, 1974. [4] Prokhorov D V, Wunsch D C. Adaptive critic designs. IEEE Transactions on Neural Networks, 1997, 8(5):997-1007 doi: 10.1109/72.623201 [5] Padhi R, Unnikrishnan N, Wang X H, Balakrishnan S N. A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems. Neural Networks, 2006, 19(10):1648-1660 doi: 10.1016/j.neunet.2006.08.010 [6] Wang Y, O'Donoghue B, Boyd S. Approximate dynamic programming via iterated Bellman inequalities. International Journal of Robust and Nonlinear Control, 2015, 25(10):1472-1496 doi: 10.1002/rnc.v25.10 [7] Bertsekas D P, Tsitsiklis J N. Neuro-dynamic programming:an overview. In:Proceedings of the 34th IEEE Conference on Decision and Control. New Orleans, LA, USA:IEEE, 1995. 560-564 [8] Zhu L M, Modares H, Peen G O, Lewis F L, Yue B Z. Adaptive suboptimal output-feedback control for linear systems using integral reinforcement learning. IEEE Transactions on Control Systems Technology, 2015, 23(1):264-273 doi: 10.1109/TCST.2014.2322778 [9] Bhasin S. Reinforcement Learning and Optimal Control Methods for Uncertain Nonlinear Systems[Ph., D. dissertation], University of Florida, USA, 2011. [10] Vrabie D, Vamvoudakis K G, Lewis F L. Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles. London:IET, 2012. [11] Zhang H, Liu D, Luo Y, Wang D. Adaptive Dynamic Programming for Control:Algorithms and Stability. London:Springer-Verlag, 2013. [12] Lewis F L, Liu D R. Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. New Jersey:IEEE Press, 2013. [13] Jiang Z P, Jiang Y. Robust adaptive dynamic programming for linear and nonlinear systems:an overview. European Journal of Control, 2013, 19(5):417-425 doi: 10.1016/j.ejcon.2013.05.017 [14] Khan S G, Herrmann G, Lewis F L, Pipe T, Melhuish C. Reinforcement learning and optimal adaptive control:an overview and implementation examples. Annual Reviews in Control, 2012, 36(1):42-59 doi: 10.1016/j.arcontrol.2012.03.004 [15] Buşoniu L, Ernst D, De Schutter B, Babuška R. Approximate reinforcement learning:an overview. In:Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning. Paris, France:IEEE, 2011. 1-8 [16] 刁兆师. 导弹精确高效末制导与控制若干关键技术研究[博士学位论文], 北京理工大学, 中国, 2015. http://cdmd.cnki.com.cn/Article/CDMD-10007-1015801403.htmDiao Zhao-Shi. Research on High-precision and High-efficiency Terminal Guidance and Control Key Technologies for Missiles[Ph., D. dissertation], Beijing Institute of Technology, China, 2015. http://cdmd.cnki.com.cn/Article/CDMD-10007-1015801403.htm [17] 李运迁. 大气层内拦截弹制导控制及一体化研究[博士学位论文], 哈尔滨工业大学, 中国, 2011. http://cdmd.cnki.com.cn/Article/CDMD-10213-1012000340.htmLi Yun-Qian. Integrated Guidance and Control for Endo-Atmospheric Interceptors[Ph., D. dissertation], Harbin Institute of Technology, China, 2011. http://cdmd.cnki.com.cn/Article/CDMD-10213-1012000340.htm [18] 孙传鹏. 基于博弈论的拦截制导问题研究[博士学位论文], 哈尔滨工业大学, 中国, 2014. http://cdmd.cnki.com.cn/Article/CDMD-10213-1014081874.htmSun Chuan-Peng. Research on Interception Guidance Based on Game Theory[Ph., D. dissertation], Harbin Institute of Technology, China, 2014. http://cdmd.cnki.com.cn/Article/CDMD-10213-1014081874.htm [19] Liu D R, Wei Q L. Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(3):621-634 doi: 10.1109/TNNLS.2013.2281663 [20] Wei Q L, Liu D R, Lin Q, Song R Z. Discrete-time optimal control via local policy iteration adaptive dynamic programming. IEEE Transactions on Cybernetics, 2016, DOI: 10.1109/TCYB.2016.2586082 [21] Zhang H G, Song R Z, Wei Q L, Zhang T Y. Optimal tracking control for a class of nonlinear discrete-time systems with time delays based on heuristic dynamic programming. IEEE Transactions on Neural Networks, 2011, 22(12):1851-1862 doi: 10.1109/TNN.2011.2172628 [22] Song R Z, Xiao W D, Zhang H G, Sun C Y. Adaptive dynamic programming for a class of complex-valued nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(9):1733-1739 doi: 10.1109/TNNLS.2014.2306201 [23] Wei Q L, Liu D R, Yang X. Infinite horizon self-learning optimal control of nonaffine discrete-time nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(4):866-879 doi: 10.1109/TNNLS.2015.2401334 [24] Wei Q L, Liu D R, Lewis F L, Liu Y, Zhang J. Mixed iterative adaptive dynamic programming for optimal battery energy control in smart residential microgrids. IEEE Transactions on Industrial Electronics, 2017, DOI:10. 1109/TIE.2017.265087 [25] Wei Q L, Liu D R, Lin Q, Song R Z. Adaptive dynamic programming for discrete-time zero-sum games. IEEE Transactions on Neural Networks and Learning Systems, 2017, DOI: 10.1109/TNNLS.2016.2638863 [26] Wei Q L, Liu D R. A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems. Science China Information Sciences, 2015, 58(12):1-15 doi: 10.1007%2F978-981-10-4080-1_4 [27] Kiumarsi B, Lewis F L, Modares H, Karimpour A, Naghibi-Sistani M B. Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics. Automatica, 2014, 50(4):1167-1175 doi: 10.1016/j.automatica.2014.02.015 [28] Vamvoudakis K G. Non-zero sum Nash Q-learning for unknown deterministic continuous-time linear systems. Automatica, 2015, 61:274-281 doi: 10.1016/j.automatica.2015.08.017 [29] Murray J J, Cox C J, Lendaris G G, Saeks R. Adaptive dynamic programming. IEEE Transactions on Systems, Man, and Cybernetics, Part C:Applications and Reviews, 2002, 32(2):140-153 doi: 10.1109/TSMCC.2002.801727 [30] Al-Tamimi A, Lewis F L, Abu-Khalaf M. Discrete-time nonlinear HJB solution using approximate dynamic programming:convergence proof. IEEE Transactions on Systems, Man, and Cybernetics, Part B:Cybernetics, 2008, 38(4):943-949 doi: 10.1109/TSMCB.2008.926614 [31] Wang F Y, Jin N, Liu D R, Wei Q L. Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with şvarepsilon-error bound. IEEE Transactions on Neural Networks, 2011, 22(1):24-36 doi: 10.1109/TNN.2010.2076370 [32] Heydari A, Balakrishnan S N. Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(1):145-157 doi: 10.1109/TNNLS.2012.2227339 [33] Wei Q L, Liu D R, Xu Y C. Neuro-optimal tracking control for a class of discrete-time nonlinear systems via generalized value iteration adaptive dynamic programming approach. Soft Computing, 2016, 20(2):697-706 doi: 10.1007/s00500-014-1533-0 [34] Wei Q L, Liu D R, Lin H Q. Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems. IEEE Transactions on Cybernetics, 2016, 46(3):840-853 doi: 10.1109/TCYB.2015.2492242 [35] Wei Q L, Lewis F L, Sun Q Y, Yan P F, Song R Z. Discrete-time deterministic Q-learning:a novel convergence analysis. IEEE Transactions on Cybernetics, 2017, 47(5):1024-0237 http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7450633& [36] Wei Q L, Song R Z, Sun Q Y. Nonlinear neuro-optimal tracking control via stable iterative Q-learning algorithm. Neurocomputing, 2015, 168:520-528 doi: 10.1016/j.neucom.2015.05.075 [37] Wei Q L, Liu D R. Stable iterative adaptive dynamic programming algorithm with approximation errors for discrete-time nonlinear systems. Neural Computing and Applications, 2014, 24(6):1355-1367 doi: 10.1007/s00521-013-1361-7 [38] Wei Q L, Liu D R. Adaptive dynamic programming for optimal tracking control of unknown nonlinear systems with application to coal gasification. IEEE Transactions on Automation Science and Engineering, 2014, 11(4):1020-1036 doi: 10.1109/TASE.2013.2284545 [39] Wei Q L, Liu D R. Numerical adaptive learning control scheme for discrete-time non-linear systems. IET Control Theory and Applications, 2013, 7(11):1472-1486 doi: 10.1049/iet-cta.2012.0486 [40] Wei Q L, Liu D R, Lin Q. Discrete-time local value iteration adaptive dynamic programming:admissibility and termination analysis. IEEE Transactions on Neural Networks and Learning Systems, 2017, DOI:10.1109/TNNLS. 2016.2593743 [41] Wei Q L, Wang F Y, Liu D R, Yang X. Finite-approximation-error-based discrete-time iterative adaptive dynamic programming. IEEE Transactions on Cybernetics, 2014, 44(12):2820-2833 doi: 10.1109/TCYB.2014.2354377 [42] Zhang H G, Luo Y H, Liu D R. Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Transactions on Neural Networks, 2009, 20(9):1490-1503 doi: 10.1109/TNN.2009.2027233 [43] Wei Q L, Zhang H G, Liu D R, Zhao Y. An optimal control scheme for a class of discrete-time nonlinear systems with time delays using adaptive dynamic programming. Acta Automatica Sinica, 2010, 36(1):121-129 http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.669.1013&rep=rep1&type=pdf [44] Song R Z, Wei Q L, Sun Q Y. Nearly finite-horizon optimal control for a class of nonaffine time-delay nonlinear systems based on adaptive dynamic programming. Neurocomputing, 2015, 156:166-175 doi: 10.1016/j.neucom.2014.12.066 [45] Wang D, Liu D R, Wei Q L. Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing, 2012, 78(1):14-22 doi: 10.1016/j.neucom.2011.03.058 [46] Abu-Khalaf M, Lewis F L. Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica, 2005, 41(5):779-791 doi: 10.1016/j.automatica.2004.11.034 [47] Tassa Y, Erez T. Least squares solutions of the HJB equation with neural network value-function approximators. IEEE Transactions on Neural Networks, 2007, 18(4):1031-1041 doi: 10.1109/TNN.2007.899249 [48] Song R Z, Lewis F L, Wei Q L, Zhang H G. Off-policy actor-critic structure for optimal control of unknown systems with disturbances. IEEE Transactions on Cybernetics, 2016, 46(5):1041-1050 doi: 10.1109/TCYB.2015.2421338 [49] Song R Z, Lewis F L, Wei Q L. Off-policy integral reinforcement learning method to solve nonlinear continuous-time multiplayer nonzero-sum games. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(3):704-713 doi: 10.1109/TNNLS.2016.2582849 [50] Vrabie D, Pastravanu O, Abu-Khalaf M, Lewis F L. Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica, 2009, 45(2):477-484 doi: 10.1016/j.automatica.2008.08.017 [51] Vrabie D, Lewis F. Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Networks, 2009, 22(3):237-246 doi: 10.1016/j.neunet.2009.03.008 [52] Vamvoudakis K G, Lewis F L. Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica, 2010, 46(5):878-888 doi: 10.1016/j.automatica.2010.02.018 [53] Vamvoudakis K G, Vrabie D, Lewis F L. Online adaptive learning of optimal control solutions using integral reinforcement learning. In:Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning. Paris, France:IEEE, 2011. 250-257 [54] Zhang H G, Cui L L, Zhang X, Luo Y H. Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method. IEEE Transactions on Neural Networks, 2011, 22(12):2226-2236 doi: 10.1109/TNN.2011.2168538 [55] Liu D R, Yang X, Wang D, Wei Q L. Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints. IEEE Transactions on Cybernetics, 2015, 45(7):1372-1385 doi: 10.1109/TCYB.2015.2417170 [56] Yang X, Liu D R, Huang Y Z. Neural-network-based online optimal control for uncertain non-linear continuous-time systems with control constraints. IET Control Theory and Applications, 2013, 7(17):2037-2047 doi: 10.1049/iet-cta.2013.0472 [57] Wang D, Liu D R, Zhang Q C, Zhao D B. Data-based adaptive critic designs for nonlinear robust optimal control with uncertain dynamics. IEEE Transactions on Systems, Man, and Cybernetics:Systems, 2016, 46(11):1544-1555 doi: 10.1109/TSMC.2015.2492941 [58] Wang D, Liu D R, Li H L. Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Transactions on Automation Science and Engineering, 2014, 11(2):627-632 doi: 10.1109/TASE.2013.2296206 [59] Wang D, Liu D R, Li H L, Ma H W. Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming. Information Sciences, 2014, 282:167-179 doi: 10.1016/j.ins.2014.05.050 [60] Vamvoudakis K, Vrabie D, Lewis F. Online policy iteration based algorithms to solve the continuous-time infinite horizon optimal control problem. In:Proceedings of the 2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning. Nashville, TN, USA:IEEE, 2009. [61] Yang X, Liu D R, Wei Q L. Online approximate optimal control for affine non-linear systems with unknown internal dynamics using adaptive dynamic programming. IET Control Theory and Applications, 2014, 8(16):1676-1688 doi: 10.1049/iet-cta.2014.0186 [62] Wang Z, Liu X P, Liu K F, Li S, Wang H Q. Backstepping-based Lyapunov function construction using approximate dynamic programming and sum of square techniques. IEEE Transactions on Cybernetics, 2016, DOI: 10.1109/TCYB.2016.2574747 [63] Vamvoudakis K G, Miranda M F, Hespanha J P. Asymptotically stable adaptive-optimal control algorithm with saturating actuators and relaxed persistence of excitation. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(11):2386-2398 doi: 10.1109/TNNLS.2015.2487972 [64] Yang X, Liu D R, Wei Q L, Wang D. Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming. Neurocomputing, 2016, 198:80-90 doi: 10.1016/j.neucom.2015.08.119 [65] Zargarzadeh H, Dierks T, Jagannathan S. State and output feedback-based adaptive optimal control of nonlinear continuous-time systems in strict feedback form. In:Proceedings of the 2012 American Control Conference. Montréal, Canada:IEEE, 2012. 6412-6417 [66] Zargarzadeh H, Dierks T, Jagannathan S. Optimal control of nonlinear continuous-time systems in strict-feedback form. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(10):2535-2549 doi: 10.1109/TNNLS.2015.2441712 [67] Modares H, Lewis F L. Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning. Automatica, 2014, 50(7):1780-1792 doi: 10.1016/j.automatica.2014.05.011 [68] Kamalapurkar R, Dinh H, Bhasin S, Dixon W E. Approximate optimal trajectory tracking for continuous-time nonlinear systems. Automatica, 2015, 51:40-48 doi: 10.1016/j.automatica.2014.10.103 [69] Zhou Q, Shi P, Tian Y, Wang M Y. Approximation-based adaptive tracking control for mimo nonlinear systems with input saturation. IEEE Transactions on Cybernetics, 2015, 45(10):2119-2128 doi: 10.1109/TCYB.2014.2365778 [70] Modares H, Lewis F L, Sistani M B N. Online solution of nonquadratic two-player zero-sum games arising in the H∞ control of constrained input systems. International Journal of Adaptive Control and Signal Processing, 2014, 28(3-5):232-254 doi: 10.1002/acs.v28.3-5 [71] Modares H, Sistani M B N, Lewis F L. A policy iteration approach to online optimal control of continuous-time constrained-input systems. ISA Transactions, 2013, 52(5):611-621 doi: 10.1016/j.isatra.2013.04.004 [72] Abu-Khalaf M, Lewis F L, Huang J. Neurodynamic programming and zero-sum games for constrained control systems. IEEE Transactions on Neural Networks, 2008, 19(7):1243-1252 doi: 10.1109/TNN.2008.2000204 [73] Yang X, Liu D R, Wang D. Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints. International Journal of Control, 2014, 87(3):553-566 doi: 10.1080/00207179.2013.848292 [74] Jiang Y, Jiang Z P. Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica, 2012, 48(10):2699-2704 doi: 10.1016/j.automatica.2012.06.096 [75] Jiang Y, Jiang Z P. Robust approximate dynamic programming and global stabilization with nonlinear dynamic uncertainties. In:Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference. Orlando, FL, USA:IEEE, 2011. 115-120 [76] Jiang Y, Jiang Z P. Robust adaptive dynamic programming and feedback stabilization of nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(5):882-893 doi: 10.1109/TNNLS.2013.2294968 [77] Jiang Y, Jiang Z P. Global adaptive dynamic programming for continuous-time nonlinear systems. IEEE Transactions on Automatic Control, 2015, 60(11):2917-2929 doi: 10.1109/TAC.2015.2414811 [78] Wang D, Liu D R, Li H L, Ma H W. Adaptive dynamic programming for infinite horizon optimal robust guaranteed cost control of a class of uncertain nonlinear systems. In:Proceedings of the 2015 American Control Conference. Chicago, IL, USA:IEEE, 2015. 2900-2905 [79] Luo Y H, Sun Q Y, Zhang H G, Cui L L. Adaptive critic design-based robust neural network control for nonlinear distributed parameter systems with unknown dynamics. Neurocomputing, 2015, 148:200-208 doi: 10.1016/j.neucom.2013.08.049 [80] Fan Q Y, Yang G H. Adaptive actor-critic design-based integral sliding-mode control for partially unknown nonlinear systems with input disturbances. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(1):165-177 doi: 10.1109/TNNLS.2015.2472974 [81] Liu D R, Huang Y Z, Wang D, Wei Q L. Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programming. International Journal of Control, 2013, 86(9):1554-1566 doi: 10.1080/00207179.2013.790562 [82] Lv Y F, Na J, Yang Q M, Wu X, Guo Y. Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics. International Journal of Control, 2016, 89(1):99-112 doi: 10.1080/00207179.2015.1060362 [83] Zhu Y H, Zhao D B, Li X J. Iterative adaptive dynamic programming for solving unknown nonlinear zero-sum game based on online data. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(3):714-724 doi: 10.1109/TNNLS.2016.2561300 [84] Song R Z, Lewis F L, Wei Q L, Zhang H G, Jiang Z P, Levine D. Multiple actor-critic structures for continuous-time optimal control using input-output data. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(4):851-865 doi: 10.1109/TNNLS.2015.2399020 [85] Vamvoudakis K, Vrabie D, Lewis F L. Adaptive optimal control algorithm for zero-sum Nash games with integral reinforcement learning. In:Proceedings of the 2012 AIAA Guidance, Navigation, and Control Conference. Minneapolis, Minnesota, USA:AIAA, 2012. [86] Wei Q L, Song R Z, Yan P F. Data-driven zero-sum neuro-optimal control for a class of continuous-time unknown nonlinear systems with disturbance using ADP. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(2):444-458 doi: 10.1109/TNNLS.2015.2464080 [87] Vamvoudakis K G, Lewis F L. Multi-player non-zero-sum games:online adaptive learning solution of coupled Hamilton-Jacobi equations. Automatica, 2011, 47(8):1556-1569 doi: 10.1016/j.automatica.2011.03.005 [88] Vamvoudakis K G, Lewis F L, Hudas G R. Multi-agent differential graphical games:online adaptive learning solution for synchronization with optimality. Automatica, 2012, 48(8):1598-1611 doi: 10.1016/j.automatica.2012.05.074 [89] Wei Q L, Liu D R, Lewis F L. Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games. Information Sciences, 2015, 317:96-113 doi: 10.1016/j.ins.2015.04.044 [90] Zhang H G, Zhang J L, Yang G H, Luo Y H. Leader-based optimal coordination control for the consensus problem of multiagent differential games via fuzzy adaptive dynamic programming. IEEE Transactions on Fuzzy Systems, 2015, 23(1):152-163 doi: 10.1109/TFUZZ.2014.2310238 [91] Nguyen T L. Adaptive dynamic programming-based design of integrated neural network structure for cooperative control of multiple MIMO nonlinear systems. Neurocomputing, 2017, 237:12-24 doi: 10.1016/j.neucom.2016.05.044 [92] Jiao Q, Modares H, Xu S Y, Lewis F L, Vamvoudakis K G. Multi-agent zero-sum differential graphical games for disturbance rejection in distributed control. Automatica, 2016, 69:24-34 doi: 10.1016/j.automatica.2016.02.002 [93] Jiao Q, Modares H, Lewis F L, Xu S Y, Xie L H. Distributed L2-gain output-feedback control of homogeneous and heterogeneous systems. Automatica, 2016, 71:361-368 doi: 10.1016/j.automatica.2016.04.025 [94] Adib Yaghmaie F, Lewis F L, Su R. Output regulation of linear heterogeneous multi-agent systems via output and state feedback. Automatica, 2016, 67:157-164 doi: 10.1016/j.automatica.2016.01.040 [95] Zhang H G, Jiang H, Luo Y H, Xiao G Y. Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method. IEEE Transactions on Industrial Electronics, 2017, 64(5):4091-4100 doi: 10.1109/TIE.2016.2542134 [96] Venayagamoorthy G K, Harley R G, Wunsch D C. Dual heuristic programming excitation neurocontrol for generators in a multimachine power system. IEEE Transactions on Industry Applications, 2003, 39(2):382-394 doi: 10.1109/TIA.2003.809438 [97] Park J W, Harley R G, Venayagamoorthy G K. Adaptive-critic-based optimal neurocontrol for synchronous generators in a power system using MLP/RBF neural networks. IEEE Transactions on Industry Applications, 2003, 39(5):1529-1540 doi: 10.1109/TIA.2003.816493 [98] Wei Q L, Liu D R, Shi G, Liu Y. Multibattery optimal coordination control for home energy management systems via distributed iterative adaptive dynamic programming. IEEE Transactions on Industrial Electronics, 2015, 62(7):4203-4214 doi: 10.1109/TIE.2014.2388198 [99] Wei Q L, Liu D R, Shi G. A novel dual iterative Q-learning method for optimal battery management in smart residential environments. IEEE Transactions on Industrial Electronics, 2015, 62(4):2509-2518 doi: 10.1109/TIE.2014.2361485 [100] Cai C, Wong C K, Heydecker B G. Adaptive traffic signal control using approximate dynamic programming. Transportation Research Part C:Emerging Technologies, 2009, 17(5):456-474 doi: 10.1016/j.trc.2009.04.005 [101] 赵冬斌, 刘德荣, 易建强.基于自适应动态规划的城市交通信号优化控制方法综述.自动化学报, 2009, 35(6):676-681 http://www.aas.net.cn/CN/abstract/abstract13331.shtmlZhao Dong-Bin, Liu De-Rong, Yi Jian-Qiang. An overview on the adaptive dynamic programming based urban city traffic signal optimal control. Acta Automatica Sinica, 2009, 35(6):676-681 http://www.aas.net.cn/CN/abstract/abstract13331.shtml [102] Wang F Y. Agent-based control for networked traffic management systems. IEEE Intelligent Systems, 2005, 20(5):92-96 doi: 10.1109/MIS.2005.80 [103] Lee J M, Lee J H. An approximate dynamic programming based approach to dual adaptive control. Journal of Process Control, 2009, 19(5):859-864 doi: 10.1016/j.jprocont.2008.11.009 [104] Wei Q L, Liu D R. Data-driven neuro-optimal temperature control of water-gas shift reaction using stable iterative adaptive dynamic programming. IEEE Transactions on Industrial Electronics, 2014, 61(11):6399-6408 doi: 10.1109/TIE.2014.2301770 [105] 林小峰, 黄元君, 宋春宁.带şvarepsilon误差限的近似最优控制.控制理论与应用, 2012, 29(1):104-108 http://www.cnki.com.cn/Article/CJFDTOTAL-KZLY201201017.htmLin Xiao-Feng, Huang Yuan-Jun, Song Chun-Ning. Approximate optimal control with şvarepsilon-error bound. Control Theory and Applications, 2012, 29(1):104-108 http://www.cnki.com.cn/Article/CJFDTOTAL-KZLY201201017.htm [106] Nodland D, Zargarzadeh H, Jagannathan S. Neural network-based optimal adaptive output feedback control of a helicopter UAV. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(7):1061-1073 doi: 10.1109/TNNLS.2013.2251747 [107] Stingu E, Lewis F L. An approximate dynamic programming based controller for an underactuated 6DOF quadrotor. In:Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning. Paris, France:IEEE, 2011. [108] Xie Q Q, Luo B, Tan F X, Guan X P. Optimal control for vertical take-off and landing aircraft non-linear system by online kernel-based dual heuristic programming learning. IET Control Theory and Applications, 2015, 9(6):981-987 doi: 10.1049/iet-cta.2013.0889 [109] Mu C X, Ni Z, Sun C Y, He H B. Air-breathing hypersonic vehicle tracking control based on adaptive dynamic programming. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(3):584-598 doi: 10.1109/TNNLS.2016.2516948 [110] Balakrishnan S N, Biega V. Adaptive-critic-based neural networks for aircraft optimal control. Journal of Guidance, Control, and Dynamics, 1996, 19(4):893-898 doi: 10.2514/3.21715 [111] Enns R, Si J. Apache helicopter stabilization using neural dynamic programming. Journal of Guidance, Control, and Dynamics, 2002, 25(1):19-25 doi: 10.2514/2.4870 [112] Enns R, Si J. Helicopter trimming and tracking control using direct neural dynamic programming. IEEE Transactions on Neural Networks, 2003, 14(4):929-939 doi: 10.1109/TNN.2003.813839 [113] Ferrari S, Stengel R F. Online adaptive critic flight control. Journal of Guidance, Control, and Dynamics, 2004, 27(5):777-786 doi: 10.2514/1.12597 [114] Valasek J, Doebbler J, Tandale M D, Meade A J. Improved adaptive-reinforcement learning control for morphing unmanned air vehicles. IEEE Transactions on Systems, Man, and Cybernetics, Part B:Cybernetics, 2008, 38(4):1014-1020 doi: 10.1109/TSMCB.2008.922018 [115] Guo C, Wu H N, Luo B, Guo L. H∞ control for air-breathing hypersonic vehicle based on online simultaneous policy update algorithm. International Journal of Intelligent Computing and Cybernetics, 2013, 6(2):126-143 doi: 10.1108/IJICC-Jun-2012-0031 [116] Luo X, Chen Y, Si J, Feng L. Longitudinal control of hypersonic vehicles based on direct heuristic dynamic programming using ANFIS. In:Proceedings of the 2014 International Joint Conference on Neural Networks. Beijing, China:IEEE, 2014. 3685-3692 [117] Furfaro R, Wibben D R, Gaudet B, Simo J. Terminal multiple surface sliding guidance for planetary landing:development, tuning and optimization via reinforcement learning. The Journal of the Astronautical Sciences, 2015, 62(1):73-99 doi: 10.1007/s40295-015-0045-1 [118] Zhou Y, Van Kampen E J, Chu Q P. Nonlinear adaptive flight control using incremental approximate dynamic programming and output feedback. Journal of Guidance, Control, and Dynamics, 2017, 40(S):493-500 https://www.researchgate.net/publication/310781464_Nonlinear_Adaptive_Flight_Control_Using_Incremental_Approximate_Dynamic_Programming_and_Output_Feedback [119] Zhou Y, Van Kampen E J, Chu Q P. An incremental approximate dynamic programming flight controller based on output feedback. In:Proceedings of the 2016 AIAA Guidance, Navigation, and Control Conference. San Diego, California, USA:AIAA, 2016. [120] Ghosh S, Ghose D, Raha S. Capturability of augmented pure proportional navigation guidance against time-varying target maneuvers. Journal of Guidance, Control, and Dynamics, 2014, 37(5):1446-1461 doi: 10.2514/1.G000561 [121] Shaferman V, Shima T. Linear quadratic guidance laws for imposing a terminal intercept angle. Journal of Guidance, Control, and Dynamics, 2008, 31(5):1400-1412 doi: 10.2514/1.32836 [122] Lee Y, Kim Y, Moon G, Jun B E. Sliding-mode-based missile-integrated attitude control schemes considering velocity change. Journal of Guidance, Control, and Dynamics, 2016, 39(3):423-436 doi: 10.2514/1.G001416 [123] Kumar S R, Rao S, Ghose D. Nonsingular terminal sliding mode guidance with impact angle constraints. Journal of Guidance, Control, and Dynamics, 2014, 37(4):1114-1130 doi: 10.2514/1.62737 [124] 周慧波. 基于有限时间和滑模理论的导引律及多导弹协同制导研究[博士学位论文], 哈尔滨工业大学, 中国, 2015. http://cdmd.cnki.com.cn/Article/CDMD-10213-1015957301.htmZhou Hui-Bo. Study on Guidance Law and Cooperative Guidance for Multi-missiles Based on Finite-time and Sliding Mode Theory[Ph., D. dissertation], Harbin Institute of Technology, China, 2015. http://cdmd.cnki.com.cn/Article/CDMD-10213-1015957301.htm [125] 张友安, 黄诘, 王丽英.约束条件下的末制导律研究进展.海军航空工程学院学报, 2013, 28(6):581-586 http://www.cnki.com.cn/Article/CJFDTOTAL-HJHK201306001.htmZhang You-An, Huang Jie, Wang Li-Ying. Research progress of terminal guidance law with constraint. Journal of Naval Aeronautical and Astronautical, 2013, 28(6):581-586 http://www.cnki.com.cn/Article/CJFDTOTAL-HJHK201306001.htm [126] Imado F, Kuroda T, Miwa S. Optimal midcourse guidance for medium-range air-to-air missiles. Journal of Guidance, Control, and Dynamics, 1990, 13(4):603-608 doi: 10.2514/3.25376 [127] Balakrishnan S N, Xin M. Robust state dependent Riccati equation based guidance laws. In:Proceedings of the 2001 American Control Conference. Arlington, VA, USA:IEEE, 2001. 3352-3357 [128] Indig N, Ben-Asher J Z, Sigal E. Near-optimal minimum-time guidance under spatial angular constraint in atmospheric flight. Journal of Guidance, Control, and Dynamics, 2016, 39(7):1563-1577 doi: 10.2514/1.G001485 [129] Taub I, Shima T. Intercept angle missile guidance under time varying acceleration bounds. Journal of Guidance, Control, and Dynamics, 2013, 36(3):686-699 doi: 10.2514/1.59139 [130] 陈克俊, 赵汉元.一种适用于攻击地面固定目标的最优再入机动制导律.宇航学报, 1994, 15(1):1-7, 94 http://www.cnki.com.cn/Article/CJFDTOTAL-YHXB401.000.htmChen Ke-Jun, Zhao Han-Yuan. An optimal reentry maneuver guidance law applying to attack the ground fixed target. Journal of Astronautics, 1994, 15(1):1-7, 94 http://www.cnki.com.cn/Article/CJFDTOTAL-YHXB401.000.htm [131] 赵汉元.飞行器再入动力学和制导.北京:国防科技大学出版社, 1997.Zhao Han-Yuan. Reentry Vehicle Dynamics and Guidance. Beijing:National University of Defense Technology, 1997. [132] Lee Y I, Ryoo C K, Kim E. Optimal guidance with constraints on impact angle and terminal acceleration. In:Proceedings of the 2003 AIAA Guidance, Navigation, and Control Conference and Exhibit. Austin, Texas, USA:AIAA, 2003. [133] Lee J I, Jeon I S, Tahk M J. Guidance law to control impact time and angle. IEEE Transactions on Aerospace and Electronic Systems, 2007, 43(1):301-310 doi: 10.1109/TAES.2007.357135 [134] 胡正东, 郭才发, 蔡洪.带落角约束的再入机动弹头的复合导引律.国防科技大学学报, 2008, 30(3):21-26 http://www.cnki.com.cn/Article/CJFDTOTAL-GFKJ200803004.htmHu Zheng-Dong, Guo Cai-Fa, Cai Hong. Integrated guidance law of reentry maneuvering warhead with terminal angular constraint. Journal of National University of Defense Technology, 2008, 30(3):21-26 http://www.cnki.com.cn/Article/CJFDTOTAL-GFKJ200803004.htm [135] Bardhan R, Ghose D. Nonlinear differential games-based impact-angle-constrained guidance law. Journal of Guidance, Control, and Dynamics, 2015, 38(3):384-402 doi: 10.2514/1.G000940 [136] 方绍琨, 李登峰.微分对策及其在军事领域的研究进展.指挥控制与仿真, 2008, 30(1):114-117 http://www.cnki.com.cn/Article/CJFDTOTAL-QBZH200801032.htmFang Shao-Kun, Li Deng-Feng. Research advances on differential games and applications to military field. Command Control and Simulation, 2008, 30(1):114-117 http://www.cnki.com.cn/Article/CJFDTOTAL-QBZH200801032.htm [137] Yang C D, Chen H Y. Nonlinear H∞ robust guidance law for homing missiles. Journal of Guidance, Control, and Dynamics, 1998, 21(6):882-890 doi: 10.2514/2.4321 [138] Dalton J, Balakrishnan S N. A neighboring optimal adaptive critic for missile guidance. Mathematical and Computer Modelling, 1996, 23(1-2):175-188 doi: 10.1016/0895-7177(95)00226-X [139] Han D C, Balakrishnan S N. Adaptive critic based neural networks for control-constrained agile missile control. In:Proceedings of the 1999 American Control Conference. San Diego, California, USA:IEEE, 1999. 2600-2604 [140] Si J, Barto A, Powell W, Wunsch D. Adaptive Critic Based Neural Network for Control-Constrained Agile Missile. New Jersey:John Wiley and Sons, Inc., 2012. [141] Han D C, Balakrishnan S. Midcourse guidance law with neural networks. In:Proceedings of the 2000 AIAA Guidance, Navigation, and Control Conference and Exhibit. Denver, CO, USA:AIAA, 2000. [142] Han D C, Balakrishnan S N. State-constrained agile missile control with adaptive-critic-based neural networks. IEEE Transactions on Control Systems Technology, 2002, 10(4):481-489 doi: 10.1109/TCST.2002.1014669 [143] Bertsekas D P, Homer M L, Logan D A, Patek S D, Sandell N R. Missile defense and interceptor allocation by neuro-dynamic programming. IEEE Transactions on Systems, Man, and Cybernetics, Part A:Systems and Humans, 2000, 30(1):42-51 doi: 10.1109/3468.823480 [144] Davis M T, Robbins M J, Lunday B J. Approximate dynamic programming for missile defense interceptor fire control. European Journal of Operational Research, 2017, 259(3):873-886 doi: 10.1016/j.ejor.2016.11.023 [145] Lin C K. Adaptive critic autopilot design of bank-to-turn missiles using fuzzy basis function networks. IEEE Transactions on Systems, Man, and Cybernetics, Part B:Cybernetics, 2005, 35(2):197-207 doi: 10.1109/TSMCB.2004.842246 [146] 卢超群, 江加和, 任章.基于增强学习的空空导弹智能精确制导律研究.战术导弹控制技术, 2006, (4):19-22, 76 http://d.wanfangdata.com.cn/Periodical/zsddkzjs200604007Lu Chao-Qun, Jiang Jia-He, Ren Zhang. Research of precision guidance law based on Q-learning for air-to-air missile. Control Technology of Tactical Missile, 2006, (4):19-22, 76 http://d.wanfangdata.com.cn/Periodical/zsddkzjs200604007 [147] McGrew J S, How J P, Bush L, Williams B, Roy N. Air combat strategy using approximate dynamic programming. In:Proceedings of the 2008 AIAA Guidance, Navigation and Control Conference and Exhibit. Honolulu, Hawaii, USA:AIAA, 2008. [148] Gaudet B, Furfaro R. Missile homing-phase guidance law design using reinforcement learning. In:Proceedings of the 2012 AIAA Guidance, Navigation, and Control Conference. Minneapolis, Minnesota, USA:AIAA, 2012. [149] Lee D, Bang H. Planar evasive aircrafts maneuvers using reinforcement learning. Intelligent Autonomous Systems 12:Advances in Intelligent Systems and Computing. Berlin Heidelberg:Springer, 2013. 533-542 [150] Sun J L, Liu C S, Ye Q. Robust differential game guidance laws design for uncertain interceptor-target engagement via adaptive dynamic programming. International Journal of Control, 2017, 64(5):4091-4100 https://www.researchgate.net/publication/283513965_Robust_Adaptive_Dynamic_Programming_of_Two-Player_Zero-Sum_Games_for_Continuous-Time_Linear_Systems [151] 姚郁, 郑天宇, 贺风华, 王龙, 汪洋, 张曦, 朱柏羊, 杨宝庆.飞行器末制导中的几个热点问题与挑战.航空学报, 2015, 36(8):2696 http://www.cnki.com.cn/Article/CJFDTOTAL-HKXB201508020.htmYao Yu, Zheng Tian-Yu, He Feng-Hua, Wang Long, Wang Yang, Zhang Xi, Zhu Bai-Yang, Yang Bao-Qing. Several hot issues and challenges in terminal guidance of flight vehicles. Acta Aeronautica et Astronautica Sinica, 2015, 36(8):2696-2716 http://www.cnki.com.cn/Article/CJFDTOTAL-HKXB201508020.htm -
下载:
计量
- 文章访问数: 2604
- HTML全文浏览量: 328
- PDF下载量: 2116
- 被引次数: 0
下载:
