Derong Liu

Adaptive Dynamic Programming: An Introduction

In this article, we introduce some recent research trends within the field of adaptive/approximate dynamic programming (ADP), including the variations on the structure of ADP schemes, the development of ADP algorithms and applications of ADP schemes. For ADP algorithms, the point of focus is that iterative algorithms of ADP can be sorted into two classes: one class is the iterative algorithm with initial stable policy; the other is the one without the requirement of initial stable policy. It is generally believed that the latter one has less computation at the cost of missing the guarantee of system stability during iteration process. In addition, many recent papers have provided convergence analysis associated with the algorithms developed. Furthermore, we point out some topics for future studies.

Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints

In this paper, the near-optimal control problem for a class of nonlinear discrete-time systems with control constraints is solved by iterative adaptive dynamic programming algorithm. First, a novel nonquadratic performance functional is introduced to overcome the control constraints, and then an iterative adaptive dynamic programming algorithm is developed to solve the optimal feedback control problem of the original constrained system with convergence analysis. In the present control scheme, there are three neural networks used as parametric structures for facilitating the implementation of the iterative algorithm. Two examples are given to demonstrate the convergence and feasibility of the proposed optimal control scheme.

Global Asymptotic Stability of Recurrent Neural Networks With Multiple Time-Varying Delays

In this paper, several sufficient conditions are established for the global asymptotic stability of recurrent neural networks with multiple time-varying delays. The Lyapunov-Krasovskii stability theory for functional differential equations and the linear matrix inequality (LMI) approach are employed in our investigation. The results are shown to be generalizations of some previously published results and are less conservative than existing results. The present results are also applied to recurrent neural networks with constant time delays.

Networked Control Systems: Theory and Applications

Networked control systems (NCS) consist of sensors, actuators and controllers the operations of which may be distributed over geographically disparate locations and co-ordinated by the exchange of information passed over a communication network. The communication network may be physically wired or not. The widespread applications of the Internet have been a major driving force for research and development of NCS. NCS have advantages in terms of cost reduction, system diagnosis and flexibility, minimizing wiring and making the addition and replacement of individual elements relatively simple; efficient data sharing makes taking globally intelligent control decisions easier with an NCS. The applications of NCS are very wide, from the large scale of factory automation and plant monitoring to the smaller but complicated networks of computers in modern cars, places and autonomous robots. Networked Control Systems presents the most recent results in stability and robustness analysis as well as new developments related to networked fuzzy and optimal control. Many of the chapters contain details of case-studies, experimental, simulation and/or other application-related work showing how the theories put forward can be implemented in real systems. The state-of-the art research reported in this volume by an international team of contributors will make Networked Control Systems an essential reference for researchers and postgraduate students in control, electrical, computer and mechanical engineering and computer science.

Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With

In this paper, we study the finite-horizon optimal control problem for discrete-time nonlinear systems using the adaptive dynamic programming (ADP) approach. The idea is to use an iterative ADP algorithm to obtain the optimal control law which makes the performance index function close to the greatest lower bound of all performance indices within an -error bound. The optimal number of control steps can also be obtained by the proposed ADP algorithms. A convergence analysis of the proposed ADP algorithms in terms of performance index function and control policy is made. In order to facilitate the implementation of the iterative ADP algorithms, neural networks are used for approximating the performance index function, computing the optimal control policy, and modeling the nonlinear system. Finally, two simulation examples are employed to illustrate the applicability of the proposed method.

\varepsilon

In this paper, a new iterative adaptive dynamic programming (ADP) method is proposed to solve a class of continuous-time nonlinear two-person zero-sum differential games. The idea is to use the ADP technique to obtain the optimal control pair iteratively which makes the performance index function reach the saddle point of the zero-sum differential games. If the saddle point does not exist, the mixed optimal control pair is obtained to make the performance index function reach the mixed optimum. Stability analysis of the nonlinear systems is presented and the convergence property of the performance index function is also proved. Two simulation examples are given to illustrate the performance of the proposed method.

-Error Bound

Stability problems of continuous-time recurrent neural networks have been extensively studied, and many papers have been published in the literature. The purpose of this paper is to provide a comprehensive review of the research on stability of continuous-time recurrent neural networks, including Hopfield neural networks, Cohen-Grossberg neural networks, and related models. Since time delay is inevitable in practice, stability results of recurrent neural networks with different classes of time delays are reviewed in detail. For the case of delay-dependent stability, the results on how to deal with the constant/variable delay in recurrent neural networks are summarized. The relationship among stability results in different forms, such as algebraic inequality forms, M-matrix forms, linear matrix inequality forms, and Lyapunov diagonal stability forms, is discussed and compared. Some necessary and sufficient stability conditions for recurrent neural networks without time delays are also discussed. Concluding remarks and future directions of stability analysis of recurrent neural networks are given.

An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games

An intelligent-optimal control scheme for unknown nonaffine nonlinear discrete-time systems with discount factor in the cost function is developed in this paper. The iterative adaptive dynamic programming algorithm is introduced to solve the optimal control problem with convergence analysis. Then, the implementation of the iterative algorithm via globalized dual heuristic programming technique is presented by using three neural networks, which will approximate at each iteration the cost function, the control law, and the unknown nonlinear system, respectively. In addition, two simulation examples are provided to verify the effectiveness of the developed optimal control approach.

A Comprehensive Review of Stability Analysis of Continuous-Time Recurrent Neural Networks

feedback control of dynamic systems 6th solution PDF feedback control of dynamic systems 6th solutions PDF feedback control of dynamic systems 5th edition pdf PDF feedback control of dynamic systems solution PDF feedback control of dynamic systems 7th edition PDF feedback control of dynamic systems 6th edition PDF feedback control of dynamic systems solutions PDF feedback control of dynamic systems solution manual PDF feedback control of dynamic systems solutions manual PDF feedback control dynamic systems 5th edition solutions PDF solutions manual feedback control of dynamic systems PDF feedback control of dynamic systems solution manual 6th PDF feedback control of dynamic systems solutions manual 5th PDF feedback control of dynamic systems franklin solutions PDF feedback control of dynamic systems solutions 6th edition PDF feedback control of dynamic systems 6th edition solutions PDF solutions feedback control dynamic systems 6th edition PDF feedback control of dynamic systems 6th solutions manual PDF feedback control of dynamic systems 6th edition solution manual PDF feedback control of dynamic systems franklin 5th edition solution PDF dynamic programming and optimal control PDF dynamic programming & optimal control vol i PDF dynamic programming and optimal control solution manual PDF learning microsoft windows server 2012 dynamic access control PDF data-variant kernel analysis adaptive and cognitive dynamic systems signal processing learning communications and control PDF

Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming

In the present paper, a call admission control scheme that can learn from the network environment and user behavior is developed for code division multiple access (CDMA) cellular networks that handle both voice and data services. The idea is built upon a novel learning control architecture with only a single module instead of two or three modules in adaptive critic designs (ACDs). The use of adaptive critic approach for call admission control in wireless cellular networks is new. The call admission controller can perform learning in real-time as well as in offline environments and the controller improves its performance as it gains more experience. Another important contribution in the present work is the choice of utility function for the present self-learning control approach which makes the present learning process much more efficient than existing learning control methods. The performance of our algorithm will be shown through computer simulation and compared with existing algorithms.