Yongfeng Lv

Online H∞ control for completely unknown nonlinear systems via an identifier–critic-based ADP structure

ABSTRACT In this paper, we propose an identifier–critic-based approximate dynamic programming (ADP) structure to online solve H∞ control problem of nonlinear continuous-time systems without knowing precise system dynamics, where the actor neural network (NN) that has been widely used in the standard ADP learning structure is avoided. We first use an identifier NN to approximate the completely unknown nonlinear system dynamics and disturbances. Then, another critic NN is proposed to approximate the solution of the induced optimal equation. The H∞ control pair is obtained by using the proposed identifier–critic ADP structure. A recently developed adaptation algorithm is used to online directly estimate the unknown NN weights simultaneously, where the convergence to the optimal solution can be rigorously guaranteed, and the stability of the closed-loop system is analysed. Thus, this new ADP scheme can improve the computational efficiency of H∞ control implementation. Finally, simulation results confirm the effectiveness of the proposed methods.

Online optimal solutions for multi-player nonzero-sum game with completely unknown dynamics

Abstract In this paper, a data-driven approximate neural network (NN) learning scheme is developed to solve the multi-player nonzero-sum (NZS) game problem with completely unknown system dynamics. An augmented NN identifier based on a new parameter estimation algorithm is first established to approximate the completely unknown system dynamics. Then approximated dynamic programming (ADP) with neural networks is constructed to approximate the optimal solutions of the coupled Hamilton-Jacobi equations for each player. The approximated NN value functions are then used to synchronously calculate the optimal control policies for every player. The identifier and ADP NN weights are online updated with the system input-output data based on a novel adaptive law, which could achieve a faster convergence speed. Moreover, the convergence of all NN weights and the stability of the closed-loop system are proved based on the Lyapunov approach. Finally, a dual-driven servo motor system and a three-player nonlinear game system are simulated to verify the feasibility of the developed methods.

Optimal Controls for Dual-Driven Load System with Synchronously Approximate Dynamic Programming Method

This paper applies a synchronously approximate dynamic programming (ADP) scheme to solve the Nash controls of the dual-driven load system (DDLS) with different motor properties based on game theory. First, a neural network (NN) is applied to approximate the dual-driven servo unknown system model. Because the properties of two motors are different, they have different performance indexes. Another NN is used to approximate performance index function of each motor. In order to minimize the performance index, the Hamilton function is constructed to solve the approximate optimal controls of the load system. Based on parameter error information, an adaptive law is designed to estimate NN weights. Finally, the practical DDLS is simulated to demonstrate that the optimal control inputs can be studied by ADP algorithm.

Reconstructed Multi-innovation Gradient Algorithm for the Identification of Sandwich Systems

Inspired by multi-innovation stochastic gradient identification algorithm, a reconstructed multi-innovation stochastic gradient identification algorithm (RMISG) is presented to estimate the parameters of sandwich systems in this paper. Compared with the traditional multi-innovation stochastic gradient identification algorithm, the RMISG is constructed by using the multistep update principle which solves the multi-innovation length problem and improves the performance of the identification algorithm. To decrease the calculation burden of the RMISG, the key-term separation principle is introduced to deal with the identification model of sandwich systems. Finally, simulation example is given to validate the availability of the proposed estimator.

Parameter Estimation for Control of Hammerstein Systems with Dead-Zone Nonlinearity

This paper focuses on the parameter identification and control for Hammerstein systems with dead-zone nonlinearity by using piecewise linear parametric expression method and model predictive control approach (MPC). To linearize the dead-zone nonlinearity, the piecewise linear functions are exploited to deal with dead-zone, and then, a piecewise linear parametric expression (for short, PLPE) algorithm is applied to describe the dead-zone function. Based on the described function, the considered system is transformed to a classical regression form. The parameters of the Hammerstein systems with dead-zone can be easily estimated by using least squares method. Based on dead-zone compensation, an MPC method is introduced to achieve the signal tracking output. Numerical simulation results indicate that the control system not only achieves the tracking output of the reference signal with a small tracking error but also produces an outstanding output response.

\(H\infty \) Tracking Control for Unknown Nonlinear System Based on Augmented Matrix Algorithm

Based on the augmented matrix and approximate dynamic programming algorithms, the (Hinfty ) tracking control problem for unknown nonlinear system is addressed in this paper. An identifier NN is first used to approximate the unknown system. An augmented matrix based on the desired trajectory and system state is then constructed using the identifier, such that the tracking control problem is transformed into the regulation one. We use another NN to approximate the performance index function of the HJI equation, such that (Hinfty ) tracking control pairs are calculated without solving the HJI equation. Moreover, we use an estimation algorithm to estimate unknown parameters in neural network. Finally, a simulation is presented to demonstrate the validity of the proposed method.

Approximate Solution for Three-Player Mixed-Zero-Sum Nonlinear Game via ADP Structure

In this paper, a three-player mixed-zero-sum game situation with nonlinear dynamics is proposed, and an approximate dynamic programming (ADP) learning scheme is used to solve the proposed problem. First, the problem formulation is presented. A value function for player 1 and 2 nonzero-sum game is constructed, another value function for player 1 and 3 zero-sum game is presented for three-player nonlinear game system. Because of the difficulty to solve the nonlinear Hamilton-Jacobi (HJ) equation, the single-layer critic neural networks are used to approximate the optimal value functions. Then the approximated critic neural networks (NNs) are directly used to learn the optimal solutions for three-player mixed-zero-sum nonlinear game. A novel adaptive law with the estimation performance index is proposed to estimate the unknown coefficient vector. Finally, a simulation example is presented to illustrate the proposed methods.