Ning Jin

Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With

In this paper, we study the finite-horizon optimal control problem for discrete-time nonlinear systems using the adaptive dynamic programming (ADP) approach. The idea is to use an iterative ADP algorithm to obtain the optimal control law which makes the performance index function close to the greatest lower bound of all performance indices within an -error bound. The optimal number of control steps can also be obtained by the proposed ADP algorithms. A convergence analysis of the proposed ADP algorithms in terms of performance index function and control policy is made. In order to facilitate the implementation of the iterative ADP algorithms, neural networks are used for approximating the performance index function, computing the optimal control policy, and modeling the nonlinear system. Finally, two simulation examples are employed to illustrate the applicability of the proposed method.

\varepsilon

An intelligent-optimal control scheme for unknown nonaffine nonlinear discrete-time systems with discount factor in the cost function is developed in this paper. The iterative adaptive dynamic programming algorithm is introduced to solve the optimal control problem with convergence analysis. Then, the implementation of the iterative algorithm via globalized dual heuristic programming technique is presented by using three neural networks, which will approximate at each iteration the cost function, the control law, and the unknown nonlinear system, respectively. In addition, two simulation examples are provided to verify the effectiveness of the developed optimal control approach.

-Error Bound

In this letter, we develop the wavelet basis function neural networks (WBFNNs). It is analogous to radial basis function neural networks (RBFNNs) and to wavelet neural networks (WNNs). In WBFNNs, both the scaling function and the wavelet function of a multiresolution approximation (MRA) are adopted as the basis for approximating functions. A sequential learning algorithm for WBFNNs is presented and compared to the sequential learning algorithm of RBFNNs. Experimental results show that WBFNNs have better generalization property and require shorter training time than RBFNNs.

Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming

In this paper, we propose a novel approach to determine the positions of text areas in images with complex background using wavelet decomposition and pseudo-motion of images. In our method, a fixed image is translated and provides a sequence of moving images-the pseudomotion sequence of image. In fact, we can consider the translation of an image to be a motion of eyeshot. When an image is translated, its wavelet coefficients will oscillate. From this property, we can locate the text areas in complex-background images. Experiments were conducted to demonstrate the performance of the method In the experiments, we detect the text areas of several different types of characters in images with multi-gray level complex backgrounds.

Wavelet Basis Function Neural Networks for Sequential Learning

Abstract In this paper, we investigate the applications of neural sliding-mode control method to automotive engine control. The scheme of neural sliding-mode control is realized by two parallel neural networks. The first neural network estimates the equivalent control term and the other one generates the corrective control term. The goal of the present learning control design of automotive engines is to track the commanded torque under various operating conditions. Using the data from a test vehicle with a V8 engine, we have developed a neural network engine model and neural network controllers based on the idea of sliding-mode control to achieve optimal torque control. In simulation studies of the neural sliding-mode design method, very good transient performance and fast speed of convergence have been observed. In this process, the tedious task of parameter tuning by trial-and-error has been eliminated. Distinct features of the present technique are the controllers real-time adaptation capability based on observed real vehicle data and its rapid convergence which allow the neural network controller to be further refined and improved in real-time vehicle operation through continuous learning and adaptation.

Text area localization under complex-background using wavelet decomposition

Dynamic programming for discrete-time systems is difficult due to the ldquocurse of dimensionalityrdquo: one has to find a series of control actions that must be taken in sequence. This sequence will lead to the optimal performance cost, but the total cost of those actions will be unknown until the end of that sequence. In this paper, we present our work on dynamic programming for discrete-time system, which is referred as epsiv-adaptive dynamic programming. A single controller, epsiv-optimal controller u*(epsiv) is determined from an epsiv-optimal cost J*(epsiv) is given to approximate the optimal controller. The epsiv-optimal controller u*(epsiv) can always control the state to approach to the equilibrium state, while the performance cost is close to the biggest lower bound of all performance costs within an error according to epsiv.

Neural sliding-mode control of engine torque

In this paper, we propose a novel approach to frequency segmentation using wavelet decomposition of pseudo-motion image. In this way, a fixed image is translated such that a sequence of moving images is produced, which are called the 0 pseudo-motion images. In fact, we can consider the translation of a function to be a motion of eyeshot. When a function is translated, its wavelet coefficients will oscillate. From this property, we can detect the special areas of the image. Some experiments were conducted, which are used to find the position of license plates of vehicles in digital images. The experimental results demonstrate the performance of the method.

∈-Adaptive Dynamic Programming for discrete-time systems

In this paper, we present our work on infinite horizon adaptive dynamic programming problem, which is referred to as ∈-adaptive dynamic programming, for discrete-time systems with discount factor 0 ≪ ε ≪ 1 in the performance cost. A single controller, ∈-optimal controller μ<inf>∈</inf>^*, which is determined from an ∈-optimal cost V<inf>∈</inf>^*, is obtained to approximate the optimal controller. The ∈-optimal controller μ<inf>∈</inf>^* can always control the state to approach the equilibrium state, while the performance cost is close to the biggest lower bound of all performance costs within an error according to ∈. An algorithm for finding the ∈-optimal controller is developed and numerical experiments are given to illustrate the performance of the algorithm.

WAVELET DECOMPOSITION OF PSEUDO-MOTION IMAGE AND APPLICATION TO FREQUENCY SEGMENTATION

Sequential Monte Carlo (SMC) methods, namely, particle filters, are powerful simulation techniques for sampling sequentially from a complex probability distribution. SMC can be used to solve some problems associated with nonlinear non-Gaussian probability distribution. Sampling is a key step for these methods and has vital effects on simulation results. Various sampling strategies have been proposed to improve the simulation results of SMC methods, but degeneracy of particles sometimes is very severe so that there are only a few particles having significant weights. Diversity of particle samples is reduced significantly so that only a few particles are used to represent the corresponding probability distribution. This kind of sampling is not reasonable to approximate probability distribution. This paper addresses a new method which can avoid the phenomenon of particle degeneracy. We split particles with very big weights into two small ones and use the strategy of neural network to adjust positions of tail particles in order to increase their weights. Another advantage is that this method can efficiently make simulation results approach the actual object. Our simulation results of the typical tracking problem show that not only the phenomenon of particle degeneracy is effectively avoided but also tracking results are much better than those of the traditional particle filters. Compared with the move-resample method, our method shows better results under the same conditions.

Adaptive dynamic programming for discrete-time systems with infinite horizon and ∈-error bound in the performance cost

In this paper, a new kind of neural networks for sequential learning is proposed, which are called wavelet basis function neural networks (WBFNNs). They are analogous to radial basis function neural networks (RBFNNs) and to wavelet neural networks (WNNs). In WBFNNs, both the scaling function and the wavelet function of a multiresolution approximation (MRA) are adopted as the basis for approximating functions. A sequential learning algorithm for WBFNNs is presented and compared to the sequential learning algorithm for RBFNNs. Experimental results show that WBFNNs has better generalization property and require shorter training time than RBFNNs.