Qinmin Yang

Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems using Online Approximators

In this paper, reinforcement learning state- and output-feedback-based adaptive critic controller designs are proposed by using the online approximators (OLAs) for a general multi-input and multioutput affine unknown nonlinear discretetime systems in the presence of bounded disturbances. The proposed controller design has two entities, an action network that is designed to produce optimal signal and a critic network that evaluates the performance of the action network. The critic estimates the cost-to-go function which is tuned online using recursive equations derived from heuristic dynamic programming. Here, neural networks (NNs) are used both for the action and critic whereas any OLAs, such as radial basis functions, splines, fuzzy logic, etc., can be utilized. For the output-feedback counterpart, an additional NN is designated as the observer to estimate the unavailable system states, and thus, separation principle is not required. The NN weight tuning laws for the controller schemes are also derived while ensuring uniform ultimate boundedness of the closed-loop system using Lyapunov theory. Finally, the effectiveness of the two controllers is tested in simulation on a pendulum balancing system and a two-link robotic arm system.

Adaptive neural control of high-order uncertain nonaffine systems: A transformation to affine systems approach ☆

Abstract This brief investigates the adaptive neural network (NN) control of a class of high-order nonaffine nonlinear systems with completely unknown dynamics. Since the control terms appear within the unknown nonlinearity, traditional control schemes and stability analysis are usually rendered extremely complicated. Our main contribution includes a novel system transformation that converts the nonaffine system into an affine system through a combination of a low-pass filter and state transformation. As a result, the state-feedback control of the nonaffine system can be viewed as the output-feedback control of an affine system in normal form. The transformed system becomes linear with respect to the new input while the traditional backstepping approach is not needed thus allowing the synthesis to be extremely simplified. It is theoretically proven that all the signals in the closed-loop system are uniformly ultimately bounded (UUB). Simulation results are provided to demonstrate the performance of the developed controller.

Adaptive Power Capture Control of Variable-Speed Wind Energy Conversion Systems With Guaranteed Transient and Steady-State Performance

This paper deals with the power capture control of variable-speed wind energy conversion systems. The control objective is to optimize the capture of wind energy by tracking the desired power output. Arbitrary steady-state performance is achieved in the sense that the tracking error is guaranteed to converge to any predefined small set. In addition, to maximize the wind energy capture, transient performance is enhanced such that the convergence rate can be larger than an arbitrary value, which further limits the maximum overshoot. First, an adaptive controller is designed for the case where known aerodynamic torque is assumed. Then, by utilizing an online approximator to estimate the uncertain aerodynamics, the need for the exact knowledge of the aerodynamic torque is waived to imitate the practical experience. With the aid of a novel output error transformation technique, both of the proposed controllers are capable of shaping the system performance arbitrarily on transient and steady-state stages. Meanwhile, it is also proved that all the signals in the closed-loop system are bounded via Lyapunov synthesis. Finally, the feasibility of the proposed controllers is demonstrated on an 1.5-MW three-blade wind turbine using the FAST (Fatigue, Aerodynamics, Structures, and Turbulence) code developed by the National Renewable Energy Laboratory.

Adaptive Neural Control of a Class of Output-Constrained Nonaffine Systems

In this paper, we present a novel tracking controller for a class of uncertain nonaffine systems with time-varying asymmetric output constraints. Firstly, the original nonaffine constrained (in the sense of the output signal) control system is transformed into a output-feedback control problem of an unconstrained affine system in normal form. As a result, stabilization of the transformed system is sufficient to ensure constraint satisfaction. It is subsequently shown that the output tracking is achieved without violation of the predefined asymmetric time-varying output constraints. Therefore, we are capable of quantifying the system performance bounds as functions of time on both transient and steady-state stages. Furthermore, the transformed system is linear with respect to a new input signal and the traditional backstepping scheme is avoided, which makes the synthesis extremely simplified. All the signals in the closed-loop system are proved to be semi-globally, uniformly, and ultimately bounded via Lyapunov synthesis. Finally, the simulation results are presented to illustrate the performance of the proposed controller.

Guaranteed Performance Control of DFIG Variable-Speed Wind Turbines

This brief presents a novel power control strategy for variable-speed wind turbines equipped with doubly fed induction generators (DFIGs). The control objective is to optimize the extracted power from wind while regulating the stator reactive power to meet grid requirements. First, in order to optimize the extracted power, an adaptive control technique is designed to drive the electromagnetic torque to follow its reference generated by the maximum power point tracking algorithm. Subsequently, aiming at satisfying reactive power requirements on the grid side, an adaptive reactive power controller is proposed to manipulate the stator reactive power to follow a given desired reactive power determined by the grid. Compared with most existing studies, we are capable of quantifying and further guaranteeing the system performance on both transient and steady-state stages. All signals in the closed-loop system are proved to be bounded via standard Lyapunov synthesis. Finally, the effectiveness of the proposed scheme is validated on a 1.5-MW DFIG-based wind turbine using the FAST (Fatigue, Aerodynamics, Structures, and Turbulence) simulator developed by the National Renewable Energy Laboratory.

Adaptive actuator fault tolerant control for uncertain nonlinear systems with multiple actuators

In this paper, a novel adaptive fault tolerant controller design is proposed for a class of nonlinear unknown systems with multiple actuators. The controller consists of an adaptive learning-based control law, a Nussbaum gain, and a switching function scheme. The adaptive control law is implemented by a two-layer neural network to accommodate the unknown system dynamics. Without the requirement of additional fault detection mechanism, the switching function is designed to automatically locate and turn off the unknown faulty actuators by observing a control performance index. The asymptotic stability of the system output in the presence of actuator failures is rigidly proved through standard Lyapunov approach, while the other signals of the closed-loop system are guaranteed to be bounded. The theoretical result is substantiated by simulation on a two-tank system.

Adaptive Neural Control of Nonlinear MIMO Systems With Time-Varying Output Constraints

In this paper, adaptive neural control is investigated for a class of unknown multiple-input multiple-output nonlinear systems with time-varying asymmetric output constraints. To ensure constraint satisfaction, we employ a system transformation technique to transform the original constrained (in the sense of the output restrictions) system into an equivalent unconstrained one, whose stability is sufficient to solve the output constraint problem. It is shown that output tracking is achieved without violation of the output constraint. More specifically, we can shape the system performance arbitrarily on transient and steady-state stages with the output evolving in predefined time-varying boundaries all the time. A single neural network, whose weights are tuned online, is used in our design to approximate the unknown functions in the system dynamics, while the singularity problem of the control coefficient matrix is avoided without assumption on the prior knowledge of control inputs bound. All the signals in the closed-loop system are proved to be semiglobally uniformly ultimately bounded via Lyapunov synthesis. Finally, the merits of the proposed controller are verified in the simulation environment.

Robust Integral of Neural Network and Error Sign Control of MIMO Nonlinear Systems

This paper presents a novel state-feedback control scheme for the tracking control of a class of multi-input multioutput continuous-time nonlinear systems with unknown dynamics and bounded disturbances. First, the control law consisting of the robust integral of a neural network (NN) output plus sign of the tracking error feedback multiplied with an adaptive gain is introduced. The NN in the control law learns the system dynamics in an online manner, while the NN residual reconstruction errors and the bounded disturbances are overcome by the error sign signal. Since both of the NN output and the error sign signal are included in the integral, the continuity of the control input is ensured. The controller structure and the NN weight update law are novel in contrast with the previous effort, and the semiglobal asymptotic tracking performance is still guaranteed by using the Lyapunov analysis. In addition, the NN weights and all other signals are proved to be bounded simultaneously. The proposed approach also relaxes the need for the upper bounds of certain terms, which are usually required in the previous designs. Finally, the theoretical results are substantiated with simulations.

NN-based asymptotic tracking control for a class of strict-feedback uncertain nonlinear systems with output constraints

An asymptotic tracking control law is proposed for a class of strict-feedback nonlinear systems with unknown nonlinearities. A Barrier Lyapunov function in combination with backstepping is proposed to guarantee that the output trajectory is contained in a predefined set. A single neural network (NN), whose weights are tuned online, is utilized in our design to approximate the unknown functions in the system dynamics, while the singularity problem of the control gain function is avoided. Meanwhile, in order to compensate for the NN residual reconstruction error and system uncertainties, a robust term is introduced and asymptotic tracking stability is achieved. All the signals in the closed-loop system are proved to be bounded via Lyapunov synthesis and the output converges to the desired trajectory asymptotically without transgressing a given bound. Finally, the merits of the proposed controller are verified in the simulation environment.

Universal Neural Network Control of MIMO Uncertain Nonlinear Systems

In this brief, a continuous tracking control law is proposed for a class of high-order multi-input-multi-output uncertain nonlinear dynamic systems with external disturbance and unknown varying control direction matrix. The proposed controller consists of high-gain feedback, Nussbaum gain matrix selector, online approximator (OLA) model and a robust term. The OLA model is represented by a two-layer neural network. The continuousness of the control signal is guaranteed to relax the requirement for the actuator bandwidth and avoid the incurred chattering effect. Asymptotic tracking performance is achieved theoretically by standard Lyapunov analysis. The control feasibility is also verified in simulation environment.