Weinan Gao

Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems

This note studies the adaptive optimal output regulation problem for continuous-time linear systems, which aims to achieve asymptotic tracking and disturbance rejection by minimizing some predefined costs. Reinforcement learning and adaptive dynamic programming techniques are employed to compute an approximated optimal controller using input/partial-state data despite unknown system dynamics and unmeasurable disturbance. Rigorous stability analysis shows that the proposed controller exponentially stabilizes the closed-loop system and the output of the plant asymptotically tracks the given reference signal. Simulation results on a LCL coupled inverter-based distributed generation system demonstrate the effectiveness of the proposed approach.

Output-feedback adaptive optimal control of interconnected systems based on robust adaptive dynamic programming

This paper studies the adaptive and optimal output-feedback problem for continuous-time uncertain systems with nonlinear dynamic uncertainties. Data-driven output-feedback control policies are developed by approximate/adaptive dynamic programming (ADP) based on both policy iteration and value iteration methods. The obtained adaptive and optimal output-feedback controllers differ from the existing literature on the ADP in that they are derived from sampled-data systems theory and are guaranteed to be robust to dynamic uncertainties. A small-gain condition is given under which the overall system is globally asymptotically stable at the origin. An application to power systems is given to test the effectiveness of the proposed approaches.

Adaptive and optimal output feedback control of linear systems: An adaptive dynamic programming approach

This paper proposes a computational adaptive optimal output feedback control method for continuous-time linear systems. By periodic sampling, we use measurable input/output data to reconstruct the unmeasurable state, and then utilize adaptive dynamic programming (ADP) technique to iteratively solve the discrete-time algebraic Riccati equation. An exploration noise is introduced for online learning purpose without compromising accuracy of the proposed iterative algorithm. The stability and the optimality of the sampled-data system in close-loop with the proposed control policy are also analyzed. The feasibility of the output feedback ADP scheme is validated by simulation on a third-order linear system.

Linear optimal tracking control: An adaptive dynamic programming approach

This paper addresses the optimal output regulation problem of linear systems with unknown system dynamics. The exogenous signal is presumed to be generated by a continuous-time linear exosystem. Firstly, we formulate the linear optimal output regulation problem (LOORP). Then, we give an offline solution of LOORP to design the optimal static state-feedback servoregulator by solving an algebraic Riccati equation (ARE) and a regulator equation. Instead of solving these two equations directly, by using state, input and exogenous signals collected online, we employ an approximate/adaptive dynamic programming (ADP) technique to seek online approximations of above equations whereby we get the approximated optimal servoregulator. Rigorous stability analysis shows that the closed-loop linear system is exponentially stable. Also, the output of the plant asymptotically tracks the given reference. Simulation results demonstrate the effectiveness of the proposed approach.

Nonlinear and Adaptive Suboptimal Control of Connected Vehicles: A Global Adaptive Dynamic Programming Approach

This paper studies the cooperative adaptive cruise control (CACC) problem of connected vehicles with unknown nonlinear dynamics. Different from the present literature on CACC, data-driven feedforward and optimal feedback control policies are developed by global adaptive dynamic programming (GADP). Due to the presence of nonvanishing disturbance, a modified version of GADP is presented. Interestingly, the developed policy is guaranteed to globally stabilize the vehicular platoon system, and is robust to unmeasurable nonvanishing disturbance. Numerical simulation results are presented to validate the effectiveness of the developed approach.

Learning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear Systems

This paper proposes a novel data-driven control approach to address the problem of adaptive optimal tracking for a class of nonlinear systems taking the strict-feedback form. Adaptive dynamic programming (ADP) and nonlinear output regulation theories are integrated for the first time to compute an adaptive near-optimal tracker without any a priori knowledge of the system dynamics. Fundamentally different from adaptive optimal stabilization problems, the solution to a Hamilton-Jacobi–Bellman (HJB) equation, not necessarily a positive definite function, cannot be approximated through the existing iterative methods. This paper proposes a novel policy iteration technique for solving positive semidefinite HJB equations with rigorous convergence analysis. A two-phase data-driven learning method is developed and implemented online by ADP. The efficacy of the proposed adaptive optimal tracking control methodology is demonstrated via a Van der Pol oscillator with time-varying exogenous signals.

Adaptive optimal control of connected vehicles

In this paper, a data-driven non-model-based approach is proposed for the adaptive optimal control of connected vehicles, comprised of n human-driven vehicles only transmitting motional data and an autonomous vehicle in the tail receiving the broadcasted data from preceding vehicles by wireless vehicle-to-vehicle (V2V) communication devices. An optimal control problem is formulated to minimize the errors of distance and velocity and to optimize the fuel usage. By employing adaptive dynamic programming (ADP) technique, optimal controllers are obtained by online approximation for the connected vehicles without knowing the system dynamics. The effectiveness of the proposed approach is demonstrated via online learning control of the connected vehicles in two scenarios.

Adaptive optimal output regulation via output-feedback: An adaptive dynamic programing approach

This paper studies the problem of adaptive optimal output regulation for discrete-time linear systems. A data-driven output-feedback control approach is developed via approximate/adaptive dynamic programming (ADP). Different from the existing literature of ADP and output regulation theory, the optimal controller design proposed in this paper does not require the knowledge of the plant and exosystem dynamics. Theoretical analysis and an application on an inverted pendulum system show that the proposed methodology serves as an effective tool for solving adaptive optimal output regulation problems.

Data-Driven Nonlinear Adaptive Optimal Control of Connected Vehicles

This paper studies the cooperative adaptive cruise control (CACC) problem of connected vehicles with unknown nonlinear dynamics. Different from the existing literature on CACC, a data-driven optimal control policy is developed by global adaptive dynamic programming (GADP). Interestingly, the developed control policy achieves global stabilization of the nonlinear vehicular platoon system in the absence of the a priori knowledge of system dynamics. Numerical simulation results are presented to validate the effectiveness of the developed approach.

Cooperative optimal output regulation of multi-agent systems using adaptive dynamic programming

This paper proposes a novel solution to the adaptive optimal output regulation problem of continuous-time linear multi-agent systems. A key strategy is to resort to reinforcement learning and approximate/adaptive dynamic programming. A data-driven, non-model-based algorithm is given to design a distributed adaptive suboptimal output regulator in the presence of unknown system dynamics. The effectiveness of the proposed computational control algorithm is demonstrated via cooperative adaptive cruise control of connected and autonomous vehicles.