Travis Dierks

Output Feedback Control of a Quadrotor UAV Using Neural Networks

In this paper, a new nonlinear controller for a quadrotor unmanned aerial vehicle (UAV) is proposed using neural networks (NNs) and output feedback. The assumption on the availability of UAV dynamics is not always practical, especially in an outdoor environment. Therefore, in this work, an NN is introduced to learn the complete dynamics of the UAV online, including uncertain nonlinear terms like aerodynamic friction and blade flapping. Although a quadrotor UAV is underactuated, a novel NN virtual control input scheme is proposed which allows all six degrees of freedom (DOF) of the UAV to be controlled using only four control inputs. Furthermore, an NN observer is introduced to estimate the translational and angular velocities of the UAV, and an output feedback control law is developed in which only the position and the attitude of the UAV are considered measurable. It is shown using Lyapunov theory that the position, orientation, and velocity tracking errors, the virtual control and observer estimation errors, and the NN weight estimation errors for each NN are all semiglobally uniformly ultimately bounded (SGUUB) in the presence of bounded disturbances and NN functional reconstruction errors while simultaneously relaxing the separation principle. The effectiveness of proposed output feedback control scheme is then demonstrated in the presence of unknown nonlinear dynamics and disturbances, and simulation results are included to demonstrate the theoretical conjecture.

Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update

In this paper, the Hamilton-Jacobi-Bellman equation is solved forward-in-time for the optimal control of a class of general affine nonlinear discrete-time systems without using value and policy iterations. The proposed approach, referred to as adaptive dynamic programming, uses two neural networks (NNs), to solve the infinite horizon optimal regulation control of affine nonlinear discrete-time systems in the presence of unknown internal dynamics and a known control coefficient matrix. One NN approximates the cost function and is referred to as the critic NN, while the second NN generates the control input and is referred to as the action NN. The cost function and policy are updated once at the sampling instant and thus the proposed approach can be referred to as time-based ADP. Novel update laws for tuning the unknown weights of the NNs online are derived. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded and that the approximated control signal approaches the optimal control input with small bounded error over time. In the absence of disturbances, an optimal control is demonstrated. Simulation results are included to show the effectiveness of the approach. The end result is the systematic design of an optimal controller with guaranteed convergence that is suitable for hardware implementation.

2009 Special Issue: Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence

The optimal control of linear systems accompanied by quadratic cost functions can be achieved by solving the well-known Riccati equation. However, the optimal control of nonlinear discrete-time systems is a much more challenging task that often requires solving the nonlinear Hamilton-Jacobi-Bellman (HJB) equation. In the recent literature, discrete-time approximate dynamic programming (ADP) techniques have been widely used to determine the optimal or near optimal control policies for affine nonlinear discrete-time systems. However, an inherent assumption of ADP requires the value of the controlled system one step ahead and at least partial knowledge of the system dynamics to be known. In this work, the need of the partial knowledge of the nonlinear system dynamics is relaxed in the development of a novel approach to ADP using a two part process: online system identification and offline optimal control training. First, in the system identification process, a neural network (NN) is tuned online using novel tuning laws to learn the complete plant dynamics so that a local asymptotic stability of the identification error can be shown. Then, using only the learned NN system model, offline ADP is attempted resulting in a novel optimal control law. The proposed scheme does not require explicit knowledge of the system dynamics as only the learned NN model is needed. The proof of convergence is demonstrated. Simulation results verify theoretical conjecture.

Optimal control of affine nonlinear continuous-time systems

Solving the Hamilton-Jacobi-Isaacs (HJI) equation, commonly used in H∞ optimal control, is often referred to as a two-player differential game where one player tries to minimize the cost function while the other tries to maximize it. In this paper, the HJI equation is formulated online and forward-in-time using a novel single online approximator (SOLA)-based scheme to achieve optimal regulation and tracking control of affine nonlinear continuous-time systems. The SOLA-based adaptive approach is designed to learn the infinite horizon HJI equation, the corresponding optimal control input, and the worst case disturbance. A novel parameter tuning algorithm is derived which not only achieves the optimal cost function, control input, and the disturbance, but also ensures the system states remain bounded during the online learning. Lyapunov methods are used to show that all signals are uniformly ultimately bounded (UUB) while ensuring the approximated signals approach their optimal values with small bounded error. In the absence of OLA reconstruction errors, asymptotic convergence to the optimal signals is demonstrated, and simulation results illustrate the effectiveness of the approach.

Neural Network Output Feedback Control of Robot Formations

In this paper, a combined kinematic/torque output feedback control law is developed for leader-follower-based formation control using backstepping to accommodate the dynamics of the robots and the formation in contrast with kinematic-based formation controllers. A neural network (NN) is introduced to approximate the dynamics of the follower and its leader using online weight tuning. Furthermore, a novel NN observer is designed to estimate the linear and angular velocities of both the follower robot and its leader. It is shown, by using the Lyapunov theory, that the errors for the entire formation are uniformly ultimately bounded while relaxing the separation principle. In addition, the stability of the formation in the presence of obstacles, is examined using Lyapunov methods, and by treating other robots in the formation as obstacles, collisions within the formation are prevented. Numerical results are provided to verify the theoretical conjectures.

Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamics

In this paper, direct dynamic programming techniques are utilized to solve the Hamilton Jacobi-Bellman equation forward-in-time for the optimal tracking control of general affine nonlinear discrete-time systems using online approximators (OLAs). The proposed approach, referred as adaptive dynamic programming (ADP), is utilized to solve the infinite horizon optimal tracking control of affine nonlinear discrete-time systems in the presence of unknown internal dynamics and a known control coefficient matrix. The design is implemented using OLAs to realize the optimal feedback control signal and the associated cost function. The feedforward portion of the control input is derived and approximated using an additional OLA for steady state conditions. Novel tuning laws for the OLAs are derived, and all parameters are tuned online. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded (UUB) and that the approximated control signal approaches the optimal control input with small bounded error. In the ideal case when there are no approximation errors, the approximated control converges to the optimal value asymptotically. Simulation results are included to show the effectiveness of the approach.

Zero-Sum Two-Player Game Theoretic Formulation of Affine Nonlinear Discrete-Time Systems Using Neural Networks

In this paper, the nearly optimal solution for discrete-time (DT) affine nonlinear control systems in the presence of partially unknown internal system dynamics and disturbances is considered. The approach is based on successive approximate solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which appears in optimal control. Successive approximation approach for updating control input and disturbance for DT nonlinear affine systems are proposed. Moreover, sufficient conditions for the convergence of the approximate HJI solution to the saddle-point are derived, and an iterative approach to approximate the HJI equation using a neural network (NN) is presented. Then, the requirement of full knowledge of the internal dynamics of the nonlinear DT system is relaxed by using a second NN online approximator. The result is a closed-loop optimal NN controller via offline learning. Numerical example is provided illustrating the effectiveness of the approach.

Neural Network-Based Optimal Control of Mobile Robot Formations With Reduced Information Exchange

A novel formation control scheme for mobile robots is introduced in the context of leader-follower framework with reduced communication exchange. The dynamical controller inputs for the robots are approximated from nonlinear optimal control techniques in order to track the designed control velocities generated by the kinematic controller. The proposed nonlinear optimal control technique, referred to as adaptive dynamic programming, uses neural networks (NNs) to solve the optimal formation control problem in discrete time in the presence of unknown internal dynamics and a known control coefficient matrix. A modification to the followers kinematic controller is used to allow the desired formation to change in order to navigate around obstacles. The proposed obstacle avoidance technique modifies the desired separation and bearing of the follower to guide the follower around obstacles. Minimal wireless communication is utilized between the leader and the follower to allow the follower to approximate and compensate for the formation dynamics. All NNs are tuned online, and the stability of the entire formation is demonstrated using Lyapunov methods. Hardware results demonstrate the effectiveness of our approach.

Optimal Control of Nonlinear Continuous-Time Systems in Strict-Feedback Form

This paper proposes a novel optimal tracking control scheme for nonlinear continuous-time systems in strict-feedback form with uncertain dynamics. The optimal tracking problem is transformed into an equivalent optimal regulation problem through a feedforward adaptive control input that is generated by modifying the standard backstepping technique. Subsequently, a neural network-based optimal control scheme is introduced to estimate the cost, or value function, over an infinite horizon for the resulting nonlinear continuous-time systems in affine form when the internal dynamics are unknown. The estimated cost function is then used to obtain the optimal feedback control input; therefore, the overall optimal control input for the nonlinear continuous-time system in strict-feedback form includes the feedforward plus the optimal feedback terms. It is shown that the estimated cost function minimizes the Hamilton-Jacobi-Bellman estimation error in a forward-in-time manner without using any value or policy iterations. Finally, optimal output feedback control is introduced through the design of a suitable observer. Lyapunov theory is utilized to show the overall stability of the proposed schemes without requiring an initial admissible controller. Simulation examples are provided to validate the theoretical results.

Optimal control of affine nonlinear discrete-time systems

In this paper, direct neural dynamic programming techniques are utilized to solve the Hamilton Jacobi-Bellman equation in real time for the optimal control of general affine nonlinear discrete-time systems. In the presence of partially unknown dynamics, the optimal regulation control problem is addressed while the optimal tracking control problem is addressed in the presence of known dynamics. Each design entails two portions: an action neural network (NN) that is designed to produce a nearly optimal control signal, and a critic NN which evaluates the performance of the system. Novel weight update laws for the critic and action NNs are derived, and all parameters are tuned online. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded (UUB) and that the output of the action NN approaches the optimal control input with small bounded error. Simulation results are also presented to demonstrate the effectiveness of the approach.