Girish Chowdhary

Concurrent learning for convergence in adaptive control without persistency of excitation

We show that for an adaptive controller that uses recorded and instantaneous data concurrently for adaptation, a verifiable condition on linear independence of the recorded data is sufficient to guarantee exponential tracking error and parameter error convergence. This condition is found to be less restrictive and easier to monitor than a condition on persistently exciting exogenous input signal required by traditional adaptive laws that use only instantaneous data for adaptation.

Theory and Flight-Test Validation of a Concurrent-Learning Adaptive Controller

Theory and results of flight-test validation are presented for a novel adaptive law that concurrently uses current as well as recorded data for improving the performance of model reference adaptive control architectures. This novel adaptive law is termed concurrent learuing. This adaptive law restricts the weight updates based on stored data to the null-space of the weight updates based on current data for ensuring that learning on stored data does not affect responsiveness to current data. This adaptive law alleviates the rank-1 condition on weight updates in adaptive control, thereby improving weight convergence properties and improving tracking performance. Lyapunov-like analysis is used to show that the new adaptive law guarantees uniform ultimate boundedness of all system signals in the framework of model reference adaptive control. Flight-test results confirm expected improvements in performance.

GPS‐denied Indoor and Outdoor Monocular Vision Aided Navigation and Control of Unmanned Aircraft

GPS-denied closed-loop autonomous control of unstable Unmanned Aerial Vehicles (UAVs) such as rotorcraft using information from a monocular camera has been an open problem. Most proposed Vision aided Inertial Navigation Systems (V-INSs) have been too computationally intensive or do not have sufficient integrity for closed-loop flight. We provide an affirmative answer to the question of whether V-INSs can be used to sustain prolonged real-world GPS-denied flight by presenting a V-INS that is validated through autonomous flight-tests over prolonged closed-loop dynamic operation in both indoor and outdoor GPS-denied environments with two rotorcraft unmanned aircraft systems (UASs). The architecture efficiently combines visual feature information from a monocular camera with measurements from inertial sensors. Inertial measurements are used to predict frame-to-frame transition of online selected feature locations, and the difference between predicted and observed feature locations is used to bind in real-time the inertial measurement unit drift, estimate its bias, and account for initial misalignment errors. A novel algorithm to manage a library of features online is presented that can add or remove features based on a measure of relative confidence in each feature location. The resulting V-INS is sufficiently efficient and reliable to enable real-time implementation on resource-constrained aerial vehicles. The presented algorithms are validated on multiple platforms in real-world conditions: through a 16-min flight test, including an autonomous landing, of a 66 kg rotorcraft UAV operating in an unconctrolled outdoor environment without using GPS and through a Micro-UAV operating in a cluttered, unmapped, and gusty indoor environment.

A singular value maximizing data recording algorithm for concurrent learning

We present a singular value maximizing algorithm for recording data to be used by concurrent learning adaptive controllers. These controllers use recorded and current data concurrently and can have exponential stability guarantees, with the rate of convergence directly proportional to the minimum singular value of the matrix containing recorded data. The presented algorithm selects data for recording to improve the minimum singular value, and hence results in improved tracking performance, this is established through comparison with previously studied data recording methods that record points that are sufficiently different.

A Tutorial on Linear Function Approximators for Dynamic Programming and Reinforcement Learning

A Markov Decision Process (MDP) is a natural framework for formulating sequential decision-making problems under uncertainty. In recent years, researchers have greatly advanced algorithms for learning and acting in MDPs. This article reviews such algorithms, beginning with well-known dynamic programming methods for solving MDPs such as policy iteration and value iteration, then describes approximate dynamic programming methods such as trajectory based value iteration, and finally moves to reinforcement learning methods such as Q-Learning, SARSA, and least-squares policy iteration. We describe algorithms in a unified framework, giving pseudocode together with memory and iteration complexity analysis for each. Empirical evaluations of these techniques with four representations across four domains, provide insight into how these algorithms perform with various feature sets in terms of running time and performance.

Decentralized control of partially observable Markov decision processes

The focus of this paper is on solving multi-robot planning problems in continuous spaces with partial observability. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems, but representing and solving Dec-POMDPs is often intractable for large problems. To allow for a high-level representation that is natural for multi-robot problems and scalable to large discrete and continuous problems, this paper extends the Dec-POMDP model to the Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP). The Dec-POSMDP formulation allows asynchronous decision-making by the robots, which is crucial in multi-robot domains. We also present an algorithm for solving this Dec-POSMDP which is much more scalable than previous methods since it can incorporate closed-loop belief space macro-actions in planning. These macro-actions are automatically constructed to produce robust solutions. The proposed methods performance is evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent multi-robot problems and provide high-quality solutions for large-scale problems.

Autonomous Flight in GPS-Denied Environments Using Monocular Vision and Inertial Sensors

Published by the American Institute of Aeronautics and Astronautics, Inc., with permission.

Reproducing Kernel Hilbert Space Approach for the Online Update of Radial Bases in Neuro-Adaptive Control

Classical gradient based adaptive laws in model reference adaptive control for uncertain nonlinear dynamical systems with a Radial Basis Function (RBF) neural networks adaptive element do not guarantee that the network weights stay bounded in a compact neighborhood of the ideal weights without Persistently Exciting (PE) system signals or a-priori known bounds on ideal weights. Recent work has shown, however, that an adaptive controller using specifically recorded data concurrently with instantaneous data can guarantee such boundedness without requiring PE signals. However, in this work, the assumption has been that the RBF network centers are fixed, which requires some domain knowledge of the uncertainty. We employ a Reproducing Kernel Hilbert Space theory motivated online algorithm for updating the RBF centers to remove this assumption. Along with showing the boundedness of the resulting neuro-adaptive controller, a connection is also made between PE signals and kernel methods. Simulation results show improved performance.

Exponential parameter and tracking error convergence guarantees for adaptive controllers without persistency of excitation

In model reference adaptive control (MRAC) the modelling uncertainty is often assumed to be parameterised with time-invariant unknown ideal parameters. The convergence of parameters of the adaptive element to these ideal parameters is beneficial, as it guarantees exponential stability, and makes an online learned model of the system available. Most MRAC methods, however, require persistent excitation of the states to guarantee that the adaptive parameters converge to the ideal values. Enforcing PE may be resource intensive and often infeasible in practice. This paper presents theoretical analysis and illustrative examples of an adaptive control method that leverages the increasing ability to record and process data online by using specifically selected and online recorded data concurrently with instantaneous data for adaptation. It is shown that when the system uncertainty can be modelled as a combination of known nonlinear bases, simultaneous exponential tracking and parameter error convergence can be guaranteed if the system states are exciting over finite intervals such that rich data can be recorded online; PE is not required. Furthermore, the rate of convergence is directly proportional to the minimum singular value of the matrix containing online recorded data. Consequently, an online algorithm to record and forget data is presented and its effects on the resulting switched closed-loop dynamics are analysed. It is also shown that when radial basis function neural networks (NNs) are used as adaptive elements, the method guarantees exponential convergence of the NN parameters to a compact neighbourhood of their ideal values without requiring PE. Flight test results on a fixed-wing unmanned aerial vehicle demonstrate the effectiveness of the method.

Guidance and Control of Airplanes Under Actuator Failures and Severe Structural Damage

This paper presents control algorithms for guidance and control of airplanes under actuator failures and severe structural damage. The presented control and guidance algorithms are validated through experimentation on the Georgia Institute of Technology Twinstar twin engine, fixed-wing, unmanned aerial system. Damage scenarios executed include sudden loss of all aerodynamic actuators resulting in propulsion-only flight, 25% of the left wing missing, sudden loss of 50% of the right wing and aileron in-flight, and injected actuator time delay. A state-dependent guidance logic is described that ensures the aircraft tracks feasible commands in the presence of faults. The commands are used by an outer-loop linear controller to generate feasible attitude commands. The inner-loop attitude control can be achieved by using either a linear attitude controller or a neural network-based model reference adaptive controller. The results indicate the possibility of using control methods to ensure safe autonomous flight ...