Cairo Lúcio Nascimento

Design of attitude and path tracking controllers for quad-rotor robots using reinforcement learning

There is a lot of interest in using quad-rotor helicopters as Miniature Aerial Vehicles (MAVs) due to their simple mechanical construction and straightforward propulsion system. However, since these vehicles are highly unstable nonlinear dynamical systems, a suitable control system is required for their attitude stabilization and navigation. This article presents a simulation environment for the design and evaluation of attitude stabilization and path tracking controllers for quad-rotor aerial robots using Reinforcement Learning (RL). Firstly, the nonlinear mathematical model for a commercial X3D-BL quad-rotor robot from Ascending Technologies is introduced. The attitude stabilization and path tracking controllers for the quad-rotor robot are formulated. It is shown how the parameters of the controllers can be adjusted using a RL algorithm called Learning Automata. Next, the proposed simulation topology is presented and its main features are discussed. It employs 2 host computers where one host executes the control loops and the reinforcement learning algorithm using MATLAB/SIMULINK. The other host runs the quad-rotor robot model using the X-Plane Flight Simulator. The two hosts communicate using UDP (User Datagram Protocol) over a standard Ethernet wired network. Finally, some simulation cases are presented and the controllers adjusted by the RL algorithm are evaluated.

Prognostics of aircraft bleed valves using a SVM classification algorithm

Non planned maintenance in aircraft systems is usually associated with high costs. It is believed that at least part of these costs can be avoided if adequate system prognosis programs are used. The aim of these programs is to evaluate the current state of an aircraft component based on the available system data (e.g., flight and maintenance data) and to estimate the future performance and the remaining useful life of this component. Several algorithms can be used for this purpose. This paper proposes a method of performing prognostics on aircraft component based a binary classification Support Vector Machine (SVM) Classification algorithm. The algorithm is used to classify each flight according to a fault pattern which the algorithm was trained to recognize. Flight data and maintenance logs were used to generate the training and testing datasets. In this case study, for each flight, a number of characteristics were extracted from parameters of the aircraft Air Management System. From the classification results a degradation index is created to serve as an aid to better plan the aircraft maintenance. An advantage of the proposed method is that it does not require a deep knowledge on the system. Furthermore, it does not need a large amount of input-output samples to be trained. The method should be easily extended to other aircraft systems as long as enough flight data and maintenance logs are available.

Autonomous Construction of Multiple Structures Using Learning Automata: Description and Experimental Validation

In this paper, we develop an adaptive scheme based on reinforcement learning (RL) for planning the construction tasks using a quadrotor. Moreover, an autonomous construction system to assemble user-specified 3-D structures is proposed. Nowadays, complex construction tasks using mobile robots are characterized by three fundamental problems: assembly planning, motion planning, and path tracking control. The high-level plan to perform the construction task consists of assembly mode algorithms that are derived offline in a simulation environment through learning and heuristic search. A promising approach to design and optimize the path tracking controllers for a quadrotor as well as the attitude controllers using RL is presented. This paper describes a comprehensive validation framework that enables an aerial robot to build structures in a robust and safe manner. The experimental trials for building the 3-D structures using the designed high-level plans and path tracking controllers have provided encouraging results.

Health monitoring and remaining useful life estimation of lithium-ion aeronautical batteries

Batteries are essential components of any aircraft electrical system. They are used to start the aircraft propulsion engines and to provide power during electrical emergencies. As is the case with most aircraft components, batteries exhibit aging and health degradation during operation. Therefore, the correctly estimation of the battery state-of-health (SoH) and of the remaining useful life (RUL) is important to aircraft operators. Failure to do so can result in underutilization of the equipment (if it is removed before the end of its life cycle) or unpredicted failure events during operation (when the battery SoH is overestimated). The consequences can range from increased operation costs to reduced flight safety. This article first presents the life cycle of lithium-ion aeronautical batteries. Then a method is proposed to generate discharge, capacity and health monitoring models during the battery life cycle. It is shown how these models are used to estimate the battery SoH and RUL. The method is validated using data from the NASA Ames Prognostics Data Repository. The models are implemented using MATLAB/Simulink and used to simulate a typical battery in different operational conditions.

Accumulative Learning using Multiple ANN for Flexible Link Control

This paper presents a scheme of multiple neural networks (MNNs) with a new strategy of combination. This combination can obtain an accumulative learning: the knowledge is increased by gradually adding more neural networks to the system. This scheme is applied to flexible link control via feedback-error-learning (FEL) strategy, here called multi-network-feedback-error-learning. Three different neural control approaches are used to control a flexible link, and it is shown that a better inverse dynamic model of the plant is obtained in this case.

Autonomous construction of structures in a dynamic environment using Reinforcement Learning

This paper presents an adaptive approach based on the Reinforcement Learning (RL) method to manipulate and transport parts and also assemble 3-D structures in a moderately constrained and dynamic environment using a quad-rotor. Nowadays, complex construction tasks using mobile robots are characterized by two fundamental problems such as task planning and motion planning. However, to obtain the task and path planning that define a specific sequence of operations for construction of a given structure is generally very complex. In this context, we propose and investigate a system in which an aerial robot learns the assembly and construction tasks of multiple 3-D structures. This process involves the learning of the sequence of maneuvers of a vehicle, the assembly sequence of the parts and also the correct types of structural elements for each assembly point of the structure. A heuristic search algorithm is used in the learning process to find the optimal path for the quad-rotor so that its navigation through the dynamic environment is performed. The experimental results show that a 3-D structure can be built using the task planning approach derived from a learning algorithm combined with a heuristic search method.

Combining PHM information and system architecture to support aircraft maintenance planning

Aircraft are highly valuable assets and large budgets are spent in preventive and predictive maintenance programs. The application of PHM (Prognostics and Health Management) technologies can be a powerful decision support tool to help maintenance planners. The RUL (Remaining Useful Life) estimations obtained from a PHM system can be used in order to plan in advance for the repair of components before a failure occurs. However, when system architecture is not taken into account, the use of PHM information may lead the operator to rush to replace a component that would not affect immediately the operation of the system under consideration. This paper presents a methodology for decision support in maintenance planning with application in aeronautical systems. The methodology combines system architecture information and RUL estimations for all components comprised in the system under study, allowing the estimation of a RUL value for the whole system. This system level RUL (S-RUL) can be used as support information for identifying the best moment to repair a component. Also, when several components present high degradation levels, the proposed methodology can be used to define a set of components that, when repaired, will bring the whole system to a safe degradation level with lowest cost. A case study is used to illustrate the application of the methodology in a simplified aircraft electrical system.

Knowledge extraction using a genetic fuzzy rule-based system with increased interpretability

In this paper a fuzzy rule-based system is trained to perform a classification task using a genetic algorithm and a fitness function that simultaneously considers the accuracy of the model and its interpretability. Initially a decision tree is created using any tree induction algorithm such as CART, ID3 or C4.5. This tree is then used to generate a fuzzy rule-based system. The parameters of the membership functions are adjusted by the genetic algorithm. As a case study, the proposed method is applied to an appendicitis dataset with 106 instances (input-output pairs), 7 normalized real-valued inputs and 1 binary output.

Health monitoring using support vector classification on an Auxiliary Power Unit

Health monitoring has the challenge of monitoring the life of equipment and systems. To determine the health of systems and equipments, it is necessary to have an indication of the current state of the equipment and a health reference indicator. Often, such health reference indicator does not exist or it is not available to estimate the equipments remaining useful life (RUL). This article presents a methodology that defines the equipments health reference indicator using a data-driven classification technique and produces a degradation model to be used by the aircraft health monitoring systems. A One-Class Classifier based on Support Vector Machines estimates the region of nominal operation mode and detects abnormal behaviors that can characterize incipient failures. A dataset was collected during the operation of an aircraft Auxiliary Power Unit (APU) and it was used for testing the proposed methodology.

Modeling and simulation of nickel-cadmium batteries during discharge

A battery is an essential component of any aircraft. It has several functions such as to start the aircraft propulsion engines and to provide power during electrical emergencies. The aircraft designer must know the electrical behaviour of the batteries that have been preselected for possible use in the aircraft at several operating conditions. Typically, the manufacturers of batteries for aircrafts provide proprietary software that can be used to generate the battery voltage curves over time for constant and user-specified room temperature and discharge current, assuming that the battery is initially fully charged. This paper shows how these battery voltage curves can be used to generate and validate a battery discharge model. By using such a model, the aircraft designers can simulate the battery behaviour at several operating conditions such as a variable room temperature, a variable discharge current and a partially charged battery. As a case study, it is shown how to generate and validate the battery discharge model for a specified nickel-cadmium aeronautical battery. The model is then used to simulate the battery during an electrical emergency situation considering three types of loads: linear (constant resistance) load, constant power load and constant current load. In all three cases, the simulation results were as expected.