Sidney N. Givigi

Model Predictive Control for the dynamic encirclement of a target

Encirclement is a tactic that can be employed by a team of UAVs to neutralize a target by restricting its movement. The aim of the UAV team encircling the target is to move close to the target and maintain a formation around the target. In this paper, the problem of creating a dynamic circular formation around a target is considered, and a Decentralized Model Predictive Control (DMPC) policy is formulated. It is shown through simulation results that the derived MPC policy is effective for the case of a single UAV encircling a stationary target, a single UAV encircling a moving target, and a group of UAVs encircling a stationary target. The contributions of this paper are the application of MPC to the problem of encirclement, and the explicit objective of a dynamic circular formation around the target.

A Reinforcement Learning Adaptive Fuzzy Controller for Differential Games

In this paper we develop a reinforcement fuzzy learning scheme for robots playing a differential game. Differential games are games played in continuous time, with continuous states and actions. Fuzzy controllers are used to approximate the calculation of future reinforcements of the game due to actions taken at a specific time. If an immediate reinforcement reward function is defined, we may use a fuzzy system to tell what is the predicted reinforcement in a specified time ahead. This reinforcement is then used to adapt a fuzzy controller that stores the experience accumulated by the player. Simulations of a modified two car game are provided in order to show the potentiality of the technique. Experiments are performed in order to validate the method. Finally, it should be noted that although the game used as an example involves only two players, the technique may also be used in a multi-game environment.

Solving Multi-UAV Dynamic Encirclement via Model Predictive Control

In order for teams of unmanned aerial vehicles (UAVs) to collaborate and cooperate to perform challenging group tasks, intelligent and flexible control strategies are required. One of the complex behaviors required of a team of UAVs is dynamic encirclement, which is a tactic that can be employed for persistent surveillance and/or to neutralize a target by restricting its movement. This tactic requires a high level of cooperation such that the UAVs maintain a desired and proper encirclement radius and angular velocity around the target. In this paper, model predictive control (MPC) is used to model and implement controllers for the problem of dynamic encirclement. The linear and nonlinear control policies proposed in this paper are applied as a high-level controller to control multiple UAVs to encircle a desired target in simulations and real-time experiments with quadrotors. The nonlinear solution provides a theoretical analysis of the problem, while the linear control policy is used for real-time operation via a combination of MPC and feedback linearization applied to the nonlinear UAV system. The contributions of this paper lie in the implementation of MPC to solve the problem of dynamic encirclement of a team of UAVs in real time and the application of theoretical stability analysis to the problem.

Dynamic encirclement of a moving target using decentralized nonlinear Model Predictive Control

Dynamic encirclement is a tactic which can be employed by a group of UAVs to neutralize a target by restricting its movement, or provide constant surveillance of a target. The aim of the UAVs in the formation is to move into a position close to the target and establish a moving formation around the target. In this paper, the problem of creating a dynamic circular formation around a moving target is considered, and a Decentralized Model Predictive Control (DMPC) policy is formulated. Using theoretical results, a stabilizing control policy is derived, and the policy is validated through simulation results. Furthermore, we examine the effects of communications between the UAVs and the use of a model target on the performance of the UAVs. The contributions of this paper are the extension of the dynamic encirclement tactic to the case of a group of UAVs and a moving target, the consideration of a target model and communications, and the application of theoretical stability analysis to the problem.

Automatic Crack Detection and Measurement Based on Image Analysis

Crack detection and measurement in civil structures has been a constant field of research. Conventionally, a technician is responsible to detect and measure cracks in the field. In this paper, a system based on machine vision concepts has been developed with the goal to automate the crack measurement process. Using this method with only a single camera installed in a truck or even in a robot, a sequence of images is processed and the crack dimensions are estimated. The experimental results validate the application of the proposed method for real structures.

Encirclement of multiple targets using model predictive control

Two teams of Unmanned Aerial Vehicles (UAVs) are used in the encirclement of two targets at the same time. Encirclement is defined as the situation in which a target is isolated and surrounded by a group of UAVs. It is a tactic that can be employed by a team of UAVs to neutralize a target by restricting its movement due to a containment motion near the target while maintaining a formation around it. In this paper, the problem of choosing the correct target to create a dynamic circular formation is considered and a Decentralized Model Predictive Control (DMPC) policy is formulated. From simulation results the derived Model Predictive Control (MPC) policy is effective for the case of two teams of UAVs encircling two stationary targets, and two teams of UAVs encircling two moving targets. The contributions of this paper are the application of MPC to the problem of encirclement, the explicit objective of a dynamic circular formation around the target, and the ability of each team to choose its correct target.

Vision-Based Measurement for Localization of Objects in 3-D for Robotic Applications

Stereo vision is widely used in many 3-D image applications. In this paper, we present a methodology to estimate the localization of objects in 3-D scenes using collaborative robots. Using this method, the robots are able to measure the location of objects using only a single camera installed on each one of them. From one pair of images, each one acquired by a different robot, the method locates homologous points, and then rebuilds the object by using equations for coordinate transformations. The experimental results validate the application of the proposed method for the measurement of the pose of objects in autonomous robotic applications.

Using multiple Quadrotor aircraft and Linear Model Predictive Control for the encirclement of a target

A Multi-Unmanned Aerial Vehicle (UAV) team formed from two or more UAVs is used in the encirclement of a target. Encirclement is defined as the situation in which a target is isolated and surrounded by a UAV team in order to maintain awareness and containment of that target. In this paper, the problem of maintaining a circular path around a target is considered and a Linear Model Predictive Control (LMPC) strategy is implemented on a team of Qball-X4 quadrotor aircraft in order to follow the circular path. The linear plant controlled by the LMPC is a combination of process models found through system identification and a linear cartesian to polar transformation. A collision avoidance system, based on potential fields, is successfully implemented between the Qball-X4 quadrotors. The contribution of this paper lay in the application of LMPC to the problem of encirclement using a team of Qball-X4 quadrotors and the ability of these UAVs to apply a collision avoidance policy.

Design of attitude and path tracking controllers for quad-rotor robots using reinforcement learning

There is a lot of interest in using quad-rotor helicopters as Miniature Aerial Vehicles (MAVs) due to their simple mechanical construction and straightforward propulsion system. However, since these vehicles are highly unstable nonlinear dynamical systems, a suitable control system is required for their attitude stabilization and navigation. This article presents a simulation environment for the design and evaluation of attitude stabilization and path tracking controllers for quad-rotor aerial robots using Reinforcement Learning (RL). Firstly, the nonlinear mathematical model for a commercial X3D-BL quad-rotor robot from Ascending Technologies is introduced. The attitude stabilization and path tracking controllers for the quad-rotor robot are formulated. It is shown how the parameters of the controllers can be adjusted using a RL algorithm called Learning Automata. Next, the proposed simulation topology is presented and its main features are discussed. It employs 2 host computers where one host executes the control loops and the reinforcement learning algorithm using MATLAB/SIMULINK. The other host runs the quad-rotor robot model using the X-Plane Flight Simulator. The two hosts communicate using UDP (User Datagram Protocol) over a standard Ethernet wired network. Finally, some simulation cases are presented and the controllers adjusted by the RL algorithm are evaluated.

Autonomous Construction of Multiple Structures Using Learning Automata: Description and Experimental Validation

In this paper, we develop an adaptive scheme based on reinforcement learning (RL) for planning the construction tasks using a quadrotor. Moreover, an autonomous construction system to assemble user-specified 3-D structures is proposed. Nowadays, complex construction tasks using mobile robots are characterized by three fundamental problems: assembly planning, motion planning, and path tracking control. The high-level plan to perform the construction task consists of assembly mode algorithms that are derived offline in a simulation environment through learning and heuristic search. A promising approach to design and optimize the path tracking controllers for a quadrotor as well as the attitude controllers using RL is presented. This paper describes a comprehensive validation framework that enables an aerial robot to build structures in a robust and safe manner. The experimental trials for building the 3-D structures using the designed high-level plans and path tracking controllers have provided encouraging results.