Shankarachary Ragi

UAV Path Planning in a Dynamic Environment via Partially Observable Markov Decision Process

A path-planning algorithm to guide unmanned aerial vehicles (UAVs) for tracking multiple ground targets based on the theory of partially observable Markov decision processes (POMDPs) is presented. A variety of features of interest are shown to be easy to incorporate into the framework by plugging in the appropriate models, which demonstrates the power and flexibility of the POMDP framework. Specifically, it is shown how to incorporate the following features by appropriately formulating the POMDP action space, transition law, and objective function: 1) control UAVs with both forward acceleration and bank angle subject to constraints; 2) account for the effect of wind disturbance on UAVs; 3) avoid collisions between UAVs and obstacles and among UAVs; 4) track targets while evading threats; 5) track evasive targets; and 6) mitigate track swaps.

Dynamic UAV path planning for multitarget tracking

We design a path-planning algorithm to guide unmanned aerial vehicles (UAVs) for tracking multiple ground targets based on the theory of partially observable Markov decision processes (POMDPs). We demonstrate the power and flexibility of the POMDP framework by showing that a variety of features of interest are easy to incorporate into the framework by plugging in the appropriate models. Specifically, in this paper we show how to incorporate the following features by appropriately formulating the POMDP action space, transition law, and objective function: 1) control UAVs with both forward acceleration and bank angle subject to constraints; 2) account for the effect of wind disturbance on UAVs; and 3) mitigate track swaps.

Directional Sensor Control: Heuristic Approaches

We study the problem of controlling multiple 2-D directional sensors while maximizing an objective function based on the information gain corresponding to multiple target locations. We assume a joint prior Gaussian distribution for the target locations. A sensor generates a (noisy) measurement of a target only if the target lies within the field-of-view of the sensor, where the statistical properties of the measurement error depend on the location of the target with respect to the sensor and direction of the sensor. The measurements from the sensors are fused to form global estimates of target locations. This problem is combinatorial in nature-the computation time increases exponentially with the number of sensors. We develop heuristic methods to solve the problem approximately, and provide analytical results on performance guarantees. We then improve the performance of our heuristic approaches by applying an approximate dynamic programming approach called rollout. In addition, we address a variant of the above problem, where the goal is to map the sensors to the targets while maximizing the abovementioned objective function. This mapping problem also turns out to be combinatorial in nature, so we extend one of the above heuristics to solve this mapping problem approximately. We compare the performance of these heuristic approaches analytically and empirically.

Decentralized Guidance Control of UAVs with Explicit Optimization of Communication

We design a decentralized guidance control method for autonomous unmanned aerial vehicles (UAVs) tracking multiple targets. We formulate this guidance control problem as a decentralized partially observable Markov decision process (Dec-POMDP). As in the case of partially observable Markov decision process (POMDP), it is intractable to solve a Dec-POMDP exactly. So, we extend a POMDP approximation method called nominal belief-state optimization (NBO) to solve our control problem posed as a Dec-POMDP. We incorporate the cost of communication into the objective function of the Dec-POMDP, i.e., we explicitly optimize the communication among the UAVs at the network level along with the kinematic control commands for the UAVs. We measure the performance of our method with the following metrics: 1) average target-location error, and 2) average communication cost. The goal to maximize the performance with respect to each of the above metrics conflict with each other, and we show through empirical study how to trade off between these performance metrics using a scalar parameter. The NBO method induces coordination among the UAVs even though the system is decentralized. To demonstrate the effectiveness of this coordination, we compare our Dec-POMDP approach with a greedy approach (a noncooperative approach), where the UAVs do not communicate with each other and each UAV optimizes only its local kinematic controls. Furthermore, we compare the performance of our approach of optimizing the communication between the UAVs with a fixed communication scheme—where only the UAV kinematic controls are optimized with an underlying fixed (non-optimized) communication scheme.

Decentralized control of unmanned aerial vehicles for multitarget tracking

We design a guidance control method for a fleet of autonomous unmanned aerial vehicles (UAVs) tracking multiple targets in a decentralized setting. Our method is based on the theory of decentralized partially observable Markov decision process (Dec-POMDP). Like partially observable Markov decision processes (POMDPs), it is intractable to solve Dec-POMDPs exactly. So, we extend a POMDP approximation method called nominal belief-state optimization (NBO) to solve Dec-POMDP. We incorporate the cost of communication into the objective function of Dec-POMDP, i.e., we explicitly optimize the communication among the UAVs along with the kinematic-control commands for the UAVs. We measure the performance of our guidance method with the following metrics: 1) average target-location error, and 2) average communication cost. The goal to maximize the performance with respect to each of the above metrics conflict with each other, and we show through empirical study how to trade off between these performance metrics using a scalar parameter.

Directional sensor control for maximizing information gain

We develop tractable solutions to the problem of controlling the directions of 2-D directional sensors for maximizing information gain corresponding to multiple targets in 2-D. The target locations are known with some uncertainty given by a joint prior distribution (Gaussian). A sensor generates a (noisy) measurement of a target only if the target lies within the field-of-view of the sensor, and the measurements from all the sensors are fused to form global estimates of target locations. This problem is hard to solve exactly - the computation time increases exponentially with the number of sensors. We develop heuristic methods to solve the problem approximately and provide lower and upper bounds on the optimal information gain. We improve the solutions from these heuristic approaches by formulating the problem as a dynamic programming problem and solving it using a rollout approach.

Guidance of Autonomous Amphibious Vehicles for Flood Rescue Support

We develop a path-planning algorithm to guide autonomous amphibious vehicles (AAVs) for flood rescue support missions. Specifically, we develop an algorithm to control multiple AAVs to reach/rescue multiple victims (also called targets) in a flood scenario in 2D, where the flood water flows across the scene and the targets move (drifted by the flood water) along the flood stream. A target is said to be rescued if an AAV lies within a circular region of a certain radius around the target. The goal is to control the AAVs such that each target gets rescued while optimizing a certain performance objective. The algorithm design is based on the theory of partially observable Markov decision process (POMDP). In practice, POMDP problems are hard to solve exactly, so we use an approximation method called nominal belief-state optimization (NBO). We compare the performance of the NBO approach with a greedy approach.

Feasibility Study of POMDP in Autonomous Amphibious Vehicle Guidance

Abstract We develop a path-planning method, based on the theory of partially observable Markov decision process (POMDP), to guide an autonomous amphibious vehicle (AAV) to reach/rescue a victim in a flood situation. In practice, POMDP problems are hard to solve exactly, so we use an approximation method called nominal belief-state optimization (NBO). We compare the performance of the NBO approach with a greedy approach.