Jerome Le Ny

Multi-UAV dynamic routing with partial observations using restless bandit allocation indices

Motivated by the type of missions currently performed by unmanned aerial vehicles, we investigate a discrete dynamic vehicle routing problem with a potentially large number of targets and vehicles. Each target is modeled as an independent two-state Markov chain, whose state is not observed if the target is not visited by some vehicle. The goal for the vehicles is to collect rewards obtained when they visit the targets in a particular state. This problem can be seen as a type of restless bandits problem with partial information. We compute an upper bound on the achievable performance and obtain in closed form an index policy proposed by Whittle. Simulation results provide evidence for the outstanding performance of this index heuristic and for the quality of the upper bound.

On trajectory optimization for active sensing in Gaussian process models

We consider the problem of optimizing the trajectory of a mobile sensor with perfect localization whose task is to estimate a stochastic, perhaps multidimensional field modeling the environment. When the estimator is the Kalman filter, and for certain classes of objective functions capturing the informativeness of the sensor paths, the sensor trajectory optimization problem is a deterministic optimal control problem. This estimation problem arises in many applications besides the field estimation problem, such as active mapping with mobile robots. The main difficulties in solving this problem are computational, since the Gaussian process of interest is usually high dimensional. We review some recent work on this problem and propose a suboptimal non-greedy trajectory optimization scheme with a manageable computational cost, at least in static field models based on sparse graphical models.

Nonmyopic View Planning for Active Object Classification and Pose Estimation

One of the central problems in computer vision is the detection of semantically important objects and the estimation of their pose. Most of the work in object detection has been based on single image processing, and its performance is limited by occlusions and ambiguity in appearance and geometry. This paper proposes an active approach to object detection in which the point of view of a mobile depth camera is controlled. When an initial static detection phase identifies an object of interest, several hypotheses are made about its class and orientation. Then, a sequence of views, which balances the amount of energy used to move the sensor with the chance of identifying the correct hypothesis, is planned. We formulate an active hypothesis testing problem, which includes sensor mobility, and solve it using a point-based approximate partially observable Markov decision process algorithm. The validity of our approach is verified through simulation and realworld experiments with the PR2 robot. The results suggest that the approach outperforms the widely used greedy viewpoint selection and provides a significant improvement over static object detection.

Stochastic source seeking in complex environments

The objective of source seeking problems is to determine the minimum of an unknown signal field, which represents a physical quantity of interest, such as heat, chemical concentration, or sound. This paper proposes a strategy for source seeking in a noisy signal field using a mobile robot and based on a stochastic gradient descent algorithm. Our scheme does not require a prior map of the environment or a model of the signal field and is simple enough to be implemented on platforms with limited computational power. We discuss the asymptotic convergence guarantees of algorithm and give specific guidelines for its application to mobile robots in unknown indoor environments with obstacles. Both simulations and real-world experiments were carried out to evaluate the performance of our approach. The results suggest that the algorithm has good finite time performance in complex environments.

Distributed Algorithms for Stochastic Source Seeking With Mobile Robot Networks

Autonomous robot networks are an effective tool for monitoring large-scale environmental fields. This paper proposes distributed control strategies for localizing the source of a noisy signal, which could represent a physical quantity of interest such as magnetic force, heat, radio signal, or chemical concentration. We develop algorithms specific to two scenarios: one in which the sensors have a precise model of the signal formation process and one in which a signal model is not available. In the model-free scenario, a team of sensors is used to follow a stochastic gradient of the signal field. Our approach is distributed, robust to deformations in the group geometry, does not necessitate global localization, and is guaranteed to lead the sensors to a neighborhood of a local maximum of the field. In the model-based scenario, the sensors follow the stochastic gradient of the mutual information between their expected measurements and the location of the source in a distributed manner. The performance is demonstrated in simulation using a robot sensor network to localize the source of a wireless radio signal.

Decentralized active information acquisition: Theory and application to multi-robot SLAM

This paper addresses the problem of controlling mobile sensing systems to improve the accuracy and efficiency of gathering information autonomously. It applies to scenarios such as environmental monitoring, search and rescue, surveillance and reconnaissance, and simultaneous localization and mapping (SLAM). A multi-sensor active information acquisition problem, capturing the common characteristics of these scenarios, is formulated. The goal is to design sensor control policies which minimize the entropy of the estimation task, conditioned on the future measurements. First, we provide a non-greedy centralized solution, which is computationally fast, since it exploits linearized sensing models, and memory efficient, since it exploits sparsity in the environment model. Next, we decentralize the control task to obtain linear complexity in the number of sensors and provide suboptimality guarantees. Finally, our algorithms are applied to the multi-robot active SLAM problem to enable a decentralized nonmyopic solution that exploits sparsity in the planning process.

The Wireless Control Network: Synthesis and robustness

We consider the problem of stabilizing a plant with a network of resource constrained wireless nodes. Traditional networked control schemes are designed with one of the nodes in the network acting as a dedicated controller, while the other nodes simply route information to and from the controller and the plant. We introduce the concept of a Wireless Control Network (WCN) where the entire network itself acts as the controller. Specifically, at each time-step, each node updates its internal state to be a linear combination of the states of the nodes in its neighborhood. We show that this causes the entire network to behave as a linear dynamical system, with sparsity constraints imposed by the network topology. We then provide a numerical design procedure to determine the appropriate linear combinations to be applied by each node so that the transmissions of the nodes closest to the actuators will stabilize the plant. We also show how our design procedure can be modified to maintain mean square stability under packet drops in the network.

Closing the loop: a simple distributed method for control over wireless networks

We present a distributed scheme used for control over a network of wireless nodes. As opposed to traditional networked control schemes where the nodes simply route information to and from a dedicated controller (perhaps performing some encoding along the way), our approach, Wireless Control Network (WCN), treats the network itself as the controller. In other words, the computation of the control law is done in a fully distributed way inside the network. We extend the basic WCN strategy, where at each time-step, each node updates its internal state to be a linear combination of the states of the nodes in its neighborhood. This causes the entire network to behave as a linear dynamical system, with sparsity constraints imposed by the network topology. We demonstrate that with observer style updates, the WCNs robustness to link failures is substantially improved. Fur-thermore, we show how to design a WCN that can maintain stability even in cases of node failures. We also address the problem of WCN synthesis with guaranteed optimal performance of the plant, with respect to standard cost functions. We extend the synthesis procedure to deal with continuous-time plants and demonstrate how the WCN can be used on a practical, industrial application, using a process-in-the-loop setup with real hardware.

Multi-Agent Task Assignment in the Bandit Framework

We consider a task assignment problem for a fleet of UAVs in a surveillance/search mission. We formulate the problem as a restless bandits problem with switching costs and discounted rewards: there are TV sites to inspect, each one of them evolving as a Markov chain, with different transition probabilities if the site is inspected or not. The sites evolve independently of each other, there are transition costs c ij for moving between sites i and j isin {1,..., N}, rewards when visiting the sites, and we maximize a mixed objective function of these costs and rewards. This problem is known to be PSPACE-hard. We present a systematic method, inspired from the work of Bertsimas and Nino-Mora (2000) on restless bandits, for deriving a linear programming relaxation for such locally decomposable MDPs. The relaxation is computable in polynomial-time offline, provides a bound on the achievable performance, as well as an approximation of the cost-to-go which can be used online in conjunction with standard suboptimal stochastic control methods. In particular, the one-step lookahead policy based on this approximate cost-to-go reduces to computing the optimal value of a linear assignment problem of size N. We present numerical experiments, for which we assess the quality of the heuristics using the performance bound

Hypothesis testing framework for active object detection

One of the central problems in computer vision is the detection of semantically important objects and the estimation of their pose. Most of the work in object detection has been based on single image processing and its performance is limited by occlusions and ambiguity in appearance and geometry. This paper proposes an active approach to object detection by controlling the point of view of a mobile depth camera. When an initial static detection phase identifies an object of interest, several hypotheses are made about its class and orientation. The sensor then plans a sequence of viewpoints, which balances the amount of energy used to move with the chance of identifying the correct hypothesis. We formulate an active M-ary hypothesis testing problem, which includes sensor mobility, and solve it using a point-based approximate POMDP algorithm. The validity of our approach is verified through simulation and experiments with real scenes captured by a kinect sensor. The results suggest a significant improvement over static object detection.