Vu Anh Huynh

icLQG: Combining local and global optimization for control in information space

When a mobile robot does not have perfect knowledge of its position, conventional controllers can experience failures such as collisions because the uncertainty of the position is not considered in choosing control actions. In this paper, we show how global planning and local feedback control can be combined to generate control laws in the space of distributions over position, that is, in information space. We give a novel algorithm for computing “information-constrained” linear quadratic Gaussian (icLQG) policies for controlling a robot with imperfect state information. The icLQG algorithm uses the belief roadmap algorithm to efficiently search for a trajectory that approximates the globally-optimal motion plan in information space, and then iteratively computes a feedback control law to locally optimize the global approximation. The icLQG algorithm is not only robust to imperfect state information but also scalable to high-dimensional systems and environments. In addition, icLQG is capable of answering multiple queries efficiently. We demonstrate performance results for controlling a vehicle on the plane and a helicopter in three dimensions.

Persistent patrol with limited-range on-board sensors

We propose and analyze the Persistent Patrol Problem (PPP). An unmanned aerial vehicle (UAV) moving with constant speed and unbounded acceleration patrols a bounded region of the plane where localized incidents occur according to a renewal process with known time intensity and spatial distribution. The UAV can detect incidents using on-board sensors with a limited visibility radius. We want to minimize the expected waiting time between the occurrence of an incident, and the time that it is detected. First, we provide a lower bound on the achievable expected detection time of any patrol policy in the limit as the visibility radius goes to zero. Second, we present the Biased Tile Sweep policy whose upper bound shows i) the lower bounds tightness, ii) the policys asymptotic optimality, and iii) that the desired spatial distribution of the searching vehicles position is proportional to the square root of the underlying spatial distribution of incidents it must find. Third, we present two online policies: i) a policy whose performance is provably within a constant factor of the optimal called TSP Sampling, ii) and the TSP Sampling with Receding Horizon heuristically yielding better performance than the former in practice. Fourth, we present a decision-theoretic approach to the PPP that attempts to solve for optimal policies offline. In addition, we use numerical experiments to compare performance of the four approaches and suggest suitable operational scenarios for each one.

A martingale approach and time-consistent sampling-based algorithms for risk management in stochastic optimal control

In this paper, we consider a class of stochastic optimal control problems with risk constraints that are expressed as bounded probabilities of failure for particular initial states. We present here a martingale approach that diffuses a risk constraint into a martingale to construct time-consistent control policies. The martingale stands for the level of risk tolerance that is contingent on available information over time. By augmenting the system dynamics with the controlled martingale, the original risk-constrained problem is transformed into a stochastic target problem. We extend the incremental Markov Decision Process (iMDP) algorithm to approximate arbitrarily well an optimal feedback policy of the original problem by sampling in the augmented state space and computing proper boundary conditions for the reformulated problem. We show that the algorithm is both probabilistically sound and asymptotically optimal. The performance of the proposed algorithm is demonstrated on motion planning and control problems subject to bounded probability of collision in uncertain cluttered environments.

An incremental sampling-based algorithm for stochastic optimal control

In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Using the Markov chain approximation method and recent advances in sampling-based algorithms for deterministic path planning, we propose a novel algorithm called the incremental Markov Decision Process to incrementally compute control policies that approximate arbitrarily well an optimal policy in terms of the expected cost. The main idea behind the algorithm is to generate a sequence of finite discretizations of the original problem through random sampling of the state space. At each iteration, the discretized problem is a Markov Decision Process that serves as an incrementally refined model of the original problem. We show that with probability one, (i) the sequence of the optimal value functions for each of the discretized problems converges uniformly to the optimal value function of the original stochastic optimal control problem, and (ii) the original optimal value function can be computed efficiently in an incremental manner using asynchronous value iterations. Thus, the proposed algorithm provides an anytime approach to the computation of optimal control policies of the continuous problem. The effectiveness of the proposed approach is demonstrated on motion planning and control problems in cluttered environments in the presence of process noise.

Probabilistically-sound and asymptotically-optimal algorithm for stochastic control with trajectory constraints

In this paper, we consider a class of stochastic optimal control problems with trajectory constraints. As a special case, we can constrain the probability that a system enters undesirable regions to remain below a certain threshold. We extend the incremental Markov Decision Process (iMDP) algorithm, which is a new computationally-efficient and asymptotically-optimal sampling-based tool for stochastic optimal control, to approximate arbitrarily well an optimal feedback policy of the constrained problem. We show that with probability one, in the presence of trajectory constraints, the sequence of policies returned from the algorithm is both probabilistically sound and asymptotically optimal. We demonstrate the proposed algorithm on motion planning and control problems subject to bounded collision probability in uncertain cluttered environments.

Persistent Patrol in Stochastic Environments with Limited Sensors

The need for persistent patrol and detection arises in many contexts such as crime prevention, search and rescue, post-conflict stability operations, and peace keeping. In these situations, military or police units are not only effective deterrents to would-be adversaries but also a speedy task force to intercept any trespassers or provide swift security and assistance. With recent advances in technology, unmanned aerial vehicles (UAVs) are well-suited for these tasks because they possess a large bird’s-eye view and are unhindered by ground obstacles. The path planning algorithms used in such missions play a critical role in minimizing the required resources, and maximizing the quality of provided service. Moreover, in many of these mission scenarios, if a patrol pattern is regular or predictable, adversaries can plan a counter strategy to the patrolling effort. In other words, unpredictability is one of the key features of planning in these circumstances. In this work, we propose and analyze the Persistent Patrol and Detection Problem (PPDP), a generic mathematical model for UAVs with limited sensors performing such a mission in a stochastic environment. Incidents occur dynamically and stochastically according to a general renewal process with known time intensity and spatial distribution in a planar region. The UAV is modeled as a point mass traveling at a constant speed with unbounded acceleration. The UAV detects incidents within the footprint of its onboard sensors, i.e., within its visibility range. We want to minimize the expected waiting time between the occurrence of an incident, and its detection epoch. Furthermore, we prefer stochastic trajectories so that would-be adversaries cannot predict the UAV’s motion based on past observations to plan for evasion. Related research focuses on search and rescue missions in which the number of searched objects is known at the beginning of the missions.1,2, 3 In other words, these works present static problems in terms of the number of objects to be found. In contrast, incidents of interest in the PPDP arrive continuously with unknown arrival times, and therefore the search effort must be persistent and preventive. This inherent difference between the PPDP and previous works has made the PPDP a dynamic problem in term of the number of incidents. ∗V. A. Huynh is with the Laboratory of Information and Decision Systems, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139. vuhuynh@mit.edu †J. J. Enright is with Kiva Systems, 225 Wildwood Ave, Woburn, MA 01801. jenright@kivasystems.com ‡E. Frazzoli is with the Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, Massachusetts, 02139. frazzoli@mit.edu

Dynamic traffic congestion pricing mechanism with User-Centric considerations

We consider the problem of designing real-time traffic routing systems in urban areas. Optimal dynamic routing for multiple passengers is known to be computationally hard due to its combinatorial nature. To overcome this difficulty, we propose a novel mechanism called User-Centric Dynamic Pricing (UCDP) based on recent advances in algorithmic mechanism design. The mechanism allows for congestion-free traffic in general road networks with heterogeneous users, while satisfying each users travel preference. The mechanism first informs whether a passenger should use public transportation or the road network. In the latter case, a passenger reports his maximum accepted travel time with a lower bound announced publicly by the road authority. The mechanism then assigns the passenger a path that matches with his preference given the current traffic condition in the network. The proposed mechanism introduces a fairness constrained shortest path (FCSP) problem with a special structure, thus enabling polynomial time computation of path allocation that maximizes the sequential social surplus and guarantees fairness among passengers. The tolls of paths are then computed according to marginal cost payments. We show that reporting true preference is a weakly dominant strategy. The superior performance of the proposed mechanism is demonstrated on several simulated routing experiments in comparison to user equilibrium and system optimum.