Akshat Kumar

Scalable multiagent planning using probabilistic inference

Multiagent planning has seen much progress with the development of formal models such as Dec-POMDPs. However, the complexity of these models--NEXP-Complete even for two agents-- has limited scalability. We identify certain mild conditions that are sufficient to make multiagent planning amenable to a scalable approximation w.r.t. the number of agents. This is achieved by constructing a graphical model in which likelihood maximization is equivalent to plan optimization. Using the Expectation-Maximization framework for likelihood maximization, we show that the necessary inference can be decomposed into processes that often involve a small subset of agents, thereby facilitating scalability. We derive a global update rule that combines these local inferences to monotonically increase the overall solution quality. Experiments on a large multiagent planning benchmark confirm the benefits of the new approach in terms of runtime and scalability.

Tournament versus fitness uniform selection

In evolutionary algorithms a critical parameter that must be tuned is that of selection pressure. If it is set too low then the rate of convergence towards the optimum is likely to be slow. Alternatively if the selection pressure is set too high the system is likely to become stuck in a local optimum due to a loss of diversity in the population. The recent fitness uniform selection scheme (FUSS) is a conceptually simple but somewhat radical approach to addressing this problem - rather than biasing the selection towards higher fitness, FUSS biases selection towards sparsely populated fitness levels. In this paper, we compare the relative performance of FUSS with the well known tournament selection scheme on a range of problems.

Probabilistic inference techniques for scalable multiagent decision making

Decentralized POMDPs provide an expressive framework for multiagent sequential decision making. However, the complexity of these models--NEXP-Complete even for two agents--has limited their scalability. We present a promising new class of approximation algorithms by developing novel connections between multiagent planning and machine learning. We show how the multiagent planning problem can be reformulated as inference in a mixture of dynamic Bayesian networks (DBNs). This planning-as-inference approach paves the way for the application of efficient inference techniques in DBNs to multiagent decision making. To further improve scalability, we identify certain conditions that are sufficient to extend the approach to multiagent systems with dozens of agents. Specifically, we show that the necessary inference within the expectation-maximization framework can be decomposed into processes that often involve a small subset of agents, thereby facilitating scalability. We further show that a number of existing multiagent planning models satisfy these conditions. Experiments on large planning benchmarks confirm the benefits of our approach in terms of runtime and scalability with respect to existing techniques.

Optimization Approaches for Solving Chance Constrained Stochastic Orienteering Problems

Orienteering problems OPs are typically used to model routing and trip planning problems. OP is a variant of the well known traveling salesman problem where the goal is to compute the highest reward path that includes a subset of nodes and has an overall travel time less than the specified deadline. Stochastic orienteering problems SOPs extend OPs to account for uncertain travel times and are significantly harder to solve than deterministic OPs. In this paper, we contribute a scalable mixed integer LP formulation for solving risk aware SOPs, which is a principled approximation of the underlying stochastic optimization problem. Empirically, our approach provides significantly better solution quality than the previous best approach over a range of synthetic benchmarks and on a real-world theme park trip planning problem.

Influence diagrams with memory states: representation and algorithms

Influence diagrams (IDs) offer a powerful framework for decision making under uncertainty, but their applicability has been hindered by the exponential growth of runtime and memory usage--largely due to the no-forgetting assumption. We present a novel way to maintain a limited amount of memory to inform each decision and still obtain near-optimal policies. The approach is based on augmenting the graphical model with memory states that represent key aspects of previous observations--a method that has proved useful in POMDP solvers. We also derive an efficient EM-based message-passing algorithm to compute the policy. Experimental results show that this approach produces highquality approximate polices and offers better scalability than existing methods.

Risk-Sensitive Stochastic Orienteering Problems for Trip Optimization in Urban Environments

Orienteering Problems (OPs) are used to model many routing and trip planning problems. OPs are a variant of the well-known traveling salesman problem where the goal is to compute the highest reward path that includes a subset of vertices and has an overall travel time less than a specified deadline. However, the applicability of OPs is limited due to the assumption of deterministic and static travel times. To that end, Campbell et al. extended OPs to Stochastic OPs (SOPs) to represent uncertain travel times (Campbell et al. 2011). In this article, we make the following key contributions: (1) We extend SOPs to Dynamic SOPs (DSOPs), which allow for time-dependent travel times; (2) we introduce a new objective criterion for SOPs and DSOPs to represent a percentile measure of risk; (3) we provide non-linear optimization formulations along with their linear equivalents for solving the risk-sensitive SOPs and DSOPs; (4) we provide a local search mechanism for solving the risk-sensitive SOPs and DSOPs; and (5) we provide results on existing benchmark problems and a real-world theme park trip planning problem.

Observer: Assisted Adaptive Tracking Control of an Underactuated Autonomous Underwater Vehicle

This paper proposes an observer-assisted (indirect) adaptive trajectory tracking control scheme for an under-actuated autonomous underwater robotic vehicle. The proposed control algorithm was based on feedback linearization control, using the estimated vehicle (hydrodynamic) parameters and external disturbance parameters (e.g., underwater current, buoyancy variations, etc.). These parameters were estimated online by employing a well-known non-linear observer extended Kalman filter. Using these estimated parameters and estimated disturbance vector, a feedback linearization control system was constructed for an under-actuated underwater vehicle. The effectiveness of the proposed system demonstrated and discusses the robustness using simulation results. The simulation results were compared with those of a control scheme that employed true parameters and without compensation, as well and results show the proposed scheme works well.

Simultaneous Optimization and Sampling of Agent Trajectories over a Network

We study the problem of optimizing the trajectories of agents moving over a network given their preferences over which nodes to visit subject to operational constraints on the network. In our running example, a theme park manager optimizes which attractions to include in a day-pass to maximize the pass’s appeal to visitors while keeping operational costs within budget. The first challenge in this combinatorial optimization problem is that it involves quantities (expected visit frequencies of each attraction) that cannot be expressed analytically, for which we use the Sample Average Approximation. The second challenge is that while sampling is typically done prior to optimization, the dependence of our sampling distribution on decision variables couples optimization and sampling. Our main contribution is a mathematical program that simultaneously optimizes decision variables and implements inverse transform sampling from the distribution they induce. The third challenge is the limited scalability of the monolithic mathematical program. We present a dual decomposition approach that exploits independence among samples and demonstrate better scalability compared to the monolithic formulation in different settings.

Learning and Controlling Network Diffusion in Dependent Cascade Models

Diffusion processes have increasingly been used to represent flow of ideas, traffic and diseases in networks. Learning and controlling the diffusion dynamics through management actions has been studied extensively in the context of independent cascade models, where diffusion on outgoing edges from a node are independent of each other. Our work, in contrast, addresses (a) learning diffusion dynamics parameters and (b) taking management actions to alter the diffusion dynamics to achieve a desired outcome in dependent cascade models. A key characteristic of such dependent cascade models is the flow preservation at all nodes in the network. For example, traffic and people flow is preserved at each network node. As a case study, we address learning visitor mobility pattern at a theme park based on observed historical wait times at individual attractions, and use the learned model to plan management actions that reduce wait time at attractions. We test on real-world data from a theme park in Singapore and show that our learning approach can achieve an accuracy close to 80% for popular attractions, and the decision support algorithm can provide about 10-20% reduction in wait time.

Decomposition techniques for urban consolidation problems

Less-than-truckload deliveries is known to be a source of inefficiency in last-mile logistics leading to high transport costs, environmental pollution, traffic jam, particularly in urban settings. An Urban Consolidation Center (UCC) provides a platform to consolidate freights from various sources before delivering into the city. The operations of UCCs consist of 2 interrelated phases, consolidating freights and scheduling trucks into the city center. This problem is computationally challenging because of large urban freight volumes, which prohibits optimal solutions of conventional integer programming models to be found efficiently. In this paper, we propose two novel decomposition schemes: a vertical decomposition based on dynamic programming can achieve optimal consolidation for the single-period problem, and the horizontal decomposition based on a Lagrangian Relaxation can achieve good approximate solution for the multi-period problem. The combination of these two decompositions yield an efficient approach for dealing with large-scale problems.