Muthukumaran Chandrasekaran

Epsilon-Subjective Equivalence of Models for Interactive Dynamic Influence Diagrams

Interactive dynamic influence diagrams (I-DID) are graphical models for sequential decision making in uncertain settings shared by other agents. Algorithms for solving I-DIDs face the challenge of an exponentially growing space of candidate models ascribed to other agents, over time. Pruning behaviorally equivalent models is one way toward minimizing the model set. We seek to further reduce the complexity by additionally pruning models that are approximately subjectively equivalent. Toward this, we define subjective equivalence in terms of the distribution over the subject agents future action-observation paths, and introduce the notion of epsilon-subjective equivalence. We present a new approximation technique that reduces the candidate model space by removing models that are epsilon-subjectively equivalent with representative ones.

Can bounded and self-interested agents be teammates? Application to planning in ad hoc teams

Planning for ad hoc teamwork is challenging because it involves agents collaborating without any prior coordination or communication. The focus is on principled methods for a single agent to cooperate with others. This motivates investigating the ad hoc teamwork problem in the context of self-interested decision-making frameworks. Agents engaged in individual decision making in multiagent settings face the task of having to reason about other agents’ actions, which may in turn involve reasoning about others. An established approximation that operationalizes this approach is to bound the infinite nesting from below by introducing level 0 models. For the purposes of this study, individual, self-interested decision making in multiagent settings is modeled using interactive dynamic influence diagrams (I-DID). These are graphical models with the benefit that they naturally offer a factored representation of the problem, allowing agents to ascribe dynamic models to others and reason about them. We demonstrate that an implication of bounded, finitely-nested reasoning by a self-interested agent is that we may not obtain optimal team solutions in cooperative settings, if it is part of a team. We address this limitation by including models at level 0 whose solutions involve reinforcement learning. We show how the learning is integrated into planning in the context of I-DIDs. This facilitates optimal teammate behavior, and we demonstrate its applicability to ad hoc teamwork on several problem domains and configurations.

Approximating behavioral equivalence for scaling solutions of I-DIDs

Interactive dynamic influence diagram (I-DID) is a recognized graphical framework for sequential multiagent decision making under uncertainty. I-DIDs concisely represent the problem of how an individual agent should act in an uncertain environment shared with others of unknown types. I-DIDs face the challenge of solving a large number of models that are ascribed to other agents. A known method for solving I-DIDs is to group models of other agents that are behaviorally equivalent. Identifying model equivalence requires solving models and comparing their solutions generally represented as policy trees. Because the trees grow exponentially with the number of decision time steps, comparing entire policy trees becomes intractable, thereby limiting the scalability of previous I-DID techniques. In this article, our specific approaches focus on utilizing partial policy trees for comparison and determining the distance between updated beliefs at the leaves of the trees. We propose a principled way to determine how much of the policy trees to consider, which trades off solution quality for efficiency. We further improve on this technique by allowing the partial policy trees to have paths of differing lengths. We evaluate these approaches in multiple problem domains and demonstrate significantly improved scalability over previous approaches.

Evolving efficient sensor arrangement and obstacle avoidance control logic for a miniature robot

Evolutionary computation techniques are being frequently used in the field of robotics to develop controllers for autonomous robots. In this paper, we evaluate the use of Genetic Programming (GP) to evolve a controller that implements an Obstacle Avoidance (OA) behavior in a miniature robot. The GP system generates the OA logic equation offline on a simulated dynamic 2-D environment that transforms the sensory inputs from a simulated robot to a controller decision. The goodness of the generated logic equation is computed by using a fitness function that maximizes the exploration of the environment and minimizes the number of collisions for a fixed number of decisions allowed before the simulation is stopped. The set of motor control decisions for all possible sensor trigger sequences is applied to a real robot which is then tested on a real environment. Needless to say, the efficiency of this OA robot depends on the information it can receive from its surroundings. This information is dependant on the sensor module design. Thus, we also present a Genetic Algorithm (GA) that evolves a sensor arrangement taking into consideration economical issues as well as the usefulness of the information that can be retrieved. The evolved algorithm shows robust performance even if the robot was placed in completely different dynamically changing environments. The performance of our algorithm is compared with that of a hybrid neural network and also with an online (real time) evolution method.