Steven Okamoto

Allocating tasks in extreme teams

Extreme teams, large-scale agent teams operating in dynamic environments, are on the horizon. Such environments are problematic for current task allocation algorithms due to the lack of locality in agent interactions. We propose a novel distributed task allocation algorithm for extreme teams, called LA-DCOP, that incorporates three key ideas. First, LA-DCOPs task allocation is based on a dynamically computed minimum capability threshold which uses approximate knowledge of overall task load. Second, LA-DCOP uses tokens to represent tasks and further minimize communication. Third, it creates potential tokens to deal with inter-task constraints of simultaneous execution. We show that LA-DCOP convincingly outperforms competing distributed task allocation algorithms while using orders of magnitude fewer messages, allowing a dramatic scale-up in extreme teams, upto a fully distributed, proxybased team of 200 agents. Varying threshold are seen as a key to outperforming competing distributed algorithms in the domain of simulated disaster rescue.

An integrated token-based algorithm for scalable coordination

Efficient coordination among large numbers of heterogeneous agents promises to revolutionize the way in which some complex tasks, such as responding to urban disasters can be performed. However, state of the art coordination algorithms are not capable of achieving efficient and effective coordination when a team is very large. Building on recent successful token-based algorithms for task allocation and information sharing, we have developed an integrated and efficient approach to effective coordination of large scale teams. We use tokens to encapsulate anything that needs to be shared by the team, including information, tasks and resources. The tokens are efficiently routed through the team via the use of local decision theoretic models. Each token is used to improve the routing of other tokens leading to a dramatic performance improvement when the algorithms work together. We present results from an implementation of this approach which demonstrates its ability to coordinate large teams.

Explorative anytime local search for distributed constraint optimization

Distributed Constraint Optimization Problems (DCOPs) are an elegant model for representing and solving many realistic combinatorial problems that are distributed by nature. DCOPs are NP-hard and therefore many recent studies consider incomplete algorithms for solving them. Distributed local search algorithms, in which agents in the system hold value assignments to their variables and iteratively make decisions on whether to replace them, can be used for solving DCOPs. However, because of the differences between the global evaluation of a systems state and the private evaluation of states by agents, agents are unaware of the global best state that is explored by the algorithm. Previous attempts to use local search algorithms for solving DCOPs reported the state held by the system at the termination of the algorithm, which was not necessarily the (global) best state explored. A general framework that enhances distributed local search algorithms for DCOPs with the anytime property is proposed. The proposed framework makes use of a BFS-tree in order to accumulate the costs of the systems state during the algorithms iterative performance and to propagate the detection of a new best state when it is found. The proposed framework does not require additional network load. Agents are required to hold a small (linear) additional space (beside the requirements of the algorithm in use). We further propose a set of increased exploration heuristics that exploit the proposed anytime framework. These exploration methods implement different approaches towards exploration. Our empirical study considers various scenarios including random, realistic, and structured problems. It reveals the advantage of the use of the proposed heuristics in the anytime framework over state-of-the-art local search algorithms.

Distributed constraint optimization for teams of mobile sensing agents

Coordinating a mobile sensor team (MST) to cover targets is a challenging problem in many multiagent applications. Such applications are inherently dynamic due to changes in the environment, technology failures, and incomplete knowledge of the agents. Agents must adaptively respond by changing their locations to continually optimize the coverage of targets. We propose distributed constraint optimization problems (DCOP)_MST, a new model for representing MST problems that is based on DCOP. In DCOP_MST, agents maintain variables for their physical positions, while each target is represented by a constraint that reflects the quality of coverage of that target. In contrast to conventional, static DCOPs, DCOP_MST not only permits dynamism but exploits it by restricting variable domains to nearby locations; consequently, variable domains and constraints change as the agents move through the environment. DCOP_MST confers three major advantages. It directly represents the multiple forms of dynamism inherent in MSTs. It also provides a compact representation that can be solved efficiently with local search algorithms, with information and communication locality based on physical locality as typically occurs in MST applications. Finally, DCOP_MST facilitates organization of the team into multiple sub-teams that can specialize in different roles and coordinate their activity through dynamic events. We demonstrate how a search-and-detection team responsible for finding new targets and a surveillance sub-team tasked with coverage of known targets can effectively work together to improve performance while using the DCOP_MST framework to coordinate. We propose different algorithms to meet the specific needs of each sub-team and several methods for cooperation between sub-teams. For the search-and-detection team, we develop an algorithm based on the DSA that forces intensive exploration for new targets. For the surveillance sub-team, we adapt several incomplete DCOP algorithms, including MGM, DSA, DBA, and Max-sum, which requires us to develop an efficient method for agents to find the value assignment in their local environment that is optimal in minimizing the maximum unmet coverage requirement over all targets. The disadvantage of dynamic domains based on physical locality is that adaptations of standard local search algorithms tend to become trapped in local optima where targets beyond the immediate range of the agents go uncovered. To address this shortcoming we develop exploration methods to be used with the local search algorithms. Our algorithms are extensively evaluated in a simulation environment. We use a reputation model to determine the individual credibility of agents and consider both additive and submodular joint credibility functions for determining coverage of targets by multiple agents. The performance is measured on two objectives: minimizing the maximum remaining coverage requirement, and minimizing the sum of remaining coverage requirements. Our results show that DSA and MGM with the exploration heuristics outperform the other incomplete algorithms across a wide range of settings. Furthermore, organizing the team into two sub-teams leads to significant gains in performance, and performance continues to improve with greater cooperation between the sub-teams.

Cognition and Multi-Agent Interaction: Evolution of a Teamwork Model

INTRODUCTION For heterogeneous agents working together to achieve complex goals, teamwork (Jennings, 1995; Yen, Yin, Ioerger, Miller, Xu, & Volz, 2001; Tambe, 1997a) has emerged as the dominant coordination paradigm. For domains as diverse as rescue response, military, space, sports, and collaboration between human workmates, flexible, dynamic coordination between cooperative agents needs to be achieved despite complex, uncertain, and hostile environments. There is now emerging consensus in the multiagent arena that for flexible teamwork among agents, each team member is provided with an explicit model of teamwork, which entails commitments and responsibilities as a team member. This explicit modeling allows the coordination to be robust, despite individual failures and unpredictably changing environments. Building on the well-developed theory of joint intentions (Cohen & Levesque, 1991) and shared plans (Grosz & Kraus, 1996), the STEAM teamwork model (Tambe, 1997a) was operationalized as a set of domain independent rules that describe how teams should work together. This domain-independent teamwork model has been successfully applied to a variety of domains. From combat air missions (Hill, Chen, Gratch, Rosenbloom, & Tambe, 1997) to robot soccer (Kitano, Asada, Kuniyoshi, Noda, Osawa, & Matsubara, 1997) to teams supporting human organizations (Pynadath & Tambe, 2003) to rescue response (Scerri, Pynadath, Johnson, P., Schurr, Si, & Tambe, 2003), applying the same set of STEAM rules has resulted in successful coordination between heterogeneous agents. The successful use of the same teamwork model in a wide variety of diverse domains provides compelling evidence that it is the principles of teamwork, rather than exploitation of specific domain phenomena, that underlie the success of teamwork-based approaches.

Reducing Untruthful Manipulation in Envy-Free Pareto Optimal Resource Allocation

A natural requirement of a resource allocation system is to guarantee fairness to its participants. Fair allocation can be achieved either by distributed protocols known as cake-cutting algorithms or by centralized approaches, which first collect the agents’ preferences and then decide on the allocation. Compared with cake-cutting algorithms, centralized approaches ageless restricted and can therefore achieve more favorable allocations. Our work uses as a starting point a recent centralized algorithm that achieves an envy-free (i.e., fair) and Pareto optimal (i.e., efficient) allocation of multiple divisible goods. In fair allocation algorithms, agents who do not follow the protocol cannot prevent other agents from being allocated a fair share, but in certain situations, agents can increase their own allocation by submitting untruthful preferences. A recent article has shown that the only mechanisms that do not allow such manipulations(i.e., the only incentive-compatible mechanisms) are dictatorial. Nevertheless, we present a method that reduces possible gains from untruthful manipulation. Our mechanism uses a heuristic to approximate optimal manipulations, and compensates agents who submitted suboptimal preferences by increasing their allocation. We empirically demonstrate that when our method is used, the additional benefit that agents can achieve by untruthful manipulation is insignificant, hence they have insignificant incentives to lie.

Balancing exploration and exploitation in incomplete Min/Max-sum inference for distributed constraint optimization

Distributed Constraint Optimization Problems (DCOPs) are NP-hard and therefore the number of studies that consider incomplete algorithms for solving them is growing. Specifically, the Max-sum algorithm has drawn attention in recent years and has been applied to a number of realistic applications. Unfortunately, in many cases Max-sum does not produce high-quality solutions. More specifically, Max-sum does not converge and explores solutions of low quality when run on problems whose constraint graph representation contains multiple cycles of different sizes. In this paper we advance the state-of-the-art in incomplete algorithms for DCOPs by: (1) proposing a version of the Max-sum algorithm that operates on an alternating directed acyclic graph (Max-sum_AD), which guarantees convergence in linear time; (2) solving a major weakness of Max-sum and Max-sum_AD that causes inconsistent costs/utilities to be propagated and affect the assignment selection, by introducing value propagation to Max-sum_AD (Max-sum_ADVP); and (3) proposing exploration heuristic methods that evidently improve the algorithms performance further. We prove that Max-sum_ADVP converges to monotonically improving states after each change of direction, and that it is guaranteed to converge in pseudo-polynomial time to a stable solution that does not change with further changes of direction. Our empirical study reveals a large improvement in the quality of the solutions produced by Max-sum_ADVP on various benchmarks, compared to the solutions produced by the standard Max-sum algorithm, Bounded Max-sum and Max-sum_AD with no value propagation. It is found to be the best guaranteed convergence inference algorithm for DCOPs. The exploration methods we propose for Max-sum_ADVP improve its performance further. However, anytime results demonstrate that their exploration level is not as efficient as a version of Max-sum, which uses Damping.

Augmenting ad hoc networks for data aggregation and dissemination

Future military operations will feature network-enabled human soldiers working seamlessly with intelligent, autonomous systems such as sensor networks and unmanned vehicles. In these systems, data aggregation (e.g., data fusion, plan monitoring) and data dissemination (e.g., information sharing) will be essential communication patterns. While ad hoc networks are commonly proposed for complex battlefield environments, they can suffer from serious connectivity and capacity limitations. In this paper we study the problem of deploying additional nodes that act as both relay nodes and data aggregation/dissemination points to meet the connectivity and aggregation/dissemination needs of the network. We focus on the NP-hard problem of determining the minimum number of supplemental nodes needed to meet the known communication requirements of an ad hoc network. We formulate the problem as a mixed integer linear program and propose three heuristics for solving the problem. We present empirical results comparing the three heuristics, showing that they perform within 25% of optimal while offering substantial running time improvements.

Toward an understanding of the impact of software personal assistants on human organizations

Intelligent software personal assistants for human organizations are an active research area within the multiagent community. However, while many capabilities for these software personal assistants are imagined or already developed, there has been no quantification of how an organizations performance is improved by software personal assistants. Moreover, while intuitively organizations will adapt to take advantage of the new technology, there has been no work looking at how organizations should or will change in response to the new technology. This paper presents a first step toward addressing this oversight. Specifically, a computational model of the working of an organization and how software personal assistants will affect that organization is developed that allows effects of software personal assistants to be modeled. By varying the potential capabilities of the software personal assistants and the structure of the organization, we can explore the impact of the technology. Our results show that managing task contingencies can greatly improve organizational performance by as much as 45%.

Multiagent Communication Security in Adversarial Settings

In many exciting multiagent applications -- including future battlefields, law enforcement, and commerce -- agents must communicate in inherently or potentially hostile environments in which an adversaries disrupt or intercept the communication between agents for malicious purposes, but the wireless ad hoc networks often proposed for these applications are particularly susceptible to attack. Intelligent agents must balance network performance with possible harm suffered from an adversarys attack, while accounting for the broadcast nature of their communication and heterogenous vulnerabilities of communication links. Furthermore, they must do so when the adversary is also actively and rationally attempting to counter their efforts. We address this challenge in this paper by representing the problem as a game between a sender agent choosing communication paths through a network and an adversary choosing nodes and links to attack. We introduce a network-flow-based approach for compactly representing the competing objectives of network performance and security from adversary attack, and provide a polynomial-time algorithm for finding the equilibrium strategy for the sender. Through empirical evaluation we show how this technique improves upon existing approaches.