Sherief Abdallah

Multiagent reinforcement learning and self-organization in a network of agents

To cope with large scale, agents are usually organized in a network such that an agent interacts only with its immediate neighbors in the network. Reinforcement learning techniques have been commonly used to optimize agents local policies in such a network because they require little domain knowledge and can be fully distributed. However, all of the previous work assumed the underlying network was fixed throughout the learning process. This assumption was important because the underlying network defines the learning context of each agent. In particular, the set of actions and the state space for each agent is defined in terms of the agents neighbors. If agents dynamically change the underlying network structure (also called self-organize) during learning, then one needs a mechanism for transferring what agents have learned so far before (in the old network structure) to their new learning context (in the new network structure). In this work we develop a novel self-organization mechanism that not only allows agents to self-organize the underlying network during the learning process, but also uses information from learning to guide the self-organization process. Consequently, our work is the first to study this interaction between learning and self-organization. Our self-organization mechanism uses heuristics to transfer the learned knowledge across the different steps of self-organization. We also present a more restricted version of our mechanism that is computationally less expensive and still achieve good performance. We use a simplified version of the distributed task allocation domain as our case study. Experimental results verify the stability of our approach and show a monotonic improvement in the performance of the learning process due to self-organization.

Organization-based cooperative coalition formation

The coalition formation problem has received a considerable amount of attention in recent years. In this work we present a novel distributed algorithm that returns a solution in polynomial time and the quality of the returned solution increases as agents gain more experience. Our solution utilizes an underlying organization to guide the coalition formation process. We use reinforcement learning techniques to optimize decisions made locally by agents in the organization. Experimental results are presented, showing the potential of our approach.

Learning the task allocation game

The distributed task allocation problem occurs in domains like web services, the grid, and other distributed systems. In this problem, the system consists of servers and mediators. Servers execute tasks and may differ in their capabilities, e.g. one server may take more time than the other in executing the same task. Mediators act on behalf of users, which can potentially be other mediators, and are responsible for receiving tasks from users and allocating them to servers.This work introduces a new gradient ascent learning algorithm that outperforms state of the art multiagent learners on this problem. We experimentally show that our algorithm converges faster and is less sensitive to tuning parameters than other algorithms. We also provide an informal proof that WPL has the same convergence guarantee as the best known algorithm, GIGA-WoLF. We also show that our algorithm converges in Jordans and Shapleys games where many other algorithms fail to converge. Finally, we verify the practicality of our algorithm in the distributed task allocation domain, comparing its performance to an optimal global solution.

Behavioral Experiments for Assessing the Abstract Argumentation Semantics of Reinstatement

Argumentation is a very fertile area of research in Artificial Intelligence, and various semantics have been developed to predict when an argument can be accepted, depending on the abstract structure of its defeaters and defenders. When these semantics make conflicting predictions, theoretical arbitration typically relies on ad hoc examples and normative intuition about what prediction ought to be the correct one. We advocate a complementary, descriptive-experimental method, based on the collection of behavioral data about the way human reasoners handle these critical cases. We report two studies applying this method to the case of reinstatement (both in its simple and floating forms). Results speak for the cognitive plausibility of reinstatement and yet show that it does not yield the full expected recovery of the attacked argument. Furthermore, results show that floating reinstatement yields comparable effects to that of simple reinstatement, thus arguing in favor of preferred argumentation semantics, rather than grounded argumentation semantics. Besides their theoretical value for validating and inspiring argumentation semantics, these results have applied value for developing artificial agents meant to argue with human users.

Modeling task allocation using a decision theoretic model

Mediation is the process of decomposing a task into subtasks, finding agents suitable for these subtasks and negotiating with agents to obtain commitments to execute these subtasks. This process involves several decisions to be made by a mediator including which tasks to mediate, when to interrupt the current task mediation to pursue a better task, etc. The main contribution of this work is integrating the different aspects of a mediator decision problem into one coherent and simple decision theoretic model. This model is then used to learn an optimal policy for a mediator.We propose a generalization of the original Semi Markov Decision Process (SMDP) model, which allows efficient representation of the mediator decision problem. Also the concurrent action model (CAM) is extended to allow better performing policies to be found. Experimental results are presented showing how our model outperforms the original SMDP and CAM models.

Integrating rule-based system with classification for arabic named entity recognition

Named Entity Recognition (NER) is a subtask of information extraction that seeks to recognize and classify named entities in unstructured text into predefined categories such as the names of persons, organizations, locations, etc. The majority of researchers used machine learning, while few researchers used handcrafted rules to solve the NER problem. We focus here on NER for the Arabic language (NERA), an important language with its own distinct challenges. This paper proposes a simple method for integrating machine learning with rule-based systems and implement this proposal using the state-of-the-art rule-based system for NERA. Experimental evaluation shows that our integrated approach increases the F-measure by 8 to 14% when compared to the original (pure) rule based system and the (pure) machine learning approach, and the improvement is statistically significant for different datasets. More importantly, our system outperforms the state-of-the-art machine-learning system in NERA over a benchmark dataset.

Review: representing and classifying arguments on the semantic web

Until recently, little work has been dedicated to the representation and interchange of informal, semi-structured arguments of the type found in natural language prose and dialogue. To redress this, the research community recently initiated work towards an Argument Interchange Format (AIF). The AIF aims to facilitate the exchange of semi-structured arguments among different argument analysis and argumentation-support tools. In this paper, we present a Description Logic ontology for annotating arguments, based on a new reification of the AIF and founded in Waltons theory of argumentation schemes. We demonstrate how this ontology enables a new kind of automated reasoning over argument structures, which complements classical reasoning about argument acceptability. In particular, Web Ontology Language reasoning enables significantly enhanced querying of arguments through automatic scheme classifications, instance classification, inference of indirect support in chained argument structures, and inference of critical questions. We present the implementation of a pilot Web-based system for authoring and querying argument structures using the proposed ontology.

Efficient Management of Multi-Linked Negotiation Based on a Formalized Model

abstractA Multi-linked negotiation problem occurs when an agent needs to negotiate with multiple other agents about different subjects (tasks, conflicts, or resource requirements), and the negotiation over one subject has influence on negotiations over other subjects. The solution of the multi-linked negotiations problem will become increasingly important for the next generation of advanced multi-agent systems. However, most current negotiation research looks only at a single negotiation and thus does not present techniques to manage and reason about multi-linked negotiations. In this paper, we first present a technique based on the use of a partial-order schedule and a measure of the schedule, called flexibility, which enables an agent to reason explicitly about the interactions among multiple negotiations. Next, we introduce a formalized model of the multi-linked negotiation problem. Based on this model, a heuristic search algorithm is developed for finding a near-optimal ordering of negotiation issues and their parameters. Using this algorithm, an agent can evaluate and compare different negotiation approaches and choose the best one. We show how an agent uses this technology to effectively manage interacting negotiation issues. Experimental work is presented which shows the efficiency of this approach.

Combinatorial auctions for resource allocation in a distributed sensor network

This paper discusses a solution to the problems posed by sensor resource allocation in an adaptive, distributed radar array. We have formulated a variant of the classic resource allocation problem, called the setting-based resource allocation problem, which reflects the challenges posed in domains in which sensors have multiple settings, each of which could be useful to multiple tasks. Further, we have implemented a solution to this problem that takes advantage of the locality of resources and tasks that is common to such domains. This solution involves translating tasks and possible resource configurations into bids that can be solved by a modified combinatorial auction, thus allowing us to make use of recent developments in the solution of such auctions. We have also developed an information-theoretic procedure for accomplishing this translation, which models the effect various sensor settings would have on the networks output

Organization-Based Coalition Formation

The coalition formation problem has received a considerable amount of attention in recent years. In this work we present a novel distributed algorithm that returns a solution in polynomial time and the quality of the returned solution increases as agents gain more experience. Our solution utilizes an underlying organization to guide the coalition formation process. We use reinforcement learning techniques to optimize decisions made locally by agents in the organization. Experimental results are presented, showing the potential of our approach.