Dimitar Kazakov

Adaptive Agents and Multi-Agent Systems II

Cooperation and learning are two ways in which an agent can improve its performance. Cooperative Multiagent Learning is a framework to analyze the tradeoff between cooperation and learning in multiagent systems. We focus on multiagent systems where individual agents are capable of solving problems and learning using CBR (Case-based Reasoning). We present several collaboration strategies for agents that learn and their empirical results in several experiments. Finally we analyze the collaboration strategies and their results along several dimensions, like number of agents, redundancy, CBR technique used, and individual decision policies.

Adaptive agents and multi-agent systems: adaptation and multi-agent learning

To Adapt or Not to Adapt - Consequences of Adapting Driver and Traffic Light Agents.- Optimal Control in Large Stochastic Multi-agent Systems.- Continuous-State Reinforcement Learning with Fuzzy Approximation.- Using Evolutionary Game-Theory to Analyse the Performance of Trading Strategies in a Continuous Double Auction Market.- Parallel Reinforcement Learning with Linear Function Approximation.- Combining Reinforcement Learning with Symbolic Planning.- Agent Interactions and Implicit Trust in IPD Environments.- Collaborative Learning with Logic-Based Models.- Priority Awareness: Towards a Computational Model of Human Fairness for Multi-agent Systems.- Bifurcation Analysis of Reinforcement Learning Agents in the Seltens Horse Game.- Bee Behaviour in Multi-agent Systems.- Stable Cooperation in the N-Player Prisoners Dilemma: The Importance of Community Structure.- Solving Multi-stage Games with Hierarchical Learning Automata That Bootstrap.- Auctions, Evolution, and Multi-agent Learning.- Multi-agent Reinforcement Learning for Intrusion Detection.- Networks of Learning Automata and Limiting Games.- Multi-agent Learning by Distributed Feature Extraction.

Unsupervised Learning of Word Segmentation Rules with Genetic Algorithms and Inductive Logic Programming

This article presents a combination of unsupervised and supervised learning techniques for the generation of word segmentation rules from a raw list of words. First, a language bias for word segmentation is introduced and a simple genetic algorithm is used in the search for a segmentation that corresponds to the best bias value. In the second phase, the words segmented by the genetic algorithm are used as an input for the first order decision list learner CLOG. The result is a set of first order rules which can be used for segmentation of unseen words. When applied on either the training data or unseen data, these rules produce segmentations which are linguistically meaningful, and to a large degree conforming to the annotation provided.

Machine learning and inductive logic programming for multi-agent systems

Learning is a crucial ability of intelligent agents. Rather than presenting a complete literature review, we focus in this paper on important issues surrounding the application of machine learning (ML) techniques to agents and multi-agent systems (MAS). In this discussion we move from disembodied ML over single-agent learning to full multi-agent learning. In the second part of the paper we focus on the application of Inductive Logic Programming, a knowledge-based ML technique, to MAS, and present an implemented framework in which multi-agent learning experiments can be carried out.

A hybrid approach to word segmentation

This article presents a combination of unsupervised and supervised learning techniques for generation of word segmentation rules from a list of words. First, a bias for word segmentation is introduced and a simple genetic algorithm is used for the search of segmentation that corresponds to the best bias value. In the second phase, the segmentation obtained from the genetic algorithm is used as an input for two inductive logic programming algorithms, namely FoIDL and CLOG. The result is a logic program that can be used for segmentation of unseen words. The learnt program contains affixes which are characteristic for the given language and can be used in other morphology tasks.

Data summarization approach to relational domain learning based on frequent pattern to support the development of decision making

A new approach is needed to handle huge dataset stored in multiple tables in a very-large database. Data mining and Knowledge Discovery in Databases (KDD) promise to play a crucial role in the way people interact with databases, especially decision support databases where analysis and exploration operations are essential. In this paper, we present related works in Relational Data Mining, define the basic notions of data mining for decision support and the types of data aggregation as a means of categorizing or summarizing data. We then present a novel approach to relational domain learning to support the development of decision making models by introducing automated construction of hierarchical multi-attribute model for decision making. We will describe how relational dataset can naturally be handled to support the construction of hierarchical multi-attribute model by using relational aggregation based on pattern’s distance. In this paper, we presents the prototype of “Dynamic Aggregation of Relational Attributes” (hence called DARA) that is capable of supporting the construction of hierarchical multi-attribute model for decision making. We experimentally show these results in a multi-relational domain that shows higher percentage of correctly classified instances and illustrate set of rules extracted from the relational domains to support decision-making.

The origins of syntax: from navigation to language

This article suggests that the parser underlying human syntax may have originally evolved to assist navigation, a claim supported by computational simulations as well as evidence from neuroscience and psychology. We discuss two independent conjectures about the way in which navigation could have supported the emergence of this aspect of the human language faculty: firstly, by promoting the development of a parser; and secondly, by possibly providing a topic of discussion to which this parser could have been applied with minimum effort. The paper summarizes our previously published experiments and provides original results in support of the evolutionary advantages this type of communication can provide, compared with other foraging strategies. Another aspect studied in the experiments is the combination and range of environmental factors that make communication beneficial, focusing on the availability and volatility of resources. We suggest that the parser evolved for navigation might initially have been limited to handling regular languages, and describe a mechanism that may have created selective pressure for a context-free parser.

Particle swarm optimization of Bollinger bands

The use of technical indicators to derive stock trading signals is a foundation of financial technical analysis. Many of these indicators have several parameters which creates a difficult optimization problem given the highly non-linear and non-stationary nature of a financial time-series. This study investigates a popular financial indicator, Bollinger Bands, and the fine tuning of its parameters via particle swarm optimization under 4 different fitness functions: profitability, Sharpe ratio, Sortino ratio and accuracy. The experiment results show that the parameters optimized through PSO using the profitability fitness function produced superior out-of-sample trading results which includes transaction costs when compared to the default parameters.

Guaranteed Loop Bound Identification from Program Traces for WCET

Static analysis can be used to determine safe estimates of Worst Case Execution Time. However, overestimation of the number of loop iterations, particularly in nested loops, can result in substantial pessimism in the overall estimate. This paper presents a method of determining exact parametric values of the number of loop iterations for a particular class of arbitrarily deeply nested loops. It is proven that values are guaranteed to be correct using information obtainable from a finite and quantifiable number of program traces. Using the results of this proof, a tool is constructed and its scalability assessed.

Stochastic simulation of inherited kinship-driven altruism

The aim of this research is to assess the role of a hypothetical inherited feature (gene) promoting altruism between relatives as a factor for survival in the context of a multi-agent system simulating natural selection. Classical Darwinism and Neo-Darwinism are compared, and the principles of the latter are implemented in the system. The experiments study the factors that influence the successful propagation of altruistic behaviour in the population. The results show that the natural phenomenon of kinship-driven altruism has been successfully replicated in a multi-agent system, which implements a model of natural selection different from the one commonly used in genetic algorithms and multiagent systems, and closer to nature.