Eduardo Rodrigues Gomes

Dynamic analysis of multiagent Q -learning with ε-greedy exploration

The development of mechanisms to understand and model the expected behaviour of multiagent learners is becoming increasingly important as the area rapidly find application in a variety of domains. In this paper we present a framework to model the behaviour of Q-learning agents using the ε-greedy exploration mechanism. For this, we analyse a continuous-time version of the Q-learning update rule and study how the presence of other agents and the ε-greedy mechanism affect it. We then model the problem as a system of difference equations which is used to theoretically analyse the expected behaviour of the agents. The applicability of the framework is tested through experiments in typical games selected from the literature.

Reinforcement learning with utility-aware agents for market-based resource allocation

In this paper we propose and investigate the use of Reinforcement Learning in a market-based resource allocation mechanism called Iterative Price Adjustment. Under standard assumptions, this mechanism uses demand functions that do not allow the agents to have preferences over the attributes of the allocation, e.g. the price of the resources. To address this limitation, we study the case where the agents preferences in the resource allocation are described by utility functions and they learn the demand functions given their utility functions. The approach has been evaluated with extensive experiments.

Intelligent learning objects: an agent approach to create reusable intelligent learning environments with learning objects

The possibility of reusing learning material is very important to designing learning environments for real-life learning. The reusability of learning material is based on three main features: modularity, discoverability and interoperability. Several researchers on Intelligent Learning Environments have proposed the use of Artificial Intelligence through architectures based on agent societies. Teaching systems based on Multi-Agent architectures make it possible to support the development of more interactive and adaptable systems. We proposed an approach where learning objects are built based on agent architectures. This paper discusses how the ILO approach can be used to improve the interoperability between learning objects and pedagogical agents. It presents the ILO Agent’s communication mechanism and a case study.

Improving Interoperability Among Learning Objects Using FIPA Agent Communication Framework

The reusability of learning material is based on three main features: modularity, discoverability and interoperability. Several researchers on Intelligent Learning Environments have proposed the use of architectures based on agent societies. Learning systems based on Multi-Agent architectures support the development of more interactive and adaptable systems and the Learning Objects approach gives reusability. We proposed an approach where learning objects are built based on agent architectures. This paper discusses how the Intelligent Learning Objects approach can be used to improve the interoperability between learning objects and pedagogical agents.

Learning the IPA market with individual and social rewards

Market-based mechanisms offer a promising approach for distributed resource allocation. In this paper we consider the Iterative Price Adjustment, a pricing mechanism that can be used in commodity-market resource allocation systems. We address the scenario where agents use utility functions to describe preferences in the allocation and learn demand functions optimized for the market by Reinforcement Learning. In particular, we investigate reward functions based on the individual utilities of the agents and the Social Welfare of the market. We also evaluate the quality of demand functions obtained throughout the learning process with the aim of analyzing its influence on the behavior of the agents and exploring how much learning is enough, so the amount required can be reduced. This investigation shows that both reward functions deliver similar results when the market consists of only learning agents. We further investigate this behavior and present its theoretical-experimental explanation.

The dynamics of multiagent Q-learning in commodity market resource allocation

The Commodity Market (CM) economic model offers a promising approach for the distributed resource allocation in large-scale distributed systems. Existing CM-based mechanisms apply the Economic Equilibrium concepts, assuming price-taking entities that will not engage in strategic behaviour. In this paper we address the above issue and investigate the dynamics of strategic learning agents in a specific type of CM-based mechanism called Iterative Price Adjustment. We investigate the scenario where agents use utility functions to describe preferences in the allocation and learn demand functions adapted to the market by Reinforcement Learning. The reward functions used during the learning process are based either on the individual utility of the agents, generating selfish learning agents, or the social welfare of the market, generating altruistic learning agents. Our experiments show that the market composed exclusively of selfish learning agents achieve results similar to the results obtained by the market composed of altruistic agents. Such an outcome is significant for a series of other domains where individual and social utility should be maximized but agents are not guaranteed to act cooperatively in order to achieve it or they do not want to reveal private preferences. We further investigate this outcome and present an analysis of the agents’ behaviour from the perspective of the dynamic process generated by the learning algorithm employed by them. For this, we develop a theoretical model of Multiagent Q-learning with e-greedy exploration and apply it in simplified version of the addressed scenario.

Intelligent Learning Objects: An Agent Approach to Create Interoperable Learning Objects

Reusing learning material is very important to design learning environments for real-life learning. The reusability of learning objects results from the product of three main features: modularity, discoverability and interoperability. We proposed learning objects built based on agent architectures, called Intelligent Learning Objects (ILO). This paper discusses how the ILO approach can be used to improve the interoperability among learning objects, learning menagement systems (LMS) and pedagogical agents.

Using learning objects features to promote reusability of pedagogical agents

The Learning Object idea is based on the premise that the reuse of learning material is very important for designing learning environments. The reusability of learning objects results from the product of three main features: modularity, discoverability and interoperability. In this paper we discuss how these features can be useful when added to pedagogical agents. This approach considers learning objects built according to agent architectures: the Intelligent Learning Objects approach.

Intelligent Learning Objects

Many people have been working hard to produce metadata specification towards a construction of Learning Objects in order to improve efficiency, efficacy and reusability of learning content based on an Object Oriented design paradigm. The possibility of reusing learning material is very important to designing learning environments for real-life learning. At the same time, many researchers on Intelligent Learning Environments have proposed the use of Artificial Intelligence through architectures based on agent societies. Teaching systems based on Multi-Agent architectures make it possible to support the development of more interactive and adaptable systems. This paper proposes an agent-based approach to produce more intelligent learning objects (ILO) according to the FIPA agent architecture reference model and the LOM/IEEE 1484 learning object specification.