Carlos Henrique Costa Ribeiro

Heuristically accelerated reinforcement learning modularization for multi-agent multi-objective problems

This article presents two new algorithms for finding the optimal solution of a Multi-agent Multi-objective Reinforcement Learning problem. Both algorithms make use of the concepts of modularization and acceleration by a heuristic function applied in standard Reinforcement Learning algorithms to simplify and speed up the learning process of an agent that learns in a multi-agent multi-objective environment. In order to verify performance of the proposed algorithms, we considered a predator-prey environment in which the learning agent plays the role of prey that must escape the pursuing predator while reaching for food in a fixed location. The results show that combining modularization and acceleration using a heuristics function indeed produced simplification and speeding up of the learning process in a complex problem when comparing with algorithms that do not make use of acceleration or modularization techniques, such as Q-Learning and Minimax-Q.

WBLS: A signal presence-based Wi-Fi localisation system for mobile devices in smart environments

The proliferation of mobile computing devices and wireless networks has set the stage for the development of smart environments rich in computing and communication capabilities, yet gracefully integrated with human users. This paradigm has fostered a growing interest in localisation-based systems and services for portable devices, especially in indoor environments. However, designing indoor localisation systems with increasing estimation capabilities and decreasing cost installation is a challenge. An interesting approach to reach such requirements consists in using the wireless local area network (WLan) infrastructure that is already installed in many places. Most reported WLan localisation approaches use a map of received signal strength and signal presence frequency collected from multiple channels at different physical localisations in the environment, which can be very noisy. This work proposes a new localisation system, WBLS (Wireless Based Localisation System), that considers the unreliability of information on signal presence frequency in the estimation process, in an attempt to eliminate its associated noise. Experiments considering mobile agents carrying devices and moving at human walk speeds show that the most important feature of WBLS is a robustness to access points shutdowns that may happen without any warning in an environment where there is little control over the infrastructure.

Planning under uncertainty with abstraction hierarchies

This article presents a hierarchical planner to actuate in uncertain domains named HIPU – Hierarchical Planner under Uncertainty. The uncertainties treated by HIPU include the probabilistic distribution of the operators effects and a distribution of probabilities on the possible initial states of the domain. HIPU automatically creates the abstraction hierarchy that will be used during planning, and for this it uses an extension of the Alpine method, adapted to act under uncertainty conditions. The planning process in HIPU happens initially in the highest level of abstraction, and the solution found in this level is refined by lower levels, until reaching the lowest level. During the search the plan evaluation is carried out, indicating if the plan achieves the goal with a success probability larger or equal to a previously defined value. To evaluate this probability, the planner uses the forward projection method.

Incorporation of Social Features in the Naming Game for Community Detection

The organization of individuals in groups or communities is an observed property of complex social networks and this structural organization emerges naturally due to the relationships built between people on a daily basis. We believe that the opinion exchange among individuals is a key factor to this community construction, given that sharing opinions bounds people together, and disagreeing constantly would probably weaken a relationship. In this work, we analyse three models of opinion exchange that uncover the community structure of a network, based on the Naming Game (NG), a classic model of linguistic interactions of agreement. The NG-based models applied in this work insert time-changing social features to the NG dynamics in order to form communities of nodes sharing different language conventions. For this matter, we explore the models NG-AW—that incorporates trust—, NG-LEF—that incorporates uncertainty—and NG-SM—finally incorporating opinion preference. We test the algorithms in LFR networks and show that the separate addition of each social feature in the Naming Game results in improvements in community detection. Our simulations show that opinions coexist at the end of the game in non-convergent executions, each name tagging a different community, identifying, by a socially guided language dynamics, the topological communities present on the network. Moreover, the resulting trust in edges and uncertainty in nodes classify them according to role and position in the network, respectively. We observed this behavior in large networks with disjoint communities generated using LFR benchmark, and we compared our results with existing results from the literature, focusing on the quality of the community detection per se. Our model with secondary memory has shown accuracy comparable with algorithms designed specifically for topological community detection, while modeling social features that reveal communities as an emergent property, as observed in real-world social systems.

A Naming Game with Secondary Memory for Community Detection

Complex social networks are often arranged in communities of agents playing similar roles in the network, and detecting these communities can bring insights into the behaviour of such systems. Among many existing methods, a model of communication dynamics that involves exchange and agreement on shared words -- the Naming Game -- has been applied for community detection based on local interactions. In a particular variation of this game, agents with simulated social features can produce, in non convergent executions, an emergent classification of nodes and edges according to their community-related positions in the network. In this work, we analyze and discuss more deeply this variation and propose a new model which includes a secondary memory that keeps a record of word occurrences, to better reveal the communities present in the network. Each agent in the network has a preference for communicating a given word from the primary memory according to its occurrences in the secondary memory, as a human would give preference for an opinion that he/she heard many times before. Our simulations show that not only there is great improvement in the detection of communities, but also in the probability of global non-convergence -- necessary for guaranteeing different communities being tagged by different sets of shared words -- and in the adequate classification of both edges and nodes in all networks generated using two of the most popular Community Detection benchmarks.

Community detection based on a naming game

Opinion exchange in a social network can profoundly affect its structure, given that agreeing with another person bounds he/she more closely and vice versa. In this work, we show that communication interactions can form and reveal the community structure of a network, as we present a new model where agents change their links and behaviors according to the local history of communication successes. Our simulations show that a local node parameter based on such history changes relatively to the node proportion of extra-community connections, and that (adaptive) edge weights tend to get high for intra-community and low for extra-community connections, also as a consequence of the history of communication successes. In non-convergent executions, the model gets trapped on a regime where clusters of agents agreeing with different words emerge, corresponding to the existing communities in the network.

Detection of communities with Naming Game-based methods

Complex networks are often organized in groups or communities of agents that share the same features and/or functions, and this structural organization is built naturally with the formation of the system. In social networks, we argue that the dynamic of linguistic interactions of agreement among people can be a crucial factor in generating this community structure, given that sharing opinions with another person bounds them together, and disagreeing constantly would probably weaken the relationship. We present here a computational model of opinion exchange that uncovers the community structure of a network. Our aim is not to present a new community detection method proper, but to show how a model of social communication dynamics can reveal the (simple and overlapping) community structure in an emergent way. Our model is based on a standard Naming Game, but takes into consideration three social features: trust, uncertainty and opinion preference, that are built over time as agents communicate among themselves. We show that the separate addition of each social feature in the Naming Game results in gradual improvements with respect to community detection. In addition, the resulting uncertainty and trust values classify nodes and edges according to role and position in the network. Also, our model has shown a degree of accuracy both for non-overlapping and overlapping communities that are comparable with most algorithms specifically designed for topological community detection.

Exponential Mapping Controller Applied to an Autonomous Underwater Vehicle

Abstract A novel modified exponential function to achieve tracking and regulatory direct and indirect model-free feedback control of a class of nonlinear dynamical systems is presented. Its simplicity was a requisite to have it run on equipment and devices with memory and processor constraints. The control algorithm needs only two parameters and can be applied to systems based on knowledge about its free response and expected disturbances. Its main advantages are a relative ease of implementation and intuitive tuning. The controller is applied to the simulation of an Autonomous Underwater Vehicle (AUV) under currents, with promising results. In particular, its intuitive form of parameterization allowed in some cases for immediate good results or at least for good initial estimates for later tuning.

Exponential Mapping Controller Applied to Aircraft

A novel modified exponential function to achieve tracking and regulatory direct and indirect model-free feedback control of a class of nonlinear dynamical systems is presented. Its simplicity was a requisite to have it run on equipment and devices with memory and processor constraints. The control algorithm needs only two parameters and can be applied to systems based on knowledge about its free response and expected disturbances. Its main advantages are relative ease of implementation and intuitive tuning. The controller is applied to the simulation of two different aircraft under stochastic winds and wind shear and its results are compared to a classical implementation. EMC presented promising results. Its intuitive form of parameterization allowed in some cases for immediate good results or at least for good initial estimatives for later tuning.

MAP MATCHING BASED ON PARACONSISTENT ARTIFICIAL NEURAL NETWORKS

This paper presents a new method for matching metric maps generated by mobile robots that act cooperatively. This process of information matching makes it possible to perform global map generation from local maps (possibly partial and nonconsistent) provided by individual robots. The proposed method is based on a paraconsistent artificial neural network model that considers as input data the Euclidean distances between the points from each one of the partial maps. The use of this kind of input information makes the individual maps invariant with respect to relative rotation and translation among the robots in the mapping environment. The neural network then analyzes these distances to determine what are the matching belief relations among the points of the distinct maps. The algorithm implemented for the neural architecture achieved good results with very satisfactory computational performance, and made it possible to determine the certainty and contradiction degress in the map point matching analysis. The results show that the proposed approach is robust for the cases were it was applied. Equally important is the fact that the considered architecture allows for the combination of information from partial maps acquired in execution time during navigation.