Karla Figueiredo

INTELLIGENT MULTIAGENT COORDINATION BASED ON REINFORCEMENT HIERARCHICAL NEURO-FUZZY MODELS

This paper presents the research and development of two hybrid neuro-fuzzy models for the hierarchical coordination of multiple intelligent agents. The main objective of the models is to have multiple agents interact intelligently with each other in complex systems. We developed two new models of coordination for intelligent multiagent systems, which integrates the Reinforcement Learning Hierarchical Neuro-Fuzzy model with two proposed coordination mechanisms: the MultiAgent Reinforcement Learning Hierarchical Neuro-Fuzzy with a market-driven coordination mechanism (MA-RL-HNFP-MD) and the MultiAgent Reinforcement Learning Hierarchical Neuro-Fuzzy with graph coordination (MA-RL-HNFP-CG). In order to evaluate the proposed models and verify the contribution of the proposed coordination mechanisms, two multiagent benchmark applications were developed: the pursuit game and the robot soccer simulation. The results obtained demonstrated that the proposed coordination mechanisms greatly improve the performance of the multiagent system when compared with other strategies.

Irregularity detection on low tension electric installations by neural network ensembles

The volume of energy loss that Brazilian electric utilities have to deal with has been ever increasing. The electricity concessionaries are suffering significant and increasing loss in the last years, due to theft, measurement errors and many other kinds of irregularities. Therefore, there is a great concern from those companies to identify the profile of irregular customers, in order to reduce the volume of such losses. This paper presents the proposal of an intelligent system, composed of two neural networks ensembles, which intends to increase the level of accuracy in the identification of irregularities among low tension consumers. The data used to test the proposed system are from Light S.A. Company, the Rio de Janeiro concessionary. The results obtained presented a significant increase in the identification of irregular customers when compared to the current methodology employed by the company.

Reinforcement learning/spl I.bar/hierarchical neuro-fuzzy politree model for control of autonomous agents

This work presents a new hybrid neuro-fuzzy model for automatic learning of actions taken by agents. The main objective of this new model is to provide an agent with intelligence, making it capable, by interacting with its environment, to acquire and retain knowledge for reasoning (infer an action). This new model, named reinforcement learning hierarchical neuro-fuzzy politree (RL-HNFP), descends from the reinforcement learing hierarchical neuro-fuzzy BSP (RL-HNFB) that uses binary space partitioning. By using hierarchical partitioning methods, together with the reinforcement learning (RL) methodology, a new class of neuro-fuzzy systems (SNF) was obtained, which executes, in addition to automatically learning its structure, the autonomous learning of the actions to be taken by an agent. These characteristics represent an important differential when compared with the existing intelligent agents learning systems. The obtained results demonstrate the potential of this new model, which operates without any prior information, such as number of rules, rules specification, or number of partitions that the input space should have.

Multi-agent systems with reinforcement hierarchical neuro-fuzzy models

This paper introduces a new multi-agent model for intelligent agents, called reinforcement learning hierarchical neuro-fuzzy multi-agent system. This class of model uses a hierarchical partitioning of the input space with a reinforcement learning algorithm to overcome limitations of previous RL methods. The main contribution of the new system is to provide a flexible and generic model for multi-agent environments. The proposed generic model can be used in several applications, including competitive and cooperative problems, with the autonomous capacity to create fuzzy rules and expand their own rule structures, extracting knowledge from the direct interaction between the agents and the environment, without any use of supervised algorithms. The proposed model was tested in three different case studies, with promising results. The tests demonstrated that the developed system attained good capacity of convergence and coordination among the autonomous intelligent agents.

Neural networks for inflow forecasting using precipitation information

This work presents forecast models for the natural inflow in the Basin of Iguacu River, incorporating rainfall information, based on artificial neural networks. Two types of rainfall data are available: measurements taken from stations distributed along the basin and ten-day rainfall forecasts using the ETA model developed by CPTEC (Brazilian Weather Forecating Center). The neural nework model also employs observed inflows measured by stations along the Iguacu River, as well as historical data of the natural inflows to be predicted. Initially, we applied preprocessing methods on the various series, filling missing data and correcting outliers. This was followed by methods for selecting the most relevant variables for the forecast model. The results obtained demonstrate the potential of using artificial neural networks in this problem, which is highly non-linear and very complex, providing forecasts with good accuracy that can be used in planning the hydroelectrical operation of the Basin.

Hierarchical neuro-fuzzy models based on reinforcement learning for intelligent agents

This work introduces two new neuro-fuzzy systems for intelligent agents called Reinforcement Learning – Hierarchical Neuro-Fuzzy Systems BSP (RL-HNFB) and Reinforcement Learning – Hierarchical Neuro-Fuzzy Systems Politree (RL-HNFP). By using hierarchical partitioning methods, together with the Reinforcement Learning (RL) methodology, a new class of Neuro-Fuzzy Systems (SNF) was obtained, which executes, in addition to automatically learning its structure, the autonomous learning of the actions to be taken by an agent. These characteristics have been developed in order to bypass the traditional drawbacks of neuro-fuzzy systems. The paper details the two novel RL_HNF systems and evaluates their performance in a benchmark application – the cart-centering problem. The results obtained demonstrate the capacity of the proposed models in extracting knowledge from the agents direct interaction with large and/or continuous environments.

Decision Support Methods

This section presents a summary of the main concepts on which evolutionary algorithms are based. First, the operating principle of Genetic Algorithms (GAs) is explained and their main parts and their evolution parameters described. Next, a description of Cultural Algorithms (CAs) is presented and its main components are pointed out.

Hierarchical neuro-fuzzy systems

Robot audition in the real world should cope with environment noises and reverberation and motor noises caused by the robots own movements. This paper presents the active direction-pass filter (ADPF) to separate sounds originating from the specified direction with a pair of microphones. The ADPF is implemented by hierarchical integration of visual and auditory processing with hypothetical reasoning on interaural phase difference (IPD) and interaural intensity difference (IID) for each subband. In creating hypotheses, the reference data of IPD and IID is calculated by the auditory epipolar geometry on demand. Since the performance of the ADPF depends on the direction, the ADPF controls the direction by motor movement. The human tracking and sound source separation based on the ADPF is implemented on an upper-torso humanoid and runs in realtime with 4 PCs connected over Gigabit ethernet. The signal-to-noise ratio (SNR) of each sound separated by the ADPF from a mixture of two speeches with the same loudness is improved to about 10 dB from 0 dB.

Hybrid model based on genetic algorithms and neural networks to forecast tax collection: Application using endogenous and exogenous variables

Everywhere in the world tax revenues are rolled back for the commonwealth to invest and finance goods and public services, such as: infrastructure, health, security and education. The predict income revenue (taxes) is one of the challenges that the Secretariat of the Federal Revenue of Brazil (RFB for its Portuguese acronym) has. This is an important challenge since the obtained information is valuable to support the decisions pertaining the federal government financial planning. In this work, it is introduced a hybrid model based on Genetic Algorithms (GAs) and Neural Networks (NNs) for a multi-step forecast of tax revenue collection. The results were more accurate in comparison to the outcome the RFB had estimated with the indicators method. The forecast results using endogenous and exogenous variables were divided into two parts: (i) in 2013 (validation period), there was obtained a Mean Absolute Percentage Error (MAPE) of 2.37% and a decrease of the Relative Error of 11.38% to 0.49%; (ii) in 2014 (testing data set) a decrease of Relative Error of 10.82% to 3.51% was obtained.

Trajectory Tracking Control Using Echo State Networks for the CoroBot’s Arm

Different neural network models have proven being useful for tracking purposes in robotic devices. However, some models have shown superior performances to others that generate a large computational cost. This is the case of recurrent neural networks, which due to the temporal relationship existing allows satisfactory answers. Furthermore, training used by traditional algorithms, require a relatively high convergence time for some applications, especially those that are on-line. Given this problematic, this paper suggests use Echo State Networks (ESN) to perform such tasks. Additionally, results are presented for two sets of predefined tests, which were used to validate control behavior of trajectories in a manipulator embedded in a mobile platform. The results presented are related to the planar control of the manipulator in a closed loop.