Sebastián Basterrech

An experimental analysis of the Echo State Network initialization using the Particle Swarm Optimization

This article introduces a robust hybrid method for solving supervised learning tasks, which uses the Echo State Network (ESN) model and the Particle Swarm Optimization (PSO) algorithm. An ESN is a Recurrent Neural Network with the hidden-hidden weights fixed in the learning process. The recurrent part of the network stores the input information in internal states of the network. Another structure forms a free-memory method used as supervised learning tool. The setting procedure for initializing the recurrent structure of the ESN model can impact on the model performance. On the other hand, the PSO has been shown to be a successful technique for finding optimal points in complex spaces. Here, we present an approach to use the PSO for finding some initial hidden-hidden weights of the ESN model. We present empirical results that compare the canonical ESN model with this hybrid method on a wide range of benchmark problems.

Irradiance prediction using Echo State Queueing Networks and Differential polynomial Neural Networks

This paper investigates the estimation of a real time-series benchmark: the solar irradiance forcasting. The global solar irradiance is an important variable in the production of renewable energy sources. These variable is very unstable and hard to be predicted. For the prediction, we use two new models for time-series modeling: Echo State Queueing Networks and Differential polynomial Neural Networks. Both tools have been proven to be efficient for forecasting and time-series modeling. We compare their performances for this particular data set.

Optimal design of neural tree for solar power prediction

Today renewable energy sources are integral part of energy mix in most of countries in the world. Carbon reduction issues and other ecological activity provide a wide possibility to progressive increase of installed capacity of renewable energy sources. Huge distribution of instable renewable energy sources like wind and photovoltaic plants brings new tasks in power system control and power system reliability. Prediction of power production is one of the ways to mitigate negative impact of operation of instable energy sources. This work presents application of a neural tree method from the group of soft computing method for renewable energy prediction. In this work we focus on photovoltaic power plant production prediction.

Initializing reservoirs with exhibitory and inhibitory signals using unsupervised learning techniques

The trend of Reservoir Computing (RC) has been gaining prominence in the Neural Computation community since the 2000s. In a RC model there are at least two well-differentiated structures. One is a recurrent part called reservoir, which expands the input data and historical information into a high-dimensional space. This projection is carried out in order to enhance the linear separability of the input data. Another part is a memory-less structure designed to be robust and fast in the learning process. RC models are an alternative of Turing Machines and Recurrent Neural Networks to model cognitive processing in the neural system. Additionally, they are interesting Machine Learning tools to Time Series Modeling and Forecasting. Recently a new RC model was introduced under the name of Echo State Queueing Networks (ESQN). In this model the reservoir is a dynamical system which arises from the Queueing Theory. The initialization of the reservoir parameters may influence the model performance. Recently, some unsupervised techniques were used to improve the performance of one specific RC method. In this paper, we apply these techniques to set the reservoir parameters of the ESQN model. In particular, we study the ESQN model initialization using Self-Organizing Maps. Additionally, we test the model performance initializing the reservoir employing Hebbian rules. We present an empirical comparison of these reservoir initializations using a range of time series benchmarks.

An Experimental Analysis of Reservoir Parameters of the Echo State Queueing Network Model

During the last years, there has been a growing interest in the Reservoir Computing (RC) paradigm. Recently, a new RC model was presented under the name of Echo State Queueing Networks (ESQN). This model merges ideas from Queueing Theory and one of the two pioneering RC techniques, Echo State Networks. In a RC model there is a dynamical system called reservoir which serves to expand the input data into a larger space. This expansion can enhance the linear separability of the data. In the case of ESQN, the reservoir is a Recurrent Neural Network composed of spiking neurons which fire positive and negative signals. Unlike other RC models, an analysis of the dynamics behavior of the ESQN system is still to be done. In this work, we present an experimental analysis of these dynamics. In particular, we study the impact of the spectral radius of the reservoir in system stability. In our experiments, we use a range of benchmark time series data.

Hidden Markov models for gene sequence classification

The article presents an application of hidden Markov models (HMMs) for pattern recognition on genome sequences. We apply HMM for identifying genes encoding the variant surface glycoprotein (VSG) in the genomes of Trypanosoma brucei (T. brucei) and other African trypanosomes. These are parasitic protozoa causative agents of sleeping sickness and several diseases in domestic and wild animals. These parasites have a peculiar strategy to evade the host’s immune system that consists in periodically changing their predominant cellular surface protein (VSG). The motivation for using patterns recognition methods to identify these genes, instead of traditional homology based ones, is that the levels of sequence identity (amino acid and DNA sequence) amongst these genes is often below of what is considered reliable in these methods. Among pattern recognition approaches, HMM are particularly suitable to tackle this problem because they can handle more naturally the determination of gene edges. We evaluate the performance of the model using different number of states in the Markov model, as well as several performance metrics. The model is applied using public genomic data. Our empirical results show that the VSG genes on T. brucei can be safely identified (high sensitivity and low rate of false positives) using HMM.

Random Neural Network Model for Supervised Learning Problems

Random Neural Networks (RNNs) are a class of Neural Networks (NNs) that can also be seen as a specific type of queuing network. They have been successfully used in several domains during the last 25 years, as queuing networks to analyze the performance of resource sharing in many engineering areas, as learning tools and in combinatorial optimization, where they are seen as neural systems, and also as models of neurological aspects of living beings. In this article we focus on their learning capabilities, and more specifically, we present a practical guide for using the RNN to solve supervised learning problems. We give a general description of these models using almost indistinctly the terminology of Queuing Theory and the neural one. We present the standard learning procedures used by RNNs, adapted from similar well-established improvements in the standard NN field. We describe in particular a set of learning algorithms covering techniques based on the use of first order and, then, of second order derivatives. We also discuss some issues related to these objects and present new perspectives about their use in supervised learning problems. The tutorial describes their most relevant applications, and also provides a large bibliography.Random Neural Networks (RNNs) are a class of Neural Networks (NNs) that can also be seen as a specific type of queuing network. They have been successfully used in several domains during the last 25 years, as queuing networks to analyze the performance of resource sharing in many engineering areas, as learning tools and in combinatorial optimization, where they are seen as neural systems, and also as models of neurological aspects of living beings. In this article we focus on their learning capabilities, and more specifically, we present a practical guide for using the RNN to solve supervised learning problems. We give a general description of these models using almost indistinctly the terminology of Queuing Theory and the neural one. We present the standard learning procedures used by RNNs, adapted from similar well-established improvements in the standard NN field. We describe in particular a set of learning algorithms covering techniques based on the use of first order and, then, of second order derivatives. We also discuss some issues related to these objects and present new perspectives about their use in supervised learning problems. The tutorial describes their most relevant applications, and also provides a large bibliography.

Solar Irradiance Estimation Using the Echo State Network and the Flexible Neural Tree

Two popular models for solving temporal learning problems are the Flexible Neural Tree (FNT) and the Echo State Network (ESN). Both models belong to the the Neural Network area. The ESN is based in the projection of a recurrent neural network to model the temporal dependencies of the data. The FNT uses heuristic techniques for finding a tree topology and its parameters. There are several examples in the Machine Learning literature that shown the success for solving learning tasks of both techniques. In this paper, we have studied the performance of these methods in a specific data set about renewable energy.

A more powerful random neural network model in supervised learning applications

Since the early 1990s, Random Neural Networks (RNNs) have gained importance in the Neural Networks and Queueing Networks communities. RNNs are inspired by biological neural networks and they are also an extension of open Jacksons networks in Queueing Theory. In 1993, a learning algorithm of gradient type was introduced in order to use RNNs in supervised learning tasks. This method considers only the weight connections among the neurons as adjustable parameters. All other parameters are deemed fixed during the training process. The RNN model has been successfully utilized in several types of applications such as: supervised learning problems, pattern recognition, optimization, image processing, associative memory. In this contribution we present a modification of the classic model obtained by extending the set of adjustable parameters. The modification increases the potential of the RNN model in supervised learning tasks keeping the same network topology and the same time complexity of the algorithm. We describe the new equations implementing a gradient descent learning technique for the model.

Geometric Particle Swarm Optimization and Reservoir Computing for Solar Power Forecasting

Solar irradiance is an alternative of renewable resource that can be used for covering a relevant part of the growing demand of electrical energy. To have accurate solar irradiance predictions can help to integrate the solar power resources into the grid. We analyse the performance of an automatic procedure for selecting the most significant input features that impacts on the solar irradiance. The approach is based on a generalisation of swarm optimisation named Geometrical Particle Swarm Optimization (GPSO). Once, a good combination of weather information is defined, we use a reservoir computing model as forecasting technique. In particular, we use the Echo State Networks (ESN) model that is a Recurrent Neural Network often used for solving temporal learning problems. We evaluate our approach on a well-known public meteorological dataset obtaining promising results.