Romis Attux

2012 Special Issue: An extended echo state network using Volterra filtering and principal component analysis

Echo state networks (ESNs) can be interpreted as promoting an encouraging compromise between two seemingly conflicting objectives: (i) simplicity of the resulting mathematical model and (ii) capability to express a wide range of nonlinear dynamics. By imposing fixed weights to the recurrent connections, the echo state approach avoids the well-known difficulties faced by recurrent neural network training strategies, but still preserves, to a certain extent, the potential of the underlying structure due to the existence of feedback loops within the dynamical reservoir. Moreover, the overall training process is relatively simple, as it amounts essentially to adapting the readout, which usually corresponds to a linear combiner. However, the linear nature of the output layer may limit the capability of exploring the available information, since higher-order statistics of the signals are not taken into account. In this work, we present a novel architecture for an ESN in which the linear combiner is replaced by a Volterra filter structure. Additionally, the principal component analysis technique is used to reduce the number of effective signals transmitted to the output layer. This idea not only improves the processing capability of the network, but also preserves the simplicity of the training process. The proposed architecture is then analyzed in the context of a set of representative information extraction problems, more specifically supervised and unsupervised channel equalization, and blind separation of convolutive mixtures. The obtained results, when compared to those produced by already proposed ESN versions, highlight the benefits brought by the novel network proposal and characterize it as a promising tool to deal with challenging signal processing tasks.

Unsupervised Signal Processing : Channel Equalization and Source Separation

Introduction Channel Equalization Source Separation Organization and Contents Statistical Characterization of Signals and Systems Signals and Systems Digital Signal Processing Probability Theory and Randomness Stochastic Processes Estimation Theory Linear Optimal and Adaptive Filtering Supervised Linear Filtering Wiener Filtering The Steepest-Descent Algorithm The Least Mean Square Algorithm The Method of Least Squares A Few Remarks Concerning Structural Extensions Linear Filtering without a Reference Signal Linear Prediction Revisited Unsupervised Channel Equalization The Unsupervised Deconvolution Problem Fundamental Theorems Bussgang Algorithms The Shalvi-Weinstein Algorithm The Super-Exponential Algorithm Analysis of the Equilibrium Solutions of Unsupervised Criteria Relationships between Equalization Criteria Unsupervised Multichannel Equalization Systems withMultiple Inputs and/orMultiple Outputs SIMO Channel Equalization Methods for Blind SIMO Equalization MIMO Channels and Multiuser Processing Blind Source Separation The Problem of Blind Source Separation Independent Component Analysis Algorithms for Independent Component Analysis Other Approaches for Blind Source Separation Convolutive Mixtures Nonlinear Mixtures Nonlinear Filtering and Machine Learning Decision-Feedback Equalizers Volterra Filters Equalization as a Classification Task Artificial Neural Network Bio-Inspired Optimization Methods Why Bio-Inspired Computing? Genetic Algorithms Artificial Immune Systems Particle Swarm Optimization Appendix A: Some Properties of the Correlation Matrix Appendix B: Kalman Filter References Index

Multivariate ant colony optimization in continuous search spaces

This work introduces an ant-inspired algorithm for optimization in continuous search spaces that is based on the generation of random vectors with multivariate Gaussian pdf. The proposed approach is called MACACO -- Multivariate Ant Colony Algorithm for Continuous Optimization -- and is able to simultaneously adapt all the dimensions of the random distribution employed to generate the new individuals at each iteration. In order to analyze MACACOs search efficiency, the approach was compared to a pair of counterparts: the Continuous Ant Colony System (CACS) and the approach known as Ant Colony Optimization in en (ACOR). The comparative analysis, which involves well-known benchmark problems from the literature, has indicated that MACACO outperforms CACS and ACOR in most cases as the quality of the final solution is concerned, and it is just about two times more costly than the least expensive contender.

Blind Compensation of Nonlinear Distortions: Application to Source Separation of Post-Nonlinear Mixtures

In this paper, we address the problem of blind compensation of nonlinear distortions. Our approach relies on the assumption that the input signal is bandlimited. We then make use of the classical result that the output of a nonlinearity has a wider spectrum than the one of the input signal. However, differently from previous works, our approach does not assume knowledge of the input signal bandwidth. The proposal is considered in the development of a two-stage method for blind source separation (BSS) in post-nonlinear (PNL) models. Indeed, once the functions present in the nonlinear stage of a PNL model are compensated, one can apply the well-established linear BSS algorithms to complete the task of separating the sources. Numerical experiments performed in different scenarios attest the viability of the proposal. Moreover, the proposed method is tested in a real situation where the data are acquired by smart chemical sensor arrays.

An echo state network architecture based on volterra filtering and PCA with application to the channel equalization problem

Echo state networks represent a promising alternative to the classical approaches involving recurrent neural networks, as they ally processing capability, due to the existence of feedback loops within the dynamical reservoir, with a simplified training process. However, the existing networks cannot fully explore the potential of the underlying structure, since the outputs are computed via linear combinations of the internal states. In this work, we propose a novel architecture for an echo state network that employs the Volterra filter structure in the output layer together with the Principal Component Analysis technique. This idea not only improves the processing capability of the network, but also preserves the simplicity of the training process. The proposed architecture has been analyzed in the context of the channel equalization problem, and the obtained results highlight the adequacy and the advantages of the novel network, which achieved a convincing performance, overcoming the other echo state networks, especially in the most challenging scenarios.

Comparative analysis of strategies for feature extraction and classification in SSVEP BCIs.

Abstract Brain–computer interface (BCI) systems based on electroencephalography have been increasingly used in different contexts, engendering applications from entertainment to rehabilitation in a non-invasive framework. In this study, we perform a comparative analysis of different signal processing techniques for each BCI system stage concerning steady state visually evoked potentials (SSVEP), which includes: (1) feature extraction performed by different spectral methods (bank of filters, Welchs method and the magnitude of the short-time Fourier transform); (2) feature selection by means of an incremental wrapper, a filter using Pearsons method and a cluster measure based on the Davies–Bouldin index, in addition to a scenario with no selection strategy; (3) classification schemes using linear discriminant analysis (LDA), support vector machines (SVM) and extreme learning machines (ELM). The combination of such methodologies leads to a representative and helpful comparative overview of robustness and efficiency of classical strategies, in addition to the characterization of a relatively new classification approach (defined by ELM) applied to the BCI-SSVEP systems.

An immune-inspired information-theoretic approach to the problem of ICA over a Galois field

The problem of independent component analysis (ICA) was firstly formulated and studied in the context of real-valued signals and mixing models, but, recently, an extension of this original formulation was proposed to deal with the problem within the framework of finite fields. In this work, we propose a strategy to deal with ICA over these fields that presents two novel features: (i) it is based on the use of a cost function built directly from an estimate of the mutual information and (ii) it employs an artificial immune system to perform the search for efficient separating matrices, in contrast with the existing techniques, which are based on search schemes of an exhaustive character. The new proposal is subject to a comparative analysis based on different simulation scenarios and the work is concluded by an analysis of perspectives of practical application to digital and genomic data mining.

Blind source separation of post-nonlinear mixtures using evolutionary computation and order statistics

In this work, we address the problem of source separation of post-nonlinear mixtures based on mutual information minimization. There are two main problems related to the training of separating systems in this case: the requirement of entropy estimation and the risk of local convergence. In order to overcome both difficulties, we propose a training paradigm based on entropy estimation through order statistics and on an evolutionary-based learning algorithm. Simulation results indicate the validity of the novel approach.

Blind search for optimal Wiener equalizers using an artificial immune network model

This work proposes a framework to determine the optimal Wiener equalizer by using an artificial immune network model together with the constant modulus (CM) cost function. This study was primarily motivated by recent theoretical results concerning the CM criterion and its relation to the Wiener approach. The proposed immune-based technique was tested under different channel models and filter orders, and benchmarked against a procedure using a genetic algorithm with niching. The results demonstrated that the proposed strategy has a clear superiority when compared with the more traditional technique. The proposed algorithm presents interesting features from the perspective of multimodal search, being capable of determining the optimal Wiener equalizer in most runs for all tested channels.

Application of natural computing algorithms to maximum likelihood estimation of direction of arrival

This work presents a study of the performance of populational meta-heuristics belonging to the field of natural computing when applied to the problem of direction of arrival (DOA) estimation, as well as an overview of the literature about the use of such techniques in this problem. These heuristics offer a promising alternative to the conventional approaches in DOA estimation, as they search for the global optima of the maximum likelihood (ML) function in a framework characterized by an elegant balance between global exploration and local improvement, which are interesting features in the context of multimodal optimization, to which the ML-DOA estimation problem belongs. Thus, we shall analyze whether these algorithms are capable of implementing the ML estimator, i.e., finding the global optima of the ML function. In this work, we selected three representative natural computing algorithms to perform DOA estimation: differential evolution, clonal selection algorithm, and the particle swarm. Simulation results involving different scenarios confirm that these methods can reach the performance of the ML estimator, regardless of the number of sources and/or their nature. Moreover, the number of points evaluated by such methods is quite inferior to that associated with a grid search, which gives support to their application.