Guilherme A. Barreto

Long-term time series prediction with the NARX network: An empirical evaluation

The NARX network is a dynamical neural architecture commonly used for input-output modeling of nonlinear dynamical systems. When applied to time series prediction, the NARX network is designed as a feedforward time delay neural network (TDNN), i.e., without the feedback loop of delayed outputs, reducing substantially its predictive performance. In this paper, we show that the original architecture of the NARX network can be easily and efficiently applied to long-term (multi-step-ahead) prediction of univariate time series. We evaluate the proposed approach using two real-world data sets, namely the well-known chaotic laser time series and a variable bit rate (VBR) video traffic time series. All the results show that the proposed approach consistently outperforms standard neural network based predictors, such as the TDNN and Elman architectures.

Time Series Prediction with the Self-Organizing Map: A Review

We provide a comprehensive and updated survey on applications of Kohonen’s self-organizing map (SOM) to time series prediction (TSP). The main goal of the paper is to show that, despite being originally designed as an unsupervised learning algorithm, the SOM is flexible enough to give rise to a number of efficient supervised neural architectures devoted to TSP tasks. For each SOM-based architecture to be presented, we report its algorithm implementation in detail. Similarities and differences of such SOM-based TSP models with respect to standard linear and nonlinear TSP techniques are also highlighted. We conclude the paper with indications of possible directions for further research on this field.

Diagnostic of pathology on the vertebral column with embedded reject option

Computer aided diagnosis systems with the capability of automatically decide if a patient has or not a pathology and to hold the decision on the dificult cases, are becoming more frequent. The latter are afterwards reviewed by an expert reducing therefore time consuption on behalf of the expert. The number of cases to review depends on the cost of erring the diagnosis. In this work we analyse the incorporation of the option to hold a decision on the diagnostic of pathologies on the vertebral column. A comparison with several state of the art techniques is performed. We conclude by showing that the use of the reject option techniques is an asset in line with the current view of the research community.

WCI 04 On the Application of Ensembles of Classifiers to the Diagnosis of Pathologies of the Vertebral Column: A Comparative Analysis

This paper reports results from a comprehensive performance comparison among standalone machine learning algorithms (SVM, MLP and GRNN) and their combinations in ensembles of classifiers when applied to a medical diagnosis problem in the field of orthopedics. All the aforementioned learning strategies, which currently comprises the classification module of the SINPATCO platform, are evaluated according to their ability in discriminating patients as belonging to one out of three categories: normal, disk hernia and spondylolisthesis. Confusion matrices of all learning algorithms are also reported, as well as a study of the effect of diversity in the design of the ensembles. The obtained results clearly indicate that the ensembles of classifiers have better generalization performance than standalone classifiers.

A New Look at Nonlinear Time Series Prediction with NARX Recurrent Neural Network

The NARX network is a recurrent neural architecture commonly used for input-output modeling of nonlinear systems. The input of the NARX network is formed by two tapped-delay lines, one sliding over the input signal and the other one over the output signal. Currently, when applied to chaotic time series prediction, the NARX architecture is designed as a plain Focused Time Delay Neural Network (FTDNN); thus, limiting its predictive abilities. In this paper, we propose a strategy that allows the original architecture of the NARX network to fully explore its computational power to improve prediction performance. We use the well-known chaotic laser time series to evaluate the proposed approach in multi-step-ahead prediction tasks. The results show that the proposed approach consistently outperforms standard neural network based predictors, such as the FTDNN and Elman architectures.

Short-term memory mechanisms in neural network learning of robot navigation tasks: A case study

This paper reports results of an investigation on the degree of influence of short-term memory mechanisms on the performance of neural classifiers when applied to robot navigation tasks. In particular, we deal with the well-known strategy of navigating by “wall-following”. For this purpose, four standard neural architectures (Logistic Perceptron, Multilayer Percep-tron, Mixture of Experts and Elman network) are used to associate different spatiotemporal sensory input patterns with four predetermined action categories. All stages of the experiments — data acquisition, selection and training of the architectures in a simulator and their execution on a real mobile robot — are described. The obtained results suggest that the wall-following task, formulated as a pattern classification problem, is nonlinearly separable, a result that favors the MLP network if no memory of input patters are taken into account. If short-term memory mechanisms are used, then even a linear network is able to perform the same task successfully.

Adaptive filtering with the self-organizing map: a performance comparison

In this paper we provide an in-depth evaluation of the SOM as a feasible tool for nonlinear adaptive filtering. A comprehensive survey of existing SOM-based and related architectures for learning input-output mappings is carried out and the application of these architectures to nonlinear adaptive filtering is formulated. Then, we introduce two simple procedures for building RBF-based nonlinear filters using the Vector-Quantized Temporal Associative Memory (VQTAM), a recently proposed method for learning dynamical input-output mappings using the SOM. The aforementioned SOM-based adaptive filters are compared with standard FIR/LMS and FIR/LMS-Newton linear transversal filters, as well as with powerful MLP-based filters in nonlinear channel equalization and inverse modeling tasks. The obtained results in both tasks indicate that SOM-based filters can consistently outperform powerful MLP-based ones.

Minimal Learning Machine

In this work, a novel supervised learning method, the Minimal Learning Machine (MLM), is proposed. Learning in MLM consists in building a linear mapping between input and output distance matrices. In the generalization phase, the learned distance map is used to provide an estimate of the distance from K output reference points to the unknown target output value. Then, the output estimation is formulated as multilateration problem based on the predicted output distance and the locations of the reference points. Given its general formulation, the Minimal Learning Machine is inherently capable of operating on nonlinear regression problems as well as on multidimensional response spaces. In addition, an intuitive extension of the MLM is proposed to deal with classification problems. A comprehensive set of computer experiments illustrates that the proposed method achieves accuracies that are comparable to more traditional machine learning methods for regression and classification thus offering a computationally valid alternative to such approaches.

ARTIE and MUSCLE models: building ensemble classifiers from fuzzy ART and SOM networks

Ensemble Learning has proven to be an efficient method to improve the performance of single classifiers. In this context, the present article introduces ARTIE (ART networks in Ensembles) and MUSCLE (Multiple SOM Classifiers in Ensembles), two novel ensemble models that use Fuzzy ART and SOM networks as base classifiers, respectively. In addition, a hybrid metaheuristic solution based on Particle Swarm Optimization and Simulated Annealing is used for parameter tuning of the base classifiers. A comprehensive performance comparison using 10 benchmarking data sets indicates that the ARTIE and MUSCLE architectures consistently outperform ensembles built from standard supervised neural networks, such as the Fuzzy ARTMAP, Learning Vector Quantization, and the Extreme Learning Machine.

A Robust Extreme Learning Machine for pattern classification with outliers

In this paper we introduce a simple and efficient extension of the Extreme Learning Machine (ELM) network (Huang et al., 2006 19), which is very robust to label noise, a type of outlier occurring in classification tasks. Such outliers usually result from mistakes during labeling of the data points (e.g. misjudgment of a specialist) or from typing errors during creation of data files (e.g. by striking an incorrect key on a keyboard). The proposed variant of the ELM, henceforth named Robust ELM (RELM), is designed using M-estimators to compute the output weights instead of the standard ordinary least squares (OLS) method. We evaluate the performance of the RELM using batch and recursive learning rules, and also introduce a model selection strategy based on Particle Swarm Optimization (PSO) to find an optimal architecture for datasets contaminated with non-Gaussian noise and outliers. By means of comprehensive computer simulations using synthetic and real-world datasets, we show that the proposed Robust ELM classifiers consistently outperforms the original version.