C.A.M. Lima

A Hybrid Ensemble Model Applied to the Short-Term Load Forecasting Problem

In this paper we present a methodology based on a combination of many distinct predictors in an ensemble, named hybrid ensemble model, to obtain a more accurate output using the results of single predictors. As basic components, we have used artificial neural networks and support vector machines models. In order to evaluate the performance, the hybrid model was required to predict a 24 h daily series energy consumption of a Brazilian electrical operation unit located in the northeast of Brazil. The proposed ensemble model has reached an error 25% smaller than that achieved by the best single predictor. The model was initialized several times to confirm that ensembles of predictors also tend to produce low variance profiles.

Fuzzy systems design via ensembles of ANFIS

Neurofuzzy networks have become a powerful alternative strategy to develop fuzzy systems, since they are capable of learning and providing IF-THEN fuzzy rules in linguistic or explicit form. Amongst such models, ANFIS is recognized as a reference framework, mainly for its flexible and adaptive character. In this paper, we extend ANFIS theory by experimenting with a multi-net approach wherein two or more differently structured ANFIS instances are coupled to play together. Ensembles of ANFIS (E-ANFIS) enhance ANFIS performance skills and alleviate some of its computational bottlenecks. Moreover, they promote the automatic configuration of different ANFIS units and the a posteriori selective combination of their outputs. Experiments conducted to assess E-ANFIS generalization capability are also presented.

Hybrid genetic training of gated mixtures of experts for nonlinear time series forecasting

In this paper, we introduce a genetic algorithm-based training mechanism (HGT-GAME) toward the automatic structural design and parameter configuration of gated mixtures of experts (ME). In HGT-GAME, a whole ME instance is codified into a given chromosome. By employing regulatory genes, our approach enables the automatic pruning and growing of experts in a way to properly match the complexity of the task at hand. Moreover, to leverage HGT-GAMEs effectiveness a local search refinement upon each ME chromosome is performed in each generation via the gradient descent-learning algorithm. Forecasting experiments evaluate the performance of gated MEs trained with HGT-GAME.

Ensembles of support vector machines for regression problems

Support vector machines (SVMs) tackle classification and regression problems by nonlinearly mapping input data into high-dimensional feature spaces, wherein a linear decision surface is designed. Even though the high potential of these techniques has been demonstrated, their applicability has been swamped by the necessity of the a priori choice of the kernel function to realize the nonlinear mapping, which sometimes turns to be a complex and non-effective process. In this paper, we advocate that the application of neural ensembles theory to SVMs should alleviate such performance bottlenecks, because different networks with distinct kernel functions such as polynomials or radial basis functions may be created and properly combined into the same neural structure. Ensembles of SVMs, thus, promote the automatic configuration and tuning of SVMs, and have their generalization capability assessed here by means of some function regression experiments.

Least-squares support vector machines for DOA estimation: a step-by-step description and sensitivity analysis

Adaptive beamforming in antenna arrays aims at adjusting the weighted linear combination of the output signals provided by the antennas so that the power of the received signals at dominant paths is maximized at the same time that the power of interference and noise signals is minimized. The weight vectors, each one associated with one received signal can be directly obtained if the direction of arrival (DOA) of the corresponding signal has already been estimated. The process of DOA estimation involves the prediction of the angle of arrival by means of monitoring the output produced by the antennas in the array, given that the number of antennas is higher than the number of signals to be detected. Even though signal subspace techniques have made a good job in DOA estimation, they present some important drawbacks that are alleviated here using a supervised learning approach, in the form of a multiclass LS-SVM classification problem. The main contribution of this paper is twofold: a step-by-step description of the complete set of algebraic manipulation for data preprocessing and for the synthesis of the classification device, and an analysis of the effect in performance when relevant parameters vary in a given operational interval.

A Wrapper for Projection Pursuit Learning

Constructive algorithms have shown to be reliable and effective methods for designing artificial neural networks (ANN) with good accuracy and generalization capability, yet with parsimonious network structures. Projection pursuit learning (PPL) has demonstrated great flexibility and effectiveness in performing this task, though presenting some difficulties in the search for appropriate projection directions in input spaces with high dimensionality. Due to the existence of high-dimensional input spaces in the context of time series prediction, mainly under the existence of long-term dependencies in the time series, we propose here a method based on the wrapper methodology to perform variable selection, so that only a subset of highly-informative lags is going to be considered as the regression vector. The yearly sunspot number time series is adopted as a case study and comparative analysis is performed considering alternative approaches in the literature, guiding to competitive results.

Support Vector Clustering Applied to Digital Communications

Support vector clustering (SVC) is a recently proposed clustering methodology with promising performance for high-dimensional and noisy datasets, and for clusters with arbitrary shape. This work addresses the application of SVC, a kernel-based method, in a context in which the channel equalization problem is conceived as a clustering task. The main challenge, in this case, is to perform unsupervised clustering aiming at the design of an optimal Bayesian or a blind prediction-based receiver without resorting to a priori information about the transmission medium. The proposed technique employs a two-stage procedure -a combination between the use of SVC to obtain a first set of clusters and an auxiliary heuristic to help separating eventual multiple clouds contained in a single cluster and attribute centers to them via an iterated local search (ILS) algorithm. The obtained results indicate that kernel methods can be successfully applied to the field of signal processing.

Controlling Nonlinear Dynamic Systems with Projection Pursuit Learning

Projection pursuit learning (PPL) refers to a well-known constructive learning algorithm characterized by a very efficient and accurate computational procedure oriented to nonparametric regression. It has been employed as a means to counteract some problems related to the design of artificial neural network (ANN) models, namely, the estimation of a (usually large) number of free parameters, the proper definition of the models dimension, and the choice of the sources of nonlinearities (activation functions). In this work, the potentials of PPL are exploited through a different perspective, namely, in designing one-hidden-layer feedforward ANNs for the adaptive control of nonlinear dynamic systems. For such purpose, the proposed methodology is divided into three stages. In the first, the model identification process is undertaken. In the second, the ANN structure is defined according to an offline control setting. In these two stages, the PPL algorithm estimates not only the optimal number of hidden neurons but also the best activation function for each node. The final stage is performed online and promotes a fine-tuning in the parameters of the identification model and the controller. Simulation results indicate that it is possible to design effective neural models based on PPL for the control of nonlinear multivariate systems, with superior performance when compared to benchmarks