André L. V. Coelho

Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification

Mixture of experts (ME) models comprise a family of modular neural network architectures aiming at distilling complex problems into simple subtasks. This is done by deploying a separate gating module for softly dividing the input space into overlapping regions to be each assigned to one or more expert networks. Conversely, support vector machines (SVMs) refer to kernel-based methods, neural-network-alike models that constitute an approximate implementation of the structural risk minimization principle. Such learning machines follow the simple, but powerful idea of nonlinearly mapping input data into high-dimensional feature spaces wherein a linear decision surface discriminating different regions is properly designed. In this work, we formally characterize and empirically evaluate a novel approach, named as Mixture of Support Vector Machine Experts (MSVME), whose main purpose is to combine the complementary properties of both SVM and ME models. In the formal characterization, an algorithm based on a maximum likelihood criterion is considered for the MSVME training, and we demonstrate that it is possible to train each expert based on an SVM perspective. Regarding the empirical evaluation, simulation results involving nonlinear dynamic system identification problems are reported, contrasting the performance shown by the MSVME approach with that exhibited by conventional SVM and ME models.

Automatic EEG signal classification for epilepsy diagnosis with Relevance Vector Machines

In this paper, we investigate the potentials of applying a kernel-based learning machine, the Relevance Vector Machine (RVM), to the task of epilepsy detection through automatic electroencephalogram (EEG) signal classification. For this purpose, some experiments have been conducted over publicly available data, contrasting the performance levels exhibited by RVM models with those achieved with Support Vector Machines (SVMs), both in terms of predictive accuracy and sensitivity to the choice of the kernel function. Four settings of both types of kernel machine were considered in this study, which vary in accord with the type of input data they receive, either raw EEG signal or some statistical features extracted from the wavelet-transformed data. The empirical results indicate that: (1) in terms of accuracy, the best-calibrated RVM models have shown very satisfactory performance levels, which are rather comparable to those of SVMs; (2) an increase of accuracy is sometimes accompanied by loss of sparseness in the resulting RVM models; (3) both types of machines present similar sensitivity profiles to the kernel functions considered, having some kernel parameter values clearly associated with better accuracy rate; (4) when not making use of a feature extraction technique, the choice of the kernel function seems to be very relevant for significantly leveraging the performance of RVMs; and (5) when making use of derived features, the choice of the feature extraction technique seems to be an important factor to one take into account.

EEG signal classification for epilepsy diagnosis via optimum path forest - A systematic assessment

Abstract Epilepsy refers to a set of chronic neurological syndromes characterized by transient and unexpected electrical disturbances of the brain. The detailed analysis of the electroencephalogram (EEG) is one of the most influential steps for the proper diagnosis of this disorder. This work presents a systematic performance evaluation of the recently introduced optimum path forest (OPF) classifier when coping with the task of epilepsy diagnosis directly through EEG signal analysis. For this purpose, we have made extensive use of a benchmark dataset composed of five classes, whose full discrimination is very hard to achieve. Four types of wavelet functions and three well-known filter methods were considered for the tasks of feature extraction and selection, respectively. Moreover, support vector machines configured with radial basis function (SVM-RBF) kernel, multilayer perceptron neural networks (ANN-MLP), and Bayesian classifiers were used for comparison in terms of effectiveness and efficiency. Overall, the results evidence the outperformance of the OPF classifier in both types of criteria. Indeed, the OPF classifier was usually extremely fast, with average training/testing times much lower than those required by SVM-RBF and ANN-MLP. Moreover, when configured with Coiflets as feature extractors, the performance scores achieved by the OPF classifier include 89.2% as average accuracy and sensitivity/specificity values higher than 80% for all five classes.

Kernel machines for epilepsy diagnosis via EEG signal classification: A comparative study

OBJECTIVE We carry out a systematic assessment on a suite of kernel-based learning machines while coping with the task of epilepsy diagnosis through automatic electroencephalogram (EEG) signal classification. METHODS AND MATERIALS The kernel machines investigated include the standard support vector machine (SVM), the least squares SVM, the Lagrangian SVM, the smooth SVM, the proximal SVM, and the relevance vector machine. An extensive series of experiments was conducted on publicly available data, whose clinical EEG recordings were obtained from five normal subjects and five epileptic patients. The performance levels delivered by the different kernel machines are contrasted in terms of the criteria of predictive accuracy, sensitivity to the kernel function/parameter value, and sensitivity to the type of features extracted from the signal. For this purpose, 26 values for the kernel parameter (radius) of two well-known kernel functions (namely, Gaussian and exponential radial basis functions) were considered as well as 21 types of features extracted from the EEG signal, including statistical values derived from the discrete wavelet transform, Lyapunov exponents, and combinations thereof. RESULTS We first quantitatively assess the impact of the choice of the wavelet basis on the quality of the features extracted. Four wavelet basis functions were considered in this study. Then, we provide the average accuracy (i.e., cross-validation error) values delivered by 252 kernel machine configurations; in particular, 40%/35% of the best-calibrated models of the standard and least squares SVMs reached 100% accuracy rate for the two kernel functions considered. Moreover, we show the sensitivity profiles exhibited by a large sample of the configurations whereby one can visually inspect their levels of sensitiveness to the type of feature and to the kernel function/parameter value. CONCLUSIONS Overall, the results evidence that all kernel machines are competitive in terms of accuracy, with the standard and least squares SVMs prevailing more consistently. Moreover, the choice of the kernel function and parameter value as well as the choice of the feature extractor are critical decisions to be taken, albeit the choice of the wavelet family seems not to be so relevant. Also, the statistical values calculated over the Lyapunov exponents were good sources of signal representation, but not as informative as their wavelet counterparts. Finally, a typical sensitivity profile has emerged among all types of machines, involving some regions of stability separated by zones of sharp variation, with some kernel parameter values clearly associated with better accuracy rates (zones of optimality).

Tackling EEG signal classification with least squares support vector machines: A sensitivity analysis study

The electroencephalogram (EEG) signal captures the electrical activity of the brain and is an important source of information for studying neurological disorders. The proper analysis of this biological signal plays an important role in the domain of brain-computer interface, which aims at the construction of communication channels between human brain and computers. In this paper, we investigate the application of least squares support vector machines (LS-SVM) to the task of epilepsy diagnosis through automatic EEG signal classification. More specifically, we present a sensitivity analysis study by means of which the performance levels exhibited by standard and least squares SVM classifiers are contrasted, taking into account the setting of the kernel function and of its parameter value. Results of experiments conducted over different types of features extracted from a benchmark EEG signal dataset evidence that the sensitivity profiles of the kernel machines are qualitatively similar, both showing notable performance in terms of accuracy and generalization. In addition, the performance accomplished by optimally configured LS-SVM models is also quantitatively contrasted with that obtained by related approaches for the same dataset.

Devising adaptive migration policies for cooperative distributed genetic algorithms

Distributed genetic algorithms (DGAs) constitute an interesting approach to undertake the premature convergence problem in evolutionary optimization. This is done by spatial partitioning a huge panmitic population into several semi-isolated groups, called demes, each evolving in parallel by its own pace, and possibly exploring different regions of the search space. At the center of such approach lies the migratory process that simulates the swapping of individuals belonging to different demes, in such a way to ensure the sharing of good genetic material. In this paper, we model the migration step in DGAs as an explicit means to promote cooperation among genetic agents, autonomous entities encapsulating GA instances for possibly tackling different sub-problems of a complicated task. The focus is on the characterization of adaptive migration policies in which the choice of what individuals to migrate and/or replace is not defined a priori but according to a more knowledge-oriented rule. Comparative results obtained for a data-mining task were conducted, in order to assess the performance of adaptive migration according to efficiency/effectiveness criteria.

Towards the Early Diagnosis of Alzheimer's Disease via a Multicriteria Classification Model

The very early detection of Alzheimers disease (AD) has been deeply investigated in numerous studies in the past years. These studies have demonstrated that the pathology usually arises decades before the clinical diagnosis is effectively made, and so a reliable identification of AD in its earliest stages is one of the major challenges clinicians and researchers face nowadays. In the present study, we introduce a new approach developed upon a specific Multicriteria Decision Aid (MCDA) classification method to assist in the early AD diagnosis process. The MCDA method is centered on the concept of prototypes, that is, alternatives that serve as class representatives related to a given problem, and has its performance index very dependent upon the choice of values of some control parameters. In such regard, two techniques, one based on ELECTRE IV methodology and the other on a customized genetic algorithm, are employed in order to select the prototypes and calibrate the control parameters automatically. Moreover, a new database has been designed taking as reference both the functional and cognitive recommendations of the Scientific Department of Cognitive Neurology and Aging of the Brazilian Academy of Neurology and a neuropsychological battery of exams made available by the well-known Consortium to Establish a Registry for Alzheimers Disease (CERAD). Various experiments have been performed over this database in a manner as to either fine-tune the components of the MCDA model or to compare its performance level with that exhibited by other state-of-the-art classification algorithms.

Letters: On the evolutionary design of heterogeneous Bagging models

Bagging is a popular ensemble algorithm based on the idea of data resampling. In this paper, aiming at increasing the incurred levels of ensemble diversity, we present an evolutionary approach for optimally designing Bagging models composed of heterogeneous components. To assess its potentials, experiments with well-known learning algorithms and classification datasets are discussed whereby the accuracy, generalization and diversity levels achieved with heterogeneous Bagging are matched against those delivered by standard Bagging with homogeneous components.

A bio-inspired crime simulation model

In this paper we describe a multiagent crime simulation model that resorts to concepts of self-organizing bio-inspired systems, in particular, of the Ant Colony Optimization algorithm. As the matching between simulated and real crime data distributions depends upon the tuning of some control parameters of the simulation model (in particular, of the initial places where criminals start out), we have modeled the calibration of the simulation as an optimization problem. The solution for the allocation of criminals into gateways is also undertaken by a bio-inspired method, namely, a customized Genetic Algorithm. We show that this approach allows for the automatic discovery of gateway configurations that, when employed in the simulation, produce crime distributions that are statistically close to those observed in real data.

Tackling the container loading problem: a hybrid approach based on integer linear programming and genetic algorithms

Substituted pyridine methyl esters of tetramethylcyclopropane carboxylic acids and the optical isomers thereof are prepared which correspond to the formula: (Formula I) wherein n represents an integer of 0 to 2; X independently represents alkyl of 1 to 4 carbon atoms, alkoxy of 1 to 4 carbon atoms, alkylthio of 1 to 4 carbon atoms, alkylsulfinyl of 1 to 4 carbon atoms, alkylsulfonyl of 1 to 4 carbon atoms, trifluoromethyl, nitro, cyano, chloro, fluoro, bromo, or 3,4-methylenedioxy (when n is 2 and both Xs are taken together); and R represents hydrogen, cyano or ethynyl. These compounds have been found to exhibit a high degree of insecticidal activity and compositions containing said compounds are so employed.