Clodoaldo Ap. M. Lima

Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification

Mixture of experts (ME) models comprise a family of modular neural network architectures aiming at distilling complex problems into simple subtasks. This is done by deploying a separate gating module for softly dividing the input space into overlapping regions to be each assigned to one or more expert networks. Conversely, support vector machines (SVMs) refer to kernel-based methods, neural-network-alike models that constitute an approximate implementation of the structural risk minimization principle. Such learning machines follow the simple, but powerful idea of nonlinearly mapping input data into high-dimensional feature spaces wherein a linear decision surface discriminating different regions is properly designed. In this work, we formally characterize and empirically evaluate a novel approach, named as Mixture of Support Vector Machine Experts (MSVME), whose main purpose is to combine the complementary properties of both SVM and ME models. In the formal characterization, an algorithm based on a maximum likelihood criterion is considered for the MSVME training, and we demonstrate that it is possible to train each expert based on an SVM perspective. Regarding the empirical evaluation, simulation results involving nonlinear dynamic system identification problems are reported, contrasting the performance shown by the MSVME approach with that exhibited by conventional SVM and ME models.

Automatic EEG signal classification for epilepsy diagnosis with Relevance Vector Machines

In this paper, we investigate the potentials of applying a kernel-based learning machine, the Relevance Vector Machine (RVM), to the task of epilepsy detection through automatic electroencephalogram (EEG) signal classification. For this purpose, some experiments have been conducted over publicly available data, contrasting the performance levels exhibited by RVM models with those achieved with Support Vector Machines (SVMs), both in terms of predictive accuracy and sensitivity to the choice of the kernel function. Four settings of both types of kernel machine were considered in this study, which vary in accord with the type of input data they receive, either raw EEG signal or some statistical features extracted from the wavelet-transformed data. The empirical results indicate that: (1) in terms of accuracy, the best-calibrated RVM models have shown very satisfactory performance levels, which are rather comparable to those of SVMs; (2) an increase of accuracy is sometimes accompanied by loss of sparseness in the resulting RVM models; (3) both types of machines present similar sensitivity profiles to the kernel functions considered, having some kernel parameter values clearly associated with better accuracy rate; (4) when not making use of a feature extraction technique, the choice of the kernel function seems to be very relevant for significantly leveraging the performance of RVMs; and (5) when making use of derived features, the choice of the feature extraction technique seems to be an important factor to one take into account.

EEG signal classification for epilepsy diagnosis via optimum path forest - A systematic assessment

Abstract Epilepsy refers to a set of chronic neurological syndromes characterized by transient and unexpected electrical disturbances of the brain. The detailed analysis of the electroencephalogram (EEG) is one of the most influential steps for the proper diagnosis of this disorder. This work presents a systematic performance evaluation of the recently introduced optimum path forest (OPF) classifier when coping with the task of epilepsy diagnosis directly through EEG signal analysis. For this purpose, we have made extensive use of a benchmark dataset composed of five classes, whose full discrimination is very hard to achieve. Four types of wavelet functions and three well-known filter methods were considered for the tasks of feature extraction and selection, respectively. Moreover, support vector machines configured with radial basis function (SVM-RBF) kernel, multilayer perceptron neural networks (ANN-MLP), and Bayesian classifiers were used for comparison in terms of effectiveness and efficiency. Overall, the results evidence the outperformance of the OPF classifier in both types of criteria. Indeed, the OPF classifier was usually extremely fast, with average training/testing times much lower than those required by SVM-RBF and ANN-MLP. Moreover, when configured with Coiflets as feature extractors, the performance scores achieved by the OPF classifier include 89.2% as average accuracy and sensitivity/specificity values higher than 80% for all five classes.

Kernel machines for epilepsy diagnosis via EEG signal classification: A comparative study

OBJECTIVE We carry out a systematic assessment on a suite of kernel-based learning machines while coping with the task of epilepsy diagnosis through automatic electroencephalogram (EEG) signal classification. METHODS AND MATERIALS The kernel machines investigated include the standard support vector machine (SVM), the least squares SVM, the Lagrangian SVM, the smooth SVM, the proximal SVM, and the relevance vector machine. An extensive series of experiments was conducted on publicly available data, whose clinical EEG recordings were obtained from five normal subjects and five epileptic patients. The performance levels delivered by the different kernel machines are contrasted in terms of the criteria of predictive accuracy, sensitivity to the kernel function/parameter value, and sensitivity to the type of features extracted from the signal. For this purpose, 26 values for the kernel parameter (radius) of two well-known kernel functions (namely, Gaussian and exponential radial basis functions) were considered as well as 21 types of features extracted from the EEG signal, including statistical values derived from the discrete wavelet transform, Lyapunov exponents, and combinations thereof. RESULTS We first quantitatively assess the impact of the choice of the wavelet basis on the quality of the features extracted. Four wavelet basis functions were considered in this study. Then, we provide the average accuracy (i.e., cross-validation error) values delivered by 252 kernel machine configurations; in particular, 40%/35% of the best-calibrated models of the standard and least squares SVMs reached 100% accuracy rate for the two kernel functions considered. Moreover, we show the sensitivity profiles exhibited by a large sample of the configurations whereby one can visually inspect their levels of sensitiveness to the type of feature and to the kernel function/parameter value. CONCLUSIONS Overall, the results evidence that all kernel machines are competitive in terms of accuracy, with the standard and least squares SVMs prevailing more consistently. Moreover, the choice of the kernel function and parameter value as well as the choice of the feature extractor are critical decisions to be taken, albeit the choice of the wavelet family seems not to be so relevant. Also, the statistical values calculated over the Lyapunov exponents were good sources of signal representation, but not as informative as their wavelet counterparts. Finally, a typical sensitivity profile has emerged among all types of machines, involving some regions of stability separated by zones of sharp variation, with some kernel parameter values clearly associated with better accuracy rates (zones of optimality).

Tackling EEG signal classification with least squares support vector machines: A sensitivity analysis study

The electroencephalogram (EEG) signal captures the electrical activity of the brain and is an important source of information for studying neurological disorders. The proper analysis of this biological signal plays an important role in the domain of brain-computer interface, which aims at the construction of communication channels between human brain and computers. In this paper, we investigate the application of least squares support vector machines (LS-SVM) to the task of epilepsy diagnosis through automatic EEG signal classification. More specifically, we present a sensitivity analysis study by means of which the performance levels exhibited by standard and least squares SVM classifiers are contrasted, taking into account the setting of the kernel function and of its parameter value. Results of experiments conducted over different types of features extracted from a benchmark EEG signal dataset evidence that the sensitivity profiles of the kernel machines are qualitatively similar, both showing notable performance in terms of accuracy and generalization. In addition, the performance accomplished by optimally configured LS-SVM models is also quantitatively contrasted with that obtained by related approaches for the same dataset.

Automatic recognition of epileptic seizure in EEG via support vector machine and dimension fractal

Support vector machine (SVM) is a machine learning technique widely applied in classification problems. SVM are based on the Vapniks Statistical Learning Theory, and successively extended by a number of researchers. On the order hand, the electroencephalogram (EEG) signal captures the electrical activity of the brain and is an important source of information for studying neurological disorders. In order to extract relevant information of EEG signal, a variety of computerized-analysis methods have been developed. Recent studies indicate that methods based on the nonlinear dynamics theory can extract valuable information from neuronal dynamics. However, many these of methods need large amount of data and are computationally expensive. From chaos theory, a global value that is relatively simple to compute is the fractal dimension (FD), it can be used to measure the geometrical complexity of a time series. The FD of a waveform represents a powerful tool for transient detection. In analysis of EEG this feature can been used to identify and distinguish specific states of physiologic function. A variety of algorithms are available for the computation of FD. In this work, we employ SVM to classify the EEG signals from healthy subjects and epileptic subjects using as the features vector the FD. From the experimental results, we can see that classification based on SVM with FD perform well in EEG signals classification, which indicates this classification method is valid and has promising application.

Pattern classification with mixtures of weighted least-squares support vector machine experts

Support Vector Machine (SVM) classifiers are high-performance classification models devised to comply with the structural risk minimization principle and to properly exploit the kernel artifice of nonlinearly mapping input data into high-dimensional feature spaces toward the automatic construction of better discriminating linear decision boundaries. Among several SVM variants, Least-Squares SVMs (LS-SVMs) have gained increased attention recently due mainly to their computationally attractive properties coming as the direct result of applying a modified formulation that makes use of a sum-squared-error cost function jointly with equality, instead of inequality, constraints. In this work, we present a flexible hybrid approach aimed at augmenting the proficiency of LS-SVM classifiers with regard to accuracy/generalization as well as to hyperparameter calibration issues. Such approach, named as Mixtures of Weighted Least-Squares Support Vector Machine Experts, centers around the fusion of the weighted variant of LS-SVMs with Mixtures of Experts models. After the formal characterization of the novel learning framework, simulation results obtained with respect to both binary and multiclass pattern classification problems are reported, ratifying the suitability of the novel hybrid approach in improving the performance issues considered.

Assessing fractal dimension methods as feature extractors for EMG signal classification

The study of electromyographic (EMG) signals has gained increased attention in the last decades since the proper analysis and processing of these signals can be instrumental for the diagnosis of neuromuscular diseases and the adaptive control of prosthetic devices. As a consequence, various pattern recognition approaches, consisting of different modules for feature extraction and classification of EMG signals, have been proposed. In this paper, we conduct a systematic empirical study on the use of Fractal Dimension (FD) estimation methods as feature extractors from EMG signals. The usage of FD as feature extraction mechanism is justified by the fact that EMG signals usually show traces of self-similarity and by the ability of FD to characterize and measure the complexity inherent to different types of muscle contraction. In total, eight different methods for calculating the FD of an EMG waveform are considered here, and their performance as feature extractors is comparatively assessed taking into account nine well-known classifiers of different types and complexities. Results of experiments conducted on a dataset involving seven distinct types of limb motions are reported whereby we could observe that the normalized version of the Katzs estimation method and the Hurst exponent significantly outperform the others according to a class separability measure and five well-known accuracy measures calculated over the induced classifiers.

Classification of electromyography signals using relevance vector machines and fractal dimension

Abstract Surface electromyography (EMG) signals have been studied extensively in the last years aiming at the automatic classification of hand gestures and movements as well as the early identification of latent neuromuscular disorders. In this paper, we investigate the potentials of the conjoint use of relevance vector machines (RVM) and fractal dimension (FD) for automatically identifying EMG signals related to different classes of limb motion. The adoption of FD as the mechanism for feature extraction is justified by the fact that EMG signals usually show traces of self-similarity. In particular, four well-known FD estimation methods, namely box-counting, Higuchi’s, Katz’s and Sevcik’s methods, have been considered in this study. With respect to RVM, besides the standard formulation for binary classification, we also investigate the performance of two recently proposed variants, namely constructive mRVM and top-down mRVM, that deal specifically with multiclass problems. These classifiers operate solely over the features extracted by the FD estimation methods, and since the number of such features is relatively small, the efficiency of the classifier induction process is ensured. Results of experiments conducted on a publicly available dataset involving seven distinct types of limb motions are reported whereby we assess the performance of different configurations of the proposed RVM+FD approach. Overall, the results evidence that kernel machines equipped with the FD feature values can be useful for achieving good levels of classification performance. In particular, we have empirically observed that the features extracted by the Katz’s method is of better quality than the features generated by other methods.

Estimation of chaotic signals with applications in communications

Abstract In this paper we extend an estimation technique based on Viterbi algorithm for discrete-time chaotic signals immersed in noise. The proposed modification allows for orbits generated by maps with nonuniform invariant density to be estimated. This modified Viterbi algorithm is used in two digital modulation schemes: the Modified Maximum Likelihood Chaos Shift Keying using one and two maps. Both have better symbol error rate characteristics than non-coherent chaos communication schemes.