Tien Thanh Nguyen

Combining Multi Classifiers based on A Genetic Algorithm: A Gaussian Mixture Model framework

Combining outputs from different classifiers to achieve high accuracy in classification task is one of the most active research areas in ensemble method. Although many state-of-art approaches have been introduced, no one method performs the best on all data sources. With the aim of introducing an effective classification model, we propose a Gaussian Mixture Model (GMM) based method that combines outputs of base classifiers (called meta-data or Level1 data) resulted from Stacking Algorithm. We further apply Genetic Algorithm (GA) to that data as a feature selection strategy to explore an optimal subset of Level1 data in which our GMM-based approach can achieve high accuracy. Experiments on 21 UCI Machine Learning Repository data files and CLEF2009 medical image database demonstrate the advantage of our framework compared with other well-known combining algorithms such as Decision Template, Multiple Response Linear Regression (MLR), SCANN and fixed combining rules as well as GMM-based approaches on original data.

A novel genetic algorithm approach for simultaneous feature and classifier selection in multi classifier system

In this paper we introduce a novel approach for classifier and feature selection in a multi-classifier system using Genetic Algorithm (GA). Specifically, we propose a 2-part structure for each chromosome in which the first part is encoding for classifier and the second part is encoding for feature. Our structure is simple in the implementation of the crossover as well as the mutation stage of GA. We also study 8 different fitness functions for our GA based algorithm to explore the optimal fitness functions for our model. Experiments are conducted on both 14 UCI Machine Learning Repository and CLEF2009 medical image database to demonstrate the benefit of our model on reducing classification error rate.

A Novel Online Bayes Classifier

We present VIGO, a novel online Bayesian classifier for both binary or multiclass problems. In our model, variational inference for multivariate Gaussian distribution technique is exploited to approximate the class conditional probability density functions of data in an online manner. Besides being a conservative learner with a low number of updates compared with many other popular algorithms, VIGO algorithm can be updated in a minibatch of an arbitrary size which makes it robust with noise data. Experiments over a large number of UCI datasets demonstrate the advantage of VIGO with many state-of-the-art methods presented in LIBOL - a prevalent library for online learning algorithms.

Fuzzy If-Then Rules Classifier on Ensemble Data

This paper introduces a novel framework that uses fuzzy IF-THEN rules in an ensemble system. Our model tackles several drawbacks. First, IF-THEN rules approaches have problems with high dimensional data since computational cost is exponential. In our framework, rules are operated on outputs of base classifiers which frequently have lower dimensionality than the original data. Moreover, outputs of base classifiers are scaled within the range [0, 1] so it is convenient to apply fuzzy rules directly instead of requiring data transformation and normalization before generating fuzzy rules. The performance of this model was evaluated through experiments on 6 commonly used datasets from UCI Machine Learning Repository and compared with several state-of-art combining classifiers algorithms and fuzzy IF-THEN rules approaches. The results show that our framework can improve the classification accuracy.

Optimization of ensemble classifier system based on multiple objectives genetic algorithm

This paper introduces a mechanism to learn optimal classifier combining algorithms for an ensemble system. By using a genetic algorithm approach that focuses on 3 objectives namely the number of correct classified observations, the number of selected features and the number of selected classifiers, optimal solution can be achieved after several interactions of crossover and mutation. We also employ the Ordered Weighted Averaging operator in which a weight vector is built by a Linear Decreasing (LD) function to find average values of outputs from combining algorithms. Experiments on 2 well-known UCI Machine Learning Repository datasets demonstrate benefits of our approach compared with other state-of-the-art ensemble methods like Decision Template, SCANN and all fixed combining algorithms in the ensemble system.

A Novel 2-Stage Combining Classifier Model with Stacking and Genetic Algorithm Based Feature Selection

This paper introduces a novel 2-stage classification system with stacking and genetic algorithm (GA) based feature selection. Specifically, Level1 data is first generated by stacking on the original data (called Level0 data) with base classifiers. Level1data is then classified by a second classifier (denoted by C) with feature selection using GA. The advantage of applying GA on Level1 data is that it has lower dimension and is more uniformity than Level0 data. We conduct experiments on both 18 UCI data files and CLEF2009 medical image database to demonstrate superior performance of our model in comparison with several popular combining algorithms.

Heterogeneous classifier ensemble with fuzzy rule-based meta learner

In heterogeneous ensemble systems, each learning algorithm learns a classifier on a given training set to describe the relationship between a feature vector and its class label. As each classifier outputs different result on an observation, uncertainty is introduced. In this paper, we introduce a heterogeneous ensemble system with a fuzzy IF-THEN rule inference engine as the combiner to capture the uncertainty in the outputs of the base classifiers. In our method, fuzzy rules are generated on the outputs of an ensemble of base classifiers, which can be viewed as the class posterior probability of the observations. The performance of our method was evaluated on thirty datasets and in comparison with nine ensemble methods (AdaBoost, Decision Template, Decision Tree on meta-data, and six fixed combiners) and two single learning algorithms (SVM with L2-loss function and Decision Tree), and was shown to significantly outperforms these algorithms in terms of classification accuracy.

Learning from Data Stream Based on Random Projection and Hoeffding Tree Classifier

In this study, we introduce an ensemble-based approach for online machine learning. Here, instead of working on the original data, several Hoeffding tree classifiers classify and are updated on the lower dimensional projected data generated from originality by random projections. Since random projection is unstable, from one example, many diverse training data can be created to train the set of Hoeffding tree classifiers. The experiments conducted on a number of datasets chosen from different sources demonstrate that the proposed approach performs significantly better than the single Hoeffding tree and some well-known online learning algorithms including additive models and Online Bagging.

Fusion of Classifiers Based on a Novel 2-Stage Model

The paper introduces a novel 2-Stage model for multi-classifier system. Instead of gathering posterior probabilities resulted from base classifiers into a single dataset called meta-data or Level1 data like in the original 2-Stage model, here we separate data in K Level1 matrices corresponding to the K base classifiers. These data matrices, in turn, are classified in sequence by a new classifier at the second stage to generate output of that new classifier called Level2 data. Next, Weight Matrix algorithm is proposed to combine Level2 data and produces prediction for unlabeled observations. Experimental results on CLEF2009 medical image database demonstrate the benefit of our model in comparison with several existing ensemble learning models.

A Hough Transform Based Biclustering Algorithm for Gene Expression Data

In pattern classification, when the feature space is of high dimensionality or patterns are “similar” on a subset of features only, the traditional clustering methods do not show good performance. Biclustering is a class of methods that simultaneously carry out grouping on two dimensions and has many applications to different fields, especially gene expression data analysis. Because of simultaneous classification on both rows and columns of a data matrix, the biclustering problem is inherently intractable and computationally complex. One of the most complex models in biclustering problem is linear coherent model. Several biclustering algorithms based on this model have been proposed in recent years. However, none of them is able to perfectly recognize all linear patterns in a bicluster. In this work, we propose a novel algorithm based on Hough transform that can find all linear coherent patterns. In the sequel we apply it to gene expression data.