Khalid Benabdeslem

Constrained laplacian score for semi-supervised feature selection

In this paper, we address the problem of semi-supervised feature selection from high-dimensional data. It aims to select the most discriminative and informative features for data analysis. This is a recent addressed challenge in feature selection research when dealing with small labeled data sampled with large unlabeled data in the same set. We present a filter based approach by constraining the known Laplacian score. We evaluate the relevance of a feature according to its locality preserving and constraints preserving ability. The problem is then presented in the spectral graph theory framework with a study of the complexity of the proposed algorithm. Finally, experimental results will be provided for validating our proposal in comparison with other known feature selection methods.

Efficient Semi-Supervised Feature Selection: Constraint, Relevance, and Redundancy

This paper describes a three-level framework for semi-supervised feature selection. Most feature selection methods mainly focus on finding relevant features for optimizing high-dimensional data. In this paper, we show that the relevance requires two important procedures to provide an efficient feature selection in the semi-supervised context. The first one concerns the selection of pairwise constraints that can be extracted from the labeled part of data. The second procedure aims to reduce the redundancy that could be detected in the selected relevant features. For the relevance, we develop a filter approach based on a constrained Laplacian score. Finally, experimental results are provided to show the efficiency of our proposal in comparison with several representative methods.

Constraint Selection-Based Semi-supervised Feature Selection

In this paper, we present a novel feature selection approach based on an efficient selection of pair wise constraints. This aims at selecting the most coherent constraints extracted from labeled part of data. The relevance of features is then evaluated according to their efficient locality preserving and chosen constraint preserving ability. Finally, experimental results are provided for validating our proposal with respect to other known feature selection methods.

Bi-clustering continuous data with self-organizing map

In this paper, we present a new SOM-based bi-clustering approach for continuous data. This approach is called Bi-SOM (for Bi-clustering based on Self-Organizing Map). The main goal of bi-clustering aims to simultaneously group the rows and columns of a given data matrix. In addition, we propose in this work to deal with some issues related to this task: (1) the topological visualization of bi-clusters with respect to their neighborhood relation, (2) the optimization of these bi-clusters in macro-blocks and (3) the dimensionality reduction by eliminating noise blocks, iteratively. Finally, experiments are given over several data sets for validating our approach in comparison with other bi-clustering methods.

Soft-constrained Laplacian score for semi-supervised multi-label feature selection

Feature selection, semi-supervised learning and multi-label classification are different challenges for machine learning and data mining communities. While other works have addressed each of these problems separately, in this paper we show how they can be addressed together. We propose a unified framework for semi-supervised multi-label feature selection, based on Laplacian score. In particular, we show how to constrain the function of this score, when data are partially labeled and each instance is associated with a set of labels. We transform the labeled part of data into soft constraints and show how to integrate them in a measure of feature relevance, according to the available labels. Experiments on benchmark data sets are provided for validating the proposed approach and comparing it with some other state-of-the-art feature selection methods in a multi-label context.

Constraint selection for semi-supervised topological clustering

In this paper, we propose to adapt the batch version of selforganizing map (SOM) to background information in clustering task. It deals with constrained clustering with SOM in a deterministic paradigm. In this context we adapt the appropriate topological clustering to pairwise instance level constraints with the study of their informativeness and coherence properties for measuring their utility for the semi-supervised learning process. These measures will provide guidance in selecting the most useful constraint sets for the proposed algorithm. Experiments will be given over several databases for validating our approach in comparison with another constrained clustering ones.

Local-to-global semi-supervised feature selection

Variable-weighting approaches are well-known in the context of embedded feature selection. Generally, this task is performed in a global way, when the algorithm selects a single cluster-independent subset of features (global feature selection). However, there exist other approaches that aim to select cluster-specific subsets of features (local feature selection). Global and local feature selection have different objectives, nevertheless, in this paper we propose a novel embedded approach which locally weights the variables towards a global feature selection. The proposed approach is presented in the semi-supervised paradigm. Experiments on some known data sets are presented to validate our model and compare it with some representative methods.

Visualization and clustering of categorical data with probabilistic self-organizing map

This paper introduces a self-organizing map dedicated to clustering, analysis and visualization of categorical data. Usually, when dealing with categorical data, topological maps use an encoding stage: categorical data are changed into numerical vectors and traditional numerical algorithms (SOM) are run. In the present paper, we propose a novel probabilistic formalism of Kohonen map dedicated to categorical data where neurons are represented by probability tables. We do not need to use any coding to encode variables. We evaluate the effectiveness of our model in four examples using real data. Our experiments show that our model provides a good quality of results when dealing with categorical data.

Constrained graph b-coloring based clustering approach

Clustering is generally defined as an unsupervised data mining process which aims to divide a set of data into groups, or clusters, such that the data within the same group are similar to each other while data from different groups are dissimilar. However, additional background information (namely constraints) are available in some domains and must be considered in the clustering solutions. Recently, we have developed a new graph b-coloring clustering algorithm. It exhibits more important clustering features and enables to build a fine partition of the data set in clusters when the number of clusters is not pre-defined. In this paper, we propose an extension of this method to incorporate two types of Instance-Level clustering constraints (must-link and cannot-link constraints). In experiments with artificial constraints on benchmark data sets, we show improvements in the quality of the clustering solution and the computational complexity of the algorithm.

Ensemble constrained Laplacian score for efficient and robust semi-supervised feature selection

In this paper, we propose an efficient and robust approach for semi-supervised feature selection, based on the constrained Laplacian score. The main drawback of this method is the choice of the scant supervision information, represented by pairwise constraints. In fact, constraints are proven to have some noise which may deteriorate learning performance. In this work, we try to override any negative effects of constraint set by the variation of their sources. This is achieved by an ensemble technique using both a resampling of data (bagging) and a random subspace strategy. Experiments on high-dimensional datasets are provided for validating the proposed approach and comparing it with other representative feature selection methods.