Hichem Frigui

A robust competitive clustering algorithm with applications in computer vision

This paper addresses three major issues associated with conventional partitional clustering, namely, sensitivity to initialization, difficulty in determining the number of clusters, and sensitivity to noise and outliers. The proposed robust competitive agglomeration (RCA) algorithm starts with a large number of clusters to reduce the sensitivity to initialization, and determines the actual number of clusters by a process of competitive agglomeration. Noise immunity is achieved by incorporating concepts from robust statistics into the algorithm. RCA assigns two different sets of weights for each data point: the first set of constrained weights represents degrees of sharing, and is used to create a competitive environment and to generate a fuzzy partition of the data set. The second set corresponds to robust weights, and is used to obtain robust estimates of the cluster prototypes. By choosing an appropriate distance measure in the objective function, RCA can be used to find an unknown number of clusters of various shapes in noisy data sets, as well as to fit an unknown number of parametric models simultaneously. Several examples, such as clustering/mixture decomposition, line/plane fitting, segmentation of range images, and estimation of motion parameters of multiple objects, are shown.

Unsupervised learning of prototypes and attribute weights

Abstract In this paper, we introduce new algorithms that perform clustering and feature weighting simultaneously and in an unsupervised manner. The proposed algorithms are computationally and implementationally simple, and learn a different set of feature weights for each identified cluster. The cluster dependent feature weights offer two advantages. First, they guide the clustering process to partition the data set into more meaningful clusters. Second, they can be used in the subsequent steps of a learning system to improve its learning behavior. An extension of the algorithm to deal with an unknown number of clusters is also proposed. The extension is based on competitive agglomeration, whereby the number of clusters is over-specified, and adjacent clusters are allowed to compete for data points in a manner that causes clusters which lose in the competition to gradually become depleted and vanish. We illustrate the performance of the proposed approach by using it to segment color images, and to build a nearest prototype classifier.

Self-organization of pulse-coupled oscillators with application to clustering

We introduce an efficient synchronization model that organizes a population of integrate-and-fire oscillators into stable and structured groups. Each oscillator fires synchronously with all the others within its group, but the groups themselves fire with a constant phase difference. The structure of the synchronized groups depends on the choice of the coupling function. We show that by defining the interaction between oscillators according to the relative distance between them, our model can be used as a general clustering algorithm. Unlike existing models, our model incorporates techniques from relational and prototype-based clustering methods and results in a clustering algorithm that is simple, efficient, robust, unbiased by the size of the clusters, and that can find an arbitrary number of clusters. In addition to helping the model self-organize into stable groups, the synergy between clustering and synchronization reduces the computational complexity significantly. The resulting clustering algorithm has several advantages over conventional clustering techniques. In particular, it can generate a nested sequence of partitions and it can determine the optimum number of clusters in an efficient manner. Moreover, since our approach does not involve optimizing an objective function, it is not sensitive to initialization and it can incorporate nonmetric similarity measures. We illustrate the performance of our algorithms with several synthetic and real data sets.

Simultaneous clustering and attribute discrimination

We propose a new algorithm that performs clustering and feature weighting simultaneously and in an unsupervised manner. The proposed algorithm is computationally and implementationally simple, and learns a different set of feature weights for each cluster. The cluster dependent feature weights have two advantages. First, they help in partitioning the data set into more meaningful clusters. Second, they can be used as part of a more complex learning system to enhance its learning behavior. The performance of the proposed algorithm is illustrated by using it to segment real color images.

Simultaneous categorization of text documents and identification of cluster-dependent keywords

We propose an approach to clustering text documents based on a coupled process of clustering and cluster-dependent keyword weighting. The proposed approach is based on the the fuzzy c-means clustering algorithm. Hence it is computationally and implementationally simple. Moreover, it learns a different set of keyword weights for each cluster. This means that, as a by-product of the clustering process, each document cluster will be characterized by a possibly different set of keywords. The cluster dependent keyword weights help in partitioning the document collection into more meaningful categories. They can also be used to automatically generate a brief summary of each cluster in terms of not only the attribute values, but also their relevance. For the case of text data, this approach can be used to automatically annotate the documents. We illustrate the performance of the proposed algorithm by using it to cluster a real collection of text documents.

Interactive image retrieval using fuzzy sets

Abstract We present an image retrieval system which permits the user to submit a coarse initial query and continuously refine it. The users relevance feedbacks is modeled by fuzzy sets, and is used to discover and use the more discriminatory features for the given query. The proposed system uses a dissimilarity measure based on the fuzzy integral.

SyMP: an efficient clustering approach to identify clusters of arbitrary shapes in large data sets

We propose a new clustering algorithm, called SyMP, which is based on synchronization of pulse-coupled oscillators. SyMP represents each data point by an Integrate-and-Fire oscillator and uses the relative similarity between the points to model the interaction between the oscillators. SyMP is robust to noise and outliers, determines the number of clusters in an unsupervised manner, identifies clusters of arbitrary shapes, and can handle very large data sets. The robustness of SyMP is an intrinsic property of the synchronization mechanism. To determine the optimum number of clusters, SyMP uses a dynamic resolution parameter. To identify clusters of various shapes, SyMP models each cluster by multiple Gaussian components. The number of components is automatically determined using a dynamic intra-cluster resolution parameter. Clusters with simple shapes would be modeled by few components while clusters with more complex shapes would require a larger number of components. The scalable version of SyMP uses an efficient incremental approach that requires a simple pass through the data set. The proposed clustering approach is empirically evaluated with several synthetic and real data sets, and its performance is compared with CURE.

A synchronization based algorithm for discovering ellipsoidal clusters in large datasets

This paper introduces a new scalable approach to clustering based on the synchronization of pulse-coupled oscillators. Each data point is represented by an integrate-and-fire oscillator and the interaction between oscillators is defined according to the relative similarity between the points. The set of oscillators self-organizes into stable phase-locked subgroups. Our approach proceeds by loading only a subset of the data and allowing it to self-organize. Groups of synchronized oscillators are then summarized and purged from memory. We show that our method is robust, scales linearly and can determine the number of clusters. The proposed approach is empirically evaluated with several synthetic data sets and is used to segment large color images.

Landmine detection with ground penetrating radar using fuzzy k-nearest neighbors

This paper introduces a system for landmine detection using the sensor data generated by a ground penetrating radar (GPR). The GPR produces a three-dimensional array of intensity values, representing a volume below the surface of the ground. First, a constant false alarm rate (CFAR) detector is used to focus the attention and identify the candidates that resemble mines. Next, we apply a feature extraction algorithm based on projecting the data onto the dominant eigenvectors in the training data. The training signatures are then clustered to identify a few representatives, and a fuzzy k-nearest neighbor rule is used to distinguish true detections from false alarms.

Detection of land mines using fuzzy and possibilistic membership functions

This paper introduces a new system for real-time land mine detection using sensor data generated by a Ground Penetrating Radar (GPR). The GPR produces a three-dimensional array of intensity values, representing a volume below the surface of the ground. Features are computed from this array and two types of membership degrees are assigned to each location. A fuzzy membership value provides a degree of belongingness of a given observation in the classes of mines, false alarms, and background, while a possibilistic membership value provides a degree of typicality. Both membership degrees are combined using simple rules to assign a confidence value. The parameters of the membership functions are obtained by clustering the training data and using the statistics of each partition. Our preliminary results show that the proposed approach is simple, efficient, and yet, yields results comparable to more complex detection systems.