Taoufik Bdiri

Positive vectors clustering using inverted Dirichlet finite mixture models

In this work we present an unsupervised algorithm for learning finite mixture models from multivariate positive data. Indeed, this kind of data appears naturally in many applications, yet it has not been adequately addressed in the past. This mixture model is based on the inverted Dirichlet distribution, which offers a good representation and modeling of positive non-Gaussian data. The proposed approach for estimating the parameters of an inverted Dirichlet mixture is based on the maximum likelihood (ML) using Newton Raphson method. We also develop an approach, based on the minimum message length (MML) criterion, to select the optimal number of clusters to represent the data using such a mixture. Experimental results are presented using artificial histograms and real data sets. The challenging problem of software modules classification is investigated within the proposed statistical framework, also.

Learning inverted dirichlet mixtures for positive data clustering

In this paper, we propose a statistical model to cluster positive data. The proposed model adopts a mixture of inverted Dirichlet distributions and is learned using expectation-maximization (EM) for parameters estimation and the minimum message length criterion (MML) for model selection. Experimental results using both synthetic and real data are presented to show the advantages of the proposed model.

Bayesian learning of inverted Dirichlet mixtures for SVM kernels generation

We describe approaches for positive data modeling and classification using both finite inverted Dirichlet mixture models and support vector machines (SVMs). Inverted Dirichlet mixture models are used to tackle an outstanding challenge in SVMs namely the generation of accurate kernels. The kernels generation approaches, grounded on ideas from information theory that we consider, allow the incorporation of data structure and its structural constraints. Inverted Dirichlet mixture models are learned within a principled Bayesian framework using both Gibbs sampler and Metropolis-Hastings for parameter estimation and Bayes factor for model selection (i.e., determining the number of mixture’s components). Our Bayesian learning approach uses priors, which we derive by showing that the inverted Dirichlet distribution belongs to the family of exponential distributions, over the model parameters, and then combines these priors with information from the data to build posterior distributions. We illustrate the merits and the effectiveness of the proposed method with two real-world challenging applications namely object detection and visual scenes analysis and classification.

Robust simultaneous positive data clustering and unsupervised feature selection using generalized inverted Dirichlet mixture models

The discovery, extraction and analysis of knowledge from data rely generally upon the use of unsupervised learning methods, in particular clustering approaches. Much recent research in clustering and data engineering has focused on the consideration of finite mixture models which allow to reason in the face of uncertainty and to learn by example. The adoption of these models becomes a challenging task in the presence of outliers and in the case of high-dimensional data which necessitates the deployment of feature selection techniques. In this paper we tackle simultaneously the problems of cluster validation (i.e. model selection), feature selection and outliers rejection when clustering positive data. The proposed statistical framework is based on the generalized inverted Dirichlet distribution that offers a more practical and flexible alternative to the inverted Dirichlet which has a very restrictive covariance structure. The learning of the parameters of the resulting model is based on the minimization of a message length objective incorporating prior knowledge. We use synthetic data and real data generated from challenging applications, namely visual scenes and objects clustering, to demonstrate the feasibility and advantages of the proposed method.

Object clustering and recognition using multi-finite mixtures for semantic classes and hierarchy modeling

Model-based approaches have become important tools to model data and infer knowledge. Such approaches are often used for clustering and object recognition which are crucial steps in many applications, including but not limited to, recommendation systems, search engines, cyber security, surveillance and object tracking. Many of these applications have the urgent need to reduce the semantic gap of data representation between the system level and the human being understandable level. Indeed, the low level features extracted to represent a given object can be confusing to machines which cannot differentiate between very similar objects trivially distinguishable by human beings (e.g. apple vs. tomato). In this paper, we propose a novel hierarchical methodology for data representation using a hierarchical mixture model. The proposed approach allows to model a given object class by a set of modes deduced by the system and grouped according to a labeled training data representing the human level semantic. We have used the inverted Dirichlet distribution to build our statistical framework. The proposed approach has been validated using both synthetic data and a challenging application namely visual object clustering and recognition. The presented model is shown to have a flexible hierarchy that can be changed on the fly within costless computational time.

An infinite mixture of inverted dirichlet distributions

In this paper we present an infinite mixture model based on inverted Dirichlet distributions. The proposed mixture is learned using a fully Bayesian approach and allows to overcome a challenging issue when dealing with data clustering namely the automatic selection of the number of clusters. We explore the performance of the proposed approach on the challenging problem of text categorization. The results show that the proposed approach is effective for positive data modeling when compared to those reported using infinite Gaussian mixture.

Variational Bayesian inference for infinite generalized inverted Dirichlet mixtures with feature selection and its application to clustering

We developed a variational Bayesian learning framework for the infinite generalized Dirichlet mixture model (i.e. a weighted mixture of Dirichlet process priors based on the generalized inverted Dirichlet distribution) that has proven its capability to model complex multidimensional data. We also integrate a “feature selection” approach to highlight the features that are most informative in order to construct an appropriate model in terms of clustering accuracy. Experiments on synthetic data as well as real data generated from visual scenes and handwritten digits datasets illustrate and validate the proposed approach.

A statistical framework for online learning using adjustable model selection criteria

Model-based approaches have been for long an effective method to model data and classify it. Recently they have been used to model users interactions with a given system in order to satisfy their needs through adequate responses. The semantic gap between the system and the user perception for the data makes this modeling hard to be designed based on the features space only. Indeed the user intervention is somehow needed to inform the system how the data should be perceived according to some ontology and hierarchy when new data are introduced to the model. Such a task is challenging as the system should learn how to establish the update according to the user perception and representation of the data. In this work, we propose a new methodology to update a mixture model based on the generalized inverted Dirichlet distribution, that takes into account simultaneously users perception and the dynamic nature of real-world data. Experiments on synthetic data as well as real data generated from a challenging application namely visual objects classification indicate that the proposed approach has merits and provides promising results. HighlightsAn online learning framework for generalized inverted Dirichlet mixtures is proposed.The proposed statistical model takes into account simultaneously users perception and the dynamic nature of real-world data.The model is applied to the challenging problem of visual objects classification.

Impact of Varying Node Velocity and HELLO Interval Duration on Position-based Stable Routing in Mobile Ad Hoc Networks☆

Abstract Wireless networks have evolved considerably in the recent years thanks to the advancement of technology that has made devices more portable, smarter, and more energy efficient. In particular, Mobile Ad hoc Networks (MANETs), that are formed without any centralized infrastructure, have received a lot of attention as they can be used in many real life applications. Yet, compared to static wireless networks, less academic research has been done on MANETs, especially when all the nodes are in continuous movement. In particular, we consider MANETs that broadcast HELLO messages at regular time intervals in order to maintain dynamic neigh- borhood information. The range of velocities of the nodes and the HELLO message interval duration can significantly affect the performance of routing protocols in MANETs. In this work, we study the effect of varying these two main characteristics on the performance of MANETs in terms of delivered packets and packets delivery ratio that reflect routing paths stability. We present a comprehensive experimental analysis of the effect of such variations on three position-based stability-oriented routing protocols, namely, Greedy-based Backup Routing (GBR), LEARN-based Backup Routing (LBR), and GBR combined with a Conservative Neighborhood Range (GBR-CNR).

A Statistical Framework for Mental Targets Search Using Mixture Models

Image retrieval is usually based on specific user needs that are expressed under the form of explicit queries that lead to retrieve target images. In many cases, a given user does not possess the adequate tools and semantics to express what he/she is looking for, thus, his/her target image resides in his/her mind while he/she can visually identify it. We propose in this work, a statistical framework that enables users to start a search process and interact with the system in order to find their target “mental image”, using visual features only. Our bayesian formulation provides the possibility of searching multi target classes within the same search process. Data are modeled by a generalized inverted Dirichlet mixture that also serves to quantify the similarities between images. We run experiments including real users and we present a case study of a search process that gives promising results in terms of number of iterations needed to find the mental target classes within a given dataset.