Olivier J. J. Michel

Applications of entropic spanning graphs

This article presents applications of entropic spanning graphs to imaging and feature clustering applications. Entropic spanning graphs span a set of feature vectors in such a way that the normalized spanning length of the graph converges to the entropy of the feature distribution as the number of random feature vectors increases. This property makes these graphs naturally suited to applications where entropy and information divergence are used as discriminants: texture classification, feature clustering, image indexing, and image registration. Among other areas, these problems arise in geographical information systems, digital libraries, medical information processing, video indexing, multisensor fusion, and content-based retrieval.

Asymptotic theory of greedy approximations to minimal k-point random graphs

Let /spl chi//sub n/=(x/sub 1/,...,x/sub n/), be an independent and identically distributed (i.i.d.) sample having multivariate distribution P. We derive almost sure (a.s.) limits for the power-weighted edge weight function of greedy approximations to a class of minimal graphs spanning k of the n samples. The class includes minimal k-point graphs constructed by the partitioning method of Ravi, Sundaram, Marathe, Rosenkrantz, and Ravi (see Proc. 5th Annu. ACM-SIAM Symp. Discrete Algorithms, Arlington, VA, p.546-55, 1994), where the edge weight function satisfies the quasi-additive property of Redmond and Yukich (see Ann. Appl. Probab., vol.4, no.4, p.1057-73, 1994). In particular, this includes greedy approximations to the k-point minimal spanning tree (k-MST), Steiner tree (k-ST), and the traveling salesman problem (k-TSP). An expression for the influence function of the minimal-weight function is given which characterizes the asymptotic sensitivity of the graph weight to perturbations in the underlying distribution. The influence function takes a form which indicates that the k-point minimal graph in d>1 dimensions has robustness properties in R/sup d/ which are analogous to those of rank-order statistics in one dimension. A direct result of our theory is that the log-weight of the k-point minimal graph is a consistent nonparametric estimate of the Renyi entropy of the distribution P. Possible applications of this work include: analysis of random communication network topologies, estimation of the mixing coefficient in /spl epsiv/-contaminated mixture models, outlier discrimination and rejection, clustering, and pattern recognition, robust nonparametric regression, two-sample matching, and image registration.

Time-frequency complexity and information

Many functions have been proposed for estimating signal information content and complexity on the time-frequency plane, including moment-based measures such as the time-bandwidth product and the Shannon and Renyi(see 4th Berkeley Symp. Math., Stat., Prob., vol.1) entropies. When applied to a time-frequency representation from Cohens (1989) class, the Renyi entropy conforms closely to the visually based notion of complexity that we use when inspecting time-frequency images. A detailed discussion reveals many of the desirable properties of the Renyi information measure for both deterministic and random signals.<<ETX>>

The Relation between Granger Causality and Directed Information Theory: A Review

This report reviews the conceptual and theoretical links between Granger causality and directed information theory. We begin with a short historical tour of Granger causality, concentrating on its closeness to information theory. The definitions of Granger causality based on prediction are recalled, and the importance of the observation set is discussed. We present the definitions based on conditional independence. The notion of instantaneous coupling is included in the definitions. The concept of Granger causality graphs is discussed. We present directed information theory from the perspective of studies of causal influences between stochastic processes. Causal conditioning appears to be the cornerstone for the relation between information theory and Granger causality. In the bivariate case, the fundamental measure is the directed information, which decomposes as the sum of the transfer entropies and a term quantifying instantaneous coupling. We show the decomposition of the mutual information into the sums of the transfer entropies and the instantaneous coupling measure, a relation known for the linear Gaussian case. We study the multivariate case, showing that the useful decomposition is blurred by instantaneous coupling. The links are further developed by studying how measures based on directed information theory naturally emerge from Granger causality inference frameworks as hypothesis testing.

Estimation of Renyi information divergence via pruned minimal spanning trees

In this paper we develop robust estimators of the Renyi information divergence (I-divergence) given a reference distribution and a random sample from an unknown distribution. Estimation is performed by constructing a minimal spanning tree (MST) passing through the random sample points and applying a change of measure which flattens the reference distribution. In a mixture model where the reference distribution is contaminated by an unknown noise distribution one can use these results to reject noise samples by implementing a greedy algorithm for pruning the k-longest branches of the MST, resulting in a tree called the k-MST. We illustrate this procedure in the context of density discrimination and robust clustering for a planar mixture model.

Graph based k-means clustering

An original approach to cluster multi-component data sets is proposed that includes an estimation of the number of clusters. Using Prims algorithm to construct a minimal spanning tree (MST) we show that, under the assumption that the vertices are approximately distributed according to a spatial homogeneous Poisson process, the number of clusters can be accurately estimated by thresholding the sequence of edge lengths added to the MST by Prims algorithm. This sequence, called the Prim trajectory, contains sufficient information to determine both the number of clusters and the approximate locations of the cluster centroids. The estimated number of clusters and cluster centroids are used to initialize the generalized Lloyd algorithm, also known as k-means, which circumvents its well known initialization problems. We evaluate the false positive rate of our cluster detection algorithm, using Poisson approximations in Euclidean spaces. Applications of this method in the multi/hyper-spectral imagery domain to a satellite view of Paris and to an image of Mars are also presented.

Clustering with a new distance measure based on a dual-rooted tree

Abstract This paper introduces a novel distance measure for clustering high dimensional data based on the hitting time of two Minimal Spanning Trees (MST) grown sequentially from a pair of points by Prim’s algorithm. When the proposed measure is used in conjunction with spectral clustering, we obtain a powerful clustering algorithm that is able to separate neighboring non-convex shaped clusters and to account for local as well as global geometric features of the data set. Remarkably, the new distance measure is a true metric even if the Prim algorithm uses a non-metric dissimilarity measure to compute the edges of the MST. This metric property brings added flexibility to the proposed method. In particular, the method is applied to clustering non Euclidean quantities, such as probability distributions or spectra, using the Kullback–Leibler divergence as a base measure. We reduce computational complexity by applying consensus clustering to a small ensemble of dual rooted MSTs. We show that the resultant consensus spectral clustering with dual rooted MST is competitive with other clustering methods, both in terms of clustering performance and computational complexity. We illustrate the proposed clustering algorithm on public domain benchmark data for which the ground truth is known, on one hand, and on real-world astrophysical data on the other hand.

Detection of coherent vorticity structures using time-scale resolved acoustic spectroscopy

Abstract It seems widely accepted by the turbulence community that the intermittency observed in fully turbulent flows is closely related to the existence of intense vorticity events, localized in time and space, also known as coherent structures. We describe here an experimental technique based on the acoustic scattering phenomenon allowing the direct probing of the vorticity field in a turbulent flow. In addition, as in any scattering experiment, the information is in the Fourier domain: the scattered pressure signal is a direct image of the time evolution of a well-specified spatial Fourier mode of the vorticity field. Using time–frequency distributions (TFD), recently introduced in signal analysis theory for the analysis of the scattered acoustic signals, we show how the legibility of these signals is significantly improved (time-resolved spectroscopy). The method is illustrated on data extracted from a highly-turbulent jet flow: discrete vorticity events are clearly evidenced. The definition of a generalized time-scale correlation function allows the measurement of the spatial correlation length of these events and reveals a time continuous transfer of energy from the largest scales towards smaller scales (turbulent cascade). We claim that the recourse to TFD leads to an operational definition of coherent structures associated with phase stationarity in the time–frequency plane.

New simple implementation of the coherent signal subspace method for wide band direction of arrival estimation

The authors address the problem of source localization estimation (SLE), given the output of a sensor array, in the case of aerial acoustics in an indoor environment. The presence of echos similar to highly correlated sources necessitates the study of high-resolution methods in the case of wideband emitters. As the field of possible applications includes robotics, the authors do not assume any a priori knowledge about the source locations. Extending the coherent signal subspace (CSS) approach, first introduced by H. Wang and M. Kaveh (1983), the authors propose a method that only uses simple interpolation of array data in order to build the CSS directly.<<ETX>>

Application of methods based on higher-order statistics for chaotic time series analysis

Abstract The aim of this paper is to illustrate some applications of HOS within the context of chaotic time series analysis. After reviewing briefly some of the most popular methods and approaches used in chaotic signal analysis, we show how HOS may lead to some significant improvement. First, an HOS expansion of the mutual information is shown to provide an easy way to estimate the reconstruction delay that must be used in the embedding reconstruction method. Then, a fourth-order extension of the local intrinsic dimension analysis (LID) is proposed. The ability of this HOS extension to separate between chaotic and stochastic behaviour is illustrated by examples on simulated data and experimental time series.