Christophe Vignat

Estimating the entropy of a signal with applications

We present a new estimator of the entropy of continuous signals. We model the unknown probability density of data in the form of an AR spectrum density and use regularized long-AR models to identify the AR parameters. We then derive both an analytical expression and a practical procedure for estimating the entropy from sample data. We indicate how to incorporate recursive and adaptive features in the procedure. We evaluate and compare the new estimator with other estimators based on histograms, kernel density models, and order statistics. Finally, we give several examples of applications. An adaptive version of our entropy estimator is applied to detection of law changes, blind deconvolution, and source separation.

Analysis of signals in the Fisher–Shannon information plane

We show that the analysis of complex, possibly non-stationary signals, can be carried out in an information plane, defined by both Shannon entropy and Fisher information. Our study is exemplified by two large families of distributions with physical relevance: the Student-t and the power exponentials.  2003 Elsevier Science B.V. All rights reserved.

Some results concerning maximum Renyi entropy distributions

Abstract We consider the Student-t and Student-r distributions, which maximise Renyi entropy under a covariance condition. We show that they have information-theoretic properties which mirror those of the Gaussian distributions, which maximise Shannon entropy under the same condition. We introduce a convolution which preserves the Renyi maximising family, and show that the Renyi maximisers are the case of equality in a version of the Entropy Power Inequality. Further, we show that the Renyi maximisers satisfy a version of the heat equation, motivating the definition of a generalised Fisher information.

Central limit theorem and deformed exponentials

The central limit theorem (CLT) can be ranked among the most important ones in probability theory and statistics and plays an essential role in several basic and applied disciplines, notably in statistical thermodynamics. We show that there exists a natural extension of the CLT from exponentials to so-called deformed exponentials (also denoted as q-Gaussians). Our proposal applies exactly in the usual conditions in which the classical CLT is used.

Separation of a class of convolutive mixtures: a contrast function approach

In this paper, we address the problem of the separation of convolutive mixtures in the case where the non-Gaussian source signals are not necessarily filtered versions of i.i.d. sequences. In this context, we show that the contrast functions, used in the linear process source case, still allow to separate the sources by a deflation approach. Some particular properties of higher order cumulants based contrast functions are also given.

Some extensions of the uncertainty principle

We study the formulation of the uncertainty principle in quantum mechanics in terms of entropic inequalities, extending results recently derived by Bialynicki-Birula [I. Bialynicki-Birula, Formulation of the uncertainty relations in terms of the Renyi entropies, Physical Review A 74 (5) (2006) 052101] and Zozor et al. [S. Zozor, C. Vignat, On classes of non-Gaussian asymptotic minimizers in entropic uncertainty principles, Physica A 375 (2) (2007) 499–517]. Those inequalities can be considered as generalizations of the Heisenberg uncertainty principle, since they measure the mutual uncertainty of a wave function and its Fourier transform through their associated Renyi entropies with conjugated indices. We consider here the general case where the entropic indices are not conjugated, in both cases where the state space is discrete and continuous: we discuss the existence of an uncertainty inequality depending on the location of the entropic indices α and β in the plane (α,β). Our results explain and extend a recent study by Luis [A. Luis, Quantum properties of exponential states, Physical Review A 75 (2007) 052115], where states with quantum fluctuations below the Gaussian case are discussed at the single point (2,2).

On classes of non-Gaussian asymptotic minimizers in entropic uncertainty principles

In this paper we revisit the Bialynicki-Birula and Mycielski uncertainty principle and its cases of equality. This Shannon entropic version of the well-known Heisenberg uncertainty principle can be used when dealing with variables that admit no variance. In this paper, we extend this uncertainty principle to Renyi entropies. We recall that in both Shannon and Renyi cases, and for a given dimension n, the only case of equality occurs for Gaussian random vectors. We show that as n grows, however, the bound is also asymptotically attained in the cases of n-dimensional Student-t and Student-r distributions. A complete analytical study is performed in a special case of a Student-t distribution. We also show numerically that this effect exists for the particular case of a n-dimensional Cauchy variable, whatever the Renyi entropy considered, extending the results of Abe and illustrating the analytical asymptotic study of the Student-t case. In the Student-r case, we show numerically that the same behavior occurs for uniformly distributed vectors. These particular cases and other ones investigated in this paper are interesting since they show that this asymptotic behavior cannot be considered as a “Gaussianization” of the vector when the dimension increases.

On minimum Fisher information distributions with restricted support and fixed variance

Fisher information is of key importance in estimation theory. It also serves in inference problems as well as in the interpretation of many physical processes. The mean-squared estimation error for the location parameter of a distribution is bounded by the inverse of the Fisher information associated with this distribution. In this paper we look for minimum Fisher information distributions with a restricted support. More precisely, we study the problem of minimizing the Fisher information in the set of distributions with fixed variance defined on a bounded subset S of R or on the positive real line. We show that the solutions of the underlying differential equation can be expressed in terms of Whittaker functions. Then, in the two considered cases, we derive the explicit expressions of the solutions and investigate their behavior. We also characterize the behavior of the minimum Fisher information as a function of the imposed variance.

High SNR analysis of the MIMO interference channel

The rate region achievable by two transmitter-receiver pairs who wish to communicate over a Gaussian interference channel has been the subject of intense study over the last decades. Recently, the high SNR capacity region of this channel has been completely characterized in a work by Etkin, Tse and Wang (2007). In this paper, we study the effect of adding random fading into the picture, as well as multiple antennas at the transmitters and the receivers. Under the fast fading assumption, we recover a result of the same type as that obtained in the above mentioned paper. Under the slow fading assumption, we obtain an upper bound on the maximally achievable diversity order for a given target rate pair, which we conjecture to be tight.

In situ measurements of the complex permittivity of materials using reflection ellipsometry in the microwave band: experiments (Part II)

The aim of this series of two papers is to propose an original and low-cost tool dedicated to industrial applications and based on the reflection ellipsometry technique for in situ characterization of dielectric materials at microwave frequencies. In this first paper, different theoretical developments are presented that concern first a specific numerical method for calculating the complex permittivity of a single-layer sample from the measured parameters. Based on contour line charts, this method allows obtaining simultaneously the relative uncertainties on the real and imaginary parts of the complex permittivity. Secondly, for experimental comparisons with the classical Fresnel method, a numerical data processing method based on contour line charts has also been developed, which aims at the determination of the reflection coefficients in both parallel and perpendicular polarizations of the material.