Ioannis Kontoyiannis

Nonparametric entropy estimation for stationary processes and random fields, with applications to English text

We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesaro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem due to Maker (1940). We provide examples of their performance on English text, and we generalize our results to countable alphabet processes and to random fields.

Thinning and information projections

The law of thin numbers is a Poisson approximation theorem related to the thinning operation. We use information projections to derive lower bounds on the information divergence from a thinned distribution to a Poisson distribution. Conditions for the existence of projections are given. If an information projection exists it must be an element of the associated exponential family. Exponential families are used to derive lower bounds on information divergence and lower bounds on the rate of convergence in the law of thin numbers. A method of translating results related to Poisson distributions into results related to Gaussian distributions is developed and used to prove a new non-trivial result related to the central limit theorem.

Asymptotic Recurrence and Waiting Times for Stationary Processes

AbstractLet

Estimating the Entropy of Binary Time Series: Methodology, Some Theory and a Simulation Study

{\text{X = }}\left\{ {X_n ;n \in \mathbb{Z}} \right\}

Progressive search and retrieval in large image archives

be a discrete-valued stationary ergodic process distributed according to P and let x=(..., x−1, x0, x1,...) denote a realization from X. We investigate the asymptotic behavior of the recurrence time Rn defined as the first time that the initial n-block

Sumset and Inverse Sumset Inequalities for Differential Entropy and Mutual Information

x_1^n = (x_1 ,x_2 , \ldots ,x_n )