Gregor Chliamovitch

Maximum Entropy Rate Reconstruction of Markov Dynamics

We develop ideas proposed by Van der Straeten to extend maximum entropy principles to Markov chains. We focus in particular on the convergence of such estimates in order to explain how our approach makes possible the estimation of transition probabilities when only short samples are available, which opens the way to applications to non-stationary processes. The current work complements an earlier communication by providing numerical details, as well as a full derivation of the multi-constraint two-state and three-state maximum entropy transition matrices.

Improving predictability of time series using maximum entropy methods

We discuss how maximum entropy methods may be applied to the reconstruction of Markov processes underlying empirical time series and compare this approach to usual frequency sampling. It is shown that, at least in low dimension, there exists a subset of the space of stochastic matrices for which the MaxEnt method is more efficient than sampling, in the sense that shorter historical samples have to be considered to reach the same accuracy. Considering short samples is of particular interest when modelling smoothly non-stationary processes, for then it provides, under some conditions, a powerful forecasting tool. The method is illustrated for a discretized empirical series of exchange rates.

Uncovering Discrete Non-Linear Dependence with Information Theory

In this paper, we model discrete time series as discrete Markov processes of arbitrary order and derive the approximate distribution of the Kullback-Leibler divergence between a known transition probability matrix and its sample estimate. We introduce two new information-theoretic measurements: information memory loss and information codependence structure. The former measures the memory content within a Markov process and determines its optimal order. The latter assesses the codependence among Markov processes. Both measurements are evaluated on toy examples and applied on high frequency foreign exchange data, focusing on 2008 financial crisis and 2010/2011 Euro crisis.

A Truncation Scheme for the BBGKY2 Equation

In recent years, the maximum entropy principle has been applied to a wide range of different fields, often successfully. While these works are usually focussed on cross-disciplinary applications, the point of this letter is instead to reconsider a fundamental point of kinetic theory. Namely, we shall re-examine the Stosszahlansatz leading to the irreversible Boltzmann equation at the light of the MaxEnt principle. We assert that this way of thinking allows to move one step further than the factorization hypothesis and provides a coherent—though implicit—closure scheme for the two-particle distribution function. Such higher-order dependences are believed to open the way to a deeper understanding of fluctuating phenomena.

Kinetic Theory beyond the Stosszahlansatz

In a recent paper (Chliamovitch, et al., 2015), we suggested using the principle of maximum entropy to generalize Boltzmann’s Stosszahlansatz to higher-order distribution functions. This conceptual shift of focus allowed us to derive an analog of the Boltzmann equation for the two-particle distribution function. While we only briefly mentioned there the possibility of a hydrodynamical treatment, we complete here a crucial step towards this program. We discuss bilocal collisional invariants, from which we deduce the two-particle stationary distribution. This allows for the existence of equilibrium states in which the momenta of particles are correlated, as well as for the existence of a fourth conserved quantity besides mass, momentum and kinetic energy.

Turbulence through the Spyglass of Bilocal Kinetics

In two recent papers we introduced a generalization of Boltzmann’s assumption of molecular chaos based on a criterion of maximum entropy, which allowed setting up a bilocal version of Boltzmann’s kinetic equation. The present paper aims to investigate how the essentially non-local character of turbulent flows can be addressed through this bilocal kinetic description, instead of the more standard approach through the local Euler/Navier–Stokes equation. Balance equations appropriate to this kinetic scheme are derived and closed so as to provide bilocal hydrodynamical equations at the non-viscous order. These equations essentially consist of two copies of the usual local equations, but coupled through a bilocal pressure tensor. Interestingly, our formalism automatically produces a closed transport equation for this coupling term.

Assessing complexity in cellular automata using information theory

ABSTRACT We discuss two ways in which information theory can be used to assess complexity in a system of interacting agents. In the first part, we adopt a global viewpoint and propose a characterization of complexity based on successive maximum entropy estimations of the probability density describing the system, thereby quantifying the respective role played by low and high orders of interaction. In the second part we reconsider the question from a local perspective, focussing on the statistical dependencies between neighbouring agents. These tools are tried on simple cellular automata in order to put them in perspective with other notions of complexity usually employed for such systems. We show that these approaches are hardly comparable, despite some overlap in simple cases. However this allows to interpret complexity in terms of interactions at work in a system (instead of making reference to any particular realization of this dynamics), and to shed some light on the role of initial conditions in complex systems. Clustering of the 88 non-equivalent Elementary Cellular Automata according to their position in the space of information processing features. Rules are coloured according to their Wolfram class. ECA in class I are shown in black, class II in red, chaotic automata (class III) in green and automata displaying complex behaviour (class IV) in blue. In spite of some important important differences, information features and Wolfram class are seen to overlap to a certain extent. Graphical Abstract

On the Dynamics of Multi-information in Cellular Automata

After reviewing a few key quantities of information theory, we investigate in this paper the behaviour of multi-information in elementary cellular automata. It will turn out that the usual classification by Wolfram is not well supported in terms of this information measure, or, more likely, that multi-information is blind to the kind of complexity displayed by those automata.