Marc Vuffray

Approaching the Rate-Distortion Limit With Spatial Coupling, Belief Propagation, and Decimation

We investigate an encoding scheme for lossy compression of a binary symmetric source based on simple spatially coupled low-density generator-matrix codes. The degree of the check nodes is regular and the one of code-bits is Poisson distributed with an average depending on the compression rate. The performance of a low complexity belief propagation guided decimation algorithm is excellent. The algorithmic rate-distortion curve approaches the optimal curve of the ensemble as the width of the coupling window grows. Moreover, as the check degree grows both curves approach the ultimate Shannon rate-distortion limit. The belief propagation guided decimation encoder is based on the posterior measure of a binary symmetric test-channel. This measure can be interpreted as a random Gibbs measure at a temperature directly related to the noise level of the test-channel. We investigate the links between the algorithmic performance of the belief propagation guided decimation encoder and the phase diagram of this Gibbs measure. The phase diagram is investigated thanks to the cavity method of spin glass theory which predicts a number of phase transition thresholds. In particular, the dynamical and condensation phase transition temperatures (equivalently test-channel noise thresholds) are computed. We observe that: 1) the dynamical temperature of the spatially coupled construction saturates toward the condensation temperature and 2) for large degrees the condensation temperature approaches the temperature (i.e., noise level) related to the information theoretic Shannon test-channel noise parameter of rate-distortion theory. This provides heuristic insight into the excellent performance of the belief propagation guided decimation algorithm. This paper contains an introduction to the cavity method.

Optimal structure and parameter learning of Ising models

An arbitrary Ising model can be exactly recovered from observations using an information-theoretically optimal amount of data. Reconstruction of the structure and parameters of an Ising model from binary samples is a problem of practical importance in a variety of disciplines, ranging from statistical physics and computational biology to image processing and machine learning. The focus of the research community shifted toward developing universal reconstruction algorithms that are both computationally efficient and require the minimal amount of expensive data. We introduce a new method, interaction screening, which accurately estimates model parameters using local optimization problems. The algorithm provably achieves perfect graph structure recovery with an information-theoretically optimal number of samples, notably in the low-temperature regime, which is known to be the hardest for learning. The efficacy of interaction screening is assessed through extensive numerical tests on synthetic Ising models of various topologies with different types of interactions, as well as on real data produced by a D-Wave quantum computer. This study shows that the interaction screening method is an exact, tractable, and optimal technique that universally solves the inverse Ising problem.

Monotonicity of actuated flows on dissipative transport networks

We derive a monotonicity property for general, transient flows of a commodity transferred throughout a network, where the flow is characterized by density and mass flux dynamics on the edges with density continuity and mass balance conditions at the nodes. The dynamics on each edge are represented by a general system of partial differential equations that approximates subsonic compressible fluid flow with energy dissipation. The transferred commodity may be injected or withdrawn at any of the nodes, and is propelled throughout the network by nodally located compressors. These compressors are controllable actuators that provide a means to manipulate flows through the network, which we therefore consider as a control system. A canonical problem requires compressor control protocols to be chosen such that time-varying nodal commodity withdrawal profiles are delivered and the density remains within strict limits while an economic or operational cost objective is optimized. In this manuscript, we consider the situation where each nodal commodity withdrawal profile is uncertain, but is bounded within known maximum and minimum time-dependent limits. We introduce the monotone parameterized control system property, and prove that general dynamic dissipative network flows possess this characteristic under certain conditions. This property facilitates very efficient formulation of optimal control problems for such systems in which the solutions must be robust with respect to commodity withdrawal uncertainty. We discuss several applications in which such control problems arise and where monotonicity enables simplified characterization of system behavior.

Graphical models for optimal power flow

Optimal power flow (OPF) is the central optimization problem in electric power grids. Although solved routinely in the course of power grid operations, it is known to be strongly NP-hard in general, and weakly NP-hard over tree networks. In this paper, we formulate the optimal power flow problem over tree networks as an inference problem over a tree-structured graphical model where the nodal variables are low-dimensional vectors. We adapt the standard dynamic programming algorithm for inference over a tree-structured graphical model to the OPF problem. Combining this with an interval discretization of the nodal variables, we develop an approximation algorithm for the OPF problem. Further, we use techniques from constraint programming (CP) to perform interval computations and adaptive bound propagation to obtain practically efficient algorithms. Compared to previous algorithms that solve OPF with optimality guarantees using convex relaxations, our approach is able to work for arbitrary tree-structured distribution networks and handle mixed-integer optimization problems. Further, it can be implemented in a distributed message-passing fashion that is scalable and is suitable for “smart grid” applications like control of distributed energy resources. Numerical evaluations on several benchmark networks show that practical OPF problems can be solved effectively using this approach.

Graphical Models and Belief Propagation Hierarchy for Physics-Constrained Network Flows

We review new ideas and the first results from the application of the graphical models approach, which originated from statistical physics, information theory, computer science, and machine learning, to optimization problems of network flow type with additional constraints related to the physics of the flow. We illustrate the general concepts on a number of enabling examples from power system and natural gas transmission (continental scale) and distribution (district scale) systems.

Concentration to zero bit-error probability for regular LDPC codes on the binary symmetric channel: Proof by loop calculus

In this paper we consider regular low-density parity-check codes over a binary-symmetric channel in the decoding regime. We prove that up to a certain noise threshold the bit-error probability of the bit-sampling decoder converges in mean to zero over the code ensemble and the channel realizations. To arrive at this result we show that the bit-error probability of the sampling decoder is equal to the derivative of a Bethe free entropy. The method that we developed is new and is based on convexity of the free entropy and loop calculus. Convexity is needed to exchange limit and derivative and the loop series enables us to express the difference between the bit-error probability and the Bethe free entropy. We control the loop series using combinatorial techniques and a first moment method. We stress that our method is versatile and we believe that it can be generalized for LDPC codes with general degree distributions and for asymmetric channels.

The inviscid, compressible and rotational, 2D isotropic Burgers and pressureless Euler-Coriolis fluids: Solvable models with illustrations

The coupling between dilatation and vorticity, two coexisting and fundamental processes in fluid dynamics (Wu et al., 2006, pp. 3, 6) is investigated here, in the simplest cases of inviscid 2D isotropic Burgers and pressureless Euler Coriolis fluids respectively modeled by single vortices confined in compressible, local, inertial and global, rotating, environments. The field equations are established, inductively, starting from the equations of the characteristics solved with an initial Helmholtz decomposition of the velocity fields namely a vorticity free and a divergence free part (Wu et al., 2006, Sects. 2.3.2, 2.3.3) and, deductively, by means of a canonical Hamiltonian Clebsch like formalism (Clebsch, 1857, 1859), implying two pairs of conjugate variables. Two vector valued fields are constants of the motion: the velocity field in the Burgers case and the momentum field per unit mass in the Euler Coriolis one. Taking advantage of this property, a class of solutions for the mass densities of the fluids is given by the Jacobian of their sum with respect to the actual coordinates. Implementation of the isotropy hypothesis entails a radial dependence of the velocity potentials and of the stream functions associated to the compressible and to the rotational part of the fluids and results in the cancellation of the dilatation-rotational cross terms in the Jacobian. A simple expression is obtained for all the radially symmetric Jacobians occurring in the theory. Representative examples of regular and singular solutions are shown and the competition between dilatation and vorticity is illustrated. Inspired by thermodynamical, mean field theoretical analogies, a genuine variational formula is proposed which yields unique measure solutions for the radially symmetric fluid densities investigated. We stress that this variational formula, unlike the Hopf-Lax formula, enables us to treat systems which are both compressible and rotational. Moreover in the one-dimensional case, we show for an interesting application that both variational formulas are equivalent