Cheuk Ting Li

Exact common information

This paper introduces the notion of exact common information, which is the minimum description length of the common randomness needed for the exact distributed generation of two correlated random variables (X, Y). We introduce the quantity G(X; Y) = minX→W→Y H(W) as a natural bound on the exact common information and study its properties and computation. We then introduce the exact common information rate, which is the minimum description rate of the common randomness for the exact generation of a 2-DMS (X, Y). We give a multiletter characterization for it as the limit G̅(X; Y) = limn→∞(1/n)G(Xn; Yn). While in general G̅(X; Y) is greater than or equal to the Wyner common information, we show that they are equal for the Symmetric Binary Erasure Source. We do not know, however, if the exact common information rate has a single letter characterization in general.

An Efficient Feedback Coding Scheme With Low Error Probability for Discrete Memoryless Channels

Existing feedback communication schemes are either specialized to particular channels (Schalkwijk-Kailath, Horstein), apply to general channels but have high coding complexity (block feedback schemes), or are difficult to analyze (posterior matching). This paper introduces a feedback coding scheme that achieves the capacity for all discrete memoryless channels with a bound on the error exponent that approaches the sphere packing bound as the rate approaches the capacity and coding complexity of only O(n log n). These benefits are attained by combining features from previous schemes with new randomization technique and decoding rule.

Distributed simulation of continuous random variables

We establish the first known upper bound on the exact and Wyners common information of n continuous random variables in terms of the dual total correlation between them (which is a generalization of mutual information). In particular, we show that when the pdf of the random variables is log-concave, there is a constant gap of n2 log e + 9n log n between this upper bound and the dual total correlation lower bound that does not depend on the distribution. The upper bound is obtained using a computationally efficient dyadic decomposition scheme for constructing a discrete common randomness variable W from which the n random variables can be simulated in a distributed manner. We then bound the entropy of W using a new measure, which we refer to as the erosion entropy.

Channel diversity needed for vector interference alignment.

In this paper, we consider vector space interference alignment strategies over the K-user interference channel and derive an upper bound on the achievable degrees of freedom as a function of the channel diversity L. The channel diversity L is modeled by L independently fading real-valued parallel channels. Existing results in the literature for K = 3 show that the optimal 1/2 degrees of freedom per user can be approached at the speed of 1/L (i.e. the gap to 1/2 degrees of freedom per user decreases inversely proportional to L). In this paper, we show that when K ≥ 4, the speed of convergence is significantly slower. In particular, the gap to 1/2 degrees of freedom per user can decrease at most like 1/√L. Furthermore, when K is of the order of √logL, the speed of convergence is smaller than 1√(L)4.

Deterministic phase guarantees for robust recovery in incoherent dictionaries

This paper presents a relaxation of an assumption usually imposed in the recovery of sparse vectors with random support in pairs of orthonormal bases or incoherent dictionaries by basis pursuit. The assumption requires the phases of the entries of the sparse vector to be chosen randomly in [0, 2π). This paper provides probabilistic recovery guarantees for deterministic phases. We prove that, if a phase pattern is fixed, then a sparse vector with random support and corresponding phases can be recovered with high probability. As a result, the phases can take any distribution and can be dependent, as long as they are independent of the support. Furthermore, this improvement does not come at the expense of the maximum recoverable sparsity.

Multi-rate sequential data transmission

We investigate the data transmission problem in which a sequence of data is broadcast to a number of receivers via erasure channels with different erasure probabilities. Accordingly, the receivers wish to decode the data sequentially at different rates. We present a formulation of the problem and propose an optimal coding scheme. Our results can be employed in the streaming of a video clip by broadcasting, so that receivers with different bandwidths can play the video at different speeds. Specifically, receivers with sufficiently large bandwidth can play the video at normal speed, while others can play the video with pauses, or at a slower speed using time-scale modification. Our results completely characterize the fundamental tradeoff between the available bandwidth and the playback speed of the video.

Extended Gray-Wyner system with complementary causal side information

We establish the rate region of an extended Gray-Wyner system for 2-DMS (X, Y) with two additional decoders having complementary causal side information. This extension is interesting because in addition to the operationally significant extreme points of the Gray-Wyner rate region, which include Wyners common information, Gåcs-Körner common information and information bottleneck, the rate region for the extended system also includes the Körner graph entropy, the privacy funnel and excess functional information, as well as three new quantities of potential interest, as extreme points. To simplify the investigation of the 5-dimensional rate region of the extended Gray-Wyner system, we establish an equivalence of this region to a 3-dimensional mutual information region that consists of the set of all triples of the form (I (X; U), I (Y; U), I (X, Y; U)) for some pu\x, y. We further show that projections of this mutual information region yield the rate regions for many settings involving a 2-DMS, including lossless source coding with causal side information, distributed channel synthesis, and lossless source coding with a helper.

Strong functional representation lemma and applications to coding theorems

This paper shows that for any random variables X and Y, it is possible to represent Y as a function of (X, Z) such that Z is independent of X and I(X; Z| Y) ≤ log(I(X; Y)+1)+4. We use this strong functional representation lemma (SFRL) to establish a tighter bound on the rate needed for one-shot exact channel simulation than was previously established by Harsha et. al., and to establish achievability results for one-shot variable-length lossy source coding and multiple description coding. We also show that the SFRL can be used to reduce the channel with state noncausally known at the encoder to a point-to-point channel, which provides a simple achievability proof of the Gelfand-Pinsker theorem. Finally we present an example in which the SFRL inequality is tight to within 5 bits.

Channel Diversity Needed for Vector Space Interference Alignment

We consider vector space interference alignment strategies over the K-user interference channel and derive an upper bound on the achievable degrees of freedom as a function of the channel diversity L, where the channel diversity is modeled by L real-valued parallel channels with coefficients drawn from a nondegenerate joint distribution. The seminal work of Cadambe and Jafar shows that when L is unbounded, vector space interference alignment can achieve 1/2 degrees of freedom per user independent of the number of users K. However, wireless channels have limited diversity, in practice, dictated by their coherence time and bandwidth, and an important question is the number of degrees of freedom achievable at finite L. When K = 3 and if L is finite, Bresler et al. show that the number of degrees of freedom achievable with vector space interference alignment is bounded away from 1/2, and the gap decreases inversely proportional to L. In this paper, we show that when K ≥ 4, the gap is significantly larger. In particular, the gap to the optimal 1/2 degrees of freedom per user can decrease at most like 1/√L, and when L is smaller than the order of 2(K-2)(K-3), it decays at most like 1/4√L.

An efficient feedback coding scheme with low error probability for discrete memoryless channels

Existing fixed-length feedback communication schemes are either specialized to particular channels (Schalkwijk-Kailath, Horstein), or apply to general channels but either have high coding complexity (block feedback schemes) or are difficult to analyze (posterior matching). This paper introduces a new fixed-length feedback coding scheme which achieves the capacity for all discrete memoryless channels, has an error exponent that approaches the sphere packing bound as the rate approaches the capacity, and has O(n log n) coding complexity. These benefits are achieved by judiciously combining features from previous schemes with new randomization technique and encoding/decoding rule. These new features make the analysis of the error probability for the new scheme easier than for posterior matching.