David Stotz

Degrees of freedom in vector interference channels

This paper continues the Wu-Shamai-Verdu program [1] on characterizing the degrees of freedom (DoF) of interference channels (ICs) through Renyi information dimension. Concretely, we find a general formula for the DoF of vector ICs, encompassing multiple-input multiple-output (MIMO) ICs, time- and/or frequency-selective ICs, and combinations thereof, as well as constant single-antenna ICs considered in [1]. As in the case of constant single-antenna ICs, achieving full DoF requires the use of singular input distributions. Strikingly, in the vector case it suffices to enforce singularity on the joint distribution of individual transmit vectors. This can be realized through signaling in subspaces of the ambient signal space, which is in accordance with the idea of interference alignment, and, most importantly, allows the scalar components of the transmit vectors to have non-singular distributions. We recover the result by Cadambe and Jafar on the non-separability of parallel ICs [2] and we show that almost all parallel ICs are separable. Finally, our results extend the main finding in [1] to the complex case.

Degrees of Freedom in Vector Interference Channels

This paper continues the Wu-Shamai-Verdú program on characterizing the degrees of freedom (DoF) of interference channels (ICs) through Rényi information dimension. Specifically, we find a single-letter formula for the DoF of vector ICs, encompassing multiple-input multiple-output ICs, time- and/or frequency-selective ICs, and combinations thereof, as well as scalar ICs as considered by Wu et al., 2015. The DoF-formula we obtain lower-bounds the DoF of all channels-with respect to the choice of the channel matrix-and upper-bounds the DoF of almost all channels. It applies to a large class of noise distributions, and its proof is based on an extension of a result by Guionnet and Shlyakthenko, 2007, to the vector case in combination with the Ruzsa triangle inequality for differential entropy introduced by Kontoyiannis and Madiman, 2015. As in scalar ICs, achieving full DoF requires the use of singular input distributions. Strikingly, in the vector case, it suffices to enforce singularity on the joint distribution of each transmit vector. This can be realized through signaling in subspaces of the ambient signal space, which is in accordance with the idea of interference alignment, and, most importantly, allows the scalar entries of the transmit vectors to have non-singular distributions. The DoF-formula for vector ICs we obtain enables a unified treatment of classical interference alignment à la Cadambe and Jafar, 2008, and Maddah-Ali et al., 2008, and the number-theoretic schemes proposed by Motahari et al., 2014, and by Etkin and Ordentlich, 2009. Moreover, it allows to calculate the DoF achieved by new signaling schemes for vector ICs. We furthermore recover the result by Cadambe and Jafar on the non-separability of parallel ICs, 2009, and we show that almost all parallel ICs are separable in terms of DoF. Finally, our results apply to complex vector ICs, thereby extending the main findings of Wu et al., 2015 to the complex case.

Information-theoretic limits of matrix completion

We propose an information-theoretic framework for matrix completion. The theory goes beyond the low-rank structure and applies to general matrices of “low description complexity”. Specifically, we consider random matrices X ∈ ℝ^m×n of arbitrary distribution (continuous, discrete, discrete-continuous mixture, or even singular). With S ⊆ ℝ^m×n an ε-support set of X, i.e., P[X ∈ S] ≥ 1 - ε, and equation denoting the lower Minkowski dimension of S, we show that equation measurements of the form 〈A_i,X〉, with A_i denoting the measurement matrices, suffice to recover X with probability of error at most ε. The result holds for Lebesgue a.a. A_i and does not need incoherence between the A_i and the unknown matrix X. We furthermore show that equation measurements also suffice to recover the unknown matrix X from measurements taken with rank-one A_i, again this applies to a.a. rank-one A_i. Rank-one measurement matrices are attractive as they require less storage space than general measurement matrices and can be applied faster. Particularizing our results to the recovery of low-rank matrices, we find that k > (m+n-r)r measurements are sufficient to recover matrices of rank at most r. Finally, we construct a class of rank-r matrices that can be recovered with arbitrarily small probability of error from k <; (m + n - r)r measurements.

Super-resolution from short-time Fourier transform measurements

While spike trains are obviously not band-limited, the theory of super-resolution tells us that perfect recovery of unknown spike locations and weights from low-pass Fourier transform measurements is possible provided that the minimum spacing, Δ, between spikes is not too small. Specifically, for a cutoff frequency of fc, the work of Donoho (1992) shows that exact recovery is possible if Δ > l/fc, but does not specify a corresponding recovery method. On the other hand, Candès and Fernandez-Granda (2013) provide a recovery method based on convex optimization, which provably succeeds as long as Δ > 2/fc. In practical applications one often has access to windowed Fourier transform measurements, i.e., short-time Fourier transform (STFT) measurements, only. In this paper, we develop a theory of super-resolution from STFT measurements, and we propose a method that provably succeeds in recovering spike trains from STFT measurements provided that Δ > l/fc.

Almost lossless analog signal separation

We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals from a noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verdú (2010) on almost lossless analog compression. The main results of the present paper are a general achievability bound for the compression rate in the analog signal separation problem, an exact expression for the optimal compression rate in the case of signals that have mixed discrete-continuous distributions, and a new technique for showing that the intersection of generic subspaces with subsets of sufficiently small Minkowski dimension is empty. This technique can also be applied to obtain a simplified proof of a key result in Wu and Verdú (2010).

Explicit and almost sure conditions for K/2 degrees of freedom

It is well known that in K-user constant single-antenna interference channels K/2 degrees of freedom (DoF) can be achieved for almost all channel matrices. Explicit conditions on the channel matrix to admit K/2 DoF are, however, not available. The purpose of this paper is to identify such explicit conditions, which are satisfied for almost all channel matrices. We also provide a construction of corresponding asymptotically DoF-optimal input distributions. The main technical tool used is a recent breakthrough result by Hochman in fractal geometry [1].

Almost Lossless Analog Signal Separation and Probabilistic Uncertainty Relations

We propose an information-theoretic framework for analog signal separation. Specifically, we consider the problem of recovering two analog signals, modeled as general random vectors, from the noiseless sum of linear measurements of the signals. Our framework is inspired by the groundbreaking work of Wu and Verdú (2010) on analog compression and encompasses, inter alia, inpainting, declipping, super-resolution, the recovery of signals corrupted by impulse noise, and the separation of (e.g., audio or video) signals into two distinct components. The main results we report are general achievability bounds for the compression rate, i.e., the number of measurements relative to the dimension of the ambient space the signals live in, under either measurability or Hölder continuity imposed on the separator. Furthermore, we find a matching converse for sources of mixed discrete-continuous distribution. For measurable separators our proofs are based on a new probabilistic uncertainty relation, which shows that the intersection of generic subspaces with general sets of sufficiently small Minkowski dimension is empty. Hölder continuous separators are dealt with by introducing the concept of regularized probabilistic uncertainty relations. The probabilistic uncertainty relations we develop are inspired by embedding results in dynamical systems theory due to Sauer et al. (1991) and—conceptually—parallel classical Donoho-Stark and Elad-Bruckstein uncertainty principles at the heart of compressed sensing theory. Operationally, the new uncertainty relations take the theory of sparse signal separation beyond traditional sparsity—as measured in terms of the number of non-zero entries—to the more general notion of low description complexity as quantified by Minkowski dimension. Finally, our approach also allows to significantly strengthen key results in Wu and Verdú (2010).

Characterizing Degrees of Freedom Through Additive Combinatorics

We establish a formal connection between the problem of characterizing degrees of freedom (DoF) in constant single-antenna interference channels (ICs) with general channel matrix and the field of additive combinatorics. The theory we develop is based on a recent breakthrough result by Hochman, 2014, in fractal geometry. Our first main contribution is an explicit condition on the channel matrix to admit full, i.e., K/2 DoF; this condition is satisfied for almost all channel matrices. We also provide a construction of corresponding full DoF-achieving input distributions. The second main result is a new DoF-formula exclusively in terms of Shannon entropy. This formula is more amenable to both analytical statements and numerical evaluations than the DoF-formula by Wu et al., 2015, which is in terms of Rényi information dimension. We then use the new DoF-formula to shed light on the hardness of finding the exact number of DoF in ICs with rational channel coefficients, and to improve the best known bounds on the DoF of a well-studied channel matrix.

Canonical conditions for K/2 degrees of freedom

Stotz and Bölcskei, 2015, identified an explicit condition for K/2 degrees of freedom (DoF) in constant single-antenna interference channels (ICs). This condition is expressed in terms of linear independence-over the rationals-of monomials in the off-diagonal entries of the IC matrix and is satisfied for almost all IC matrices. There is, however, a prominent class of IC matrices that admits K/2 DoF but fails to satisfy this condition. The main contribution of the present paper is a more general condition for K/2 DoF (in fact for 1/2 DoF for each user) that, inter alia, encompasses this example class. While the existing condition by Stotz and Bölcskei is of algebraic nature, the new condition is canonical in the sense of capturing the essence of interference alignment by virtue of being expressed in terms of a generic injectivity condition that guarantees separability of signal and interference.