Dongning Guo

Mutual information and minimum mean-square error in Gaussian channels

This paper deals with arbitrarily distributed finite-power input signals observed through an additive Gaussian noise channel. It shows a new formula that connects the input-output mutual information and the minimum mean-square error (MMSE) achievable by optimal estimation of the input given the output. That is, the derivative of the mutual information (nats) with respect to the signal-to-noise ratio (SNR) is equal to half the MMSE, regardless of the input statistics. This relationship holds for both scalar and vector signals, as well as for discrete-time and continuous-time noncausal MMSE estimation. This fundamental information-theoretic result has an unexpected consequence in continuous-time nonlinear estimation: For any input signal with finite power, the causal filtering MMSE achieved at SNR is equal to the average value of the noncausal smoothing MMSE achieved with a channel whose SNR is chosen uniformly distributed between 0 and SNR.

In-Band Full-Duplex Wireless: Challenges and Opportunities

In-band full-duplex (IBFD) operation has emerged as an attractive solution for increasing the throughput of wireless communication systems and networks. With IBFD, a wireless terminal is allowed to transmit and receive simultaneously in the same frequency band. This tutorial paper reviews the main concepts of IBFD wireless. One of the biggest practical impediments to IBFD operation is the presence of self-interference, i.e., the interference that the modems transmitter causes to its own receiver. This tutorial surveys a wide range of IBFD self-interference mitigation techniques. Also discussed are numerous other research challenges and opportunities in the design and analysis of IBFD wireless systems.

Rethinking information theory for mobile ad hoc networks

The subject of this article is the long standing open problem of developing a general capacity theory for wireless networks, particularly a theory capable of describing the fundamental performance limits of mobile ad hoc networks. A MANET is a peer-to-peer network with no preexisting infrastructure. MANETs are the most general wireless networks, with single-hop, relay, interference, mesh, and star networks comprising special cases. The lack of a MANET capacity theory has stunted the development and commercialization of many types of wireless networks, including emergency, military, sensor, and community mesh networks. Information theory, which has been vital for links and centralized networks, has not been successfully applied to decentralized wireless networks. Even if this was accomplished, for such a theory to truly characterize the limits of deployed MANETs it must overcome three key roadblocks. First, most current capacity results rely on the allowance of unbounded delay and reliability. Second, spatial and timescale decompositions have not yet been developed for optimally modeling the spatial and temporal dynamics of wireless networks. Third, a useful network capacity theory must integrate rather than ignore the important role of overhead messaging and feedback. This article describes some of the shifts in thinking that may be needed to overcome these roadblocks and develop a more general theory.

A matrix-algebraic approach to linear parallel interference cancellation in CDMA

Linear parallel interference cancellation (PIC) schemes are described and analyzed using matrix algebra. It is shown that the linear PIC, whether conventional or weighted, can be seen as a linear matrix filter applied directly to the chip-matched filtered received signal vector. An expression for the exact bit-error rate (BER) is obtained, and conditions on the eigenvalues of the code correlation matrix and the weighting factors to ensure convergence are derived. The close relationship between the linear multistage PIC and the steepest descent method (SDM) for minimizing the mean squared error (MSE) is demonstrated. A modified weighted PIC structure that resembles the SDM is suggested which approaches the minimum MSE (MMSE) detector rather than the decorrelator. It is shown that for a K-user system, only K PIC stages are required for the equivalent matrix filter to be identical to the the MMSE filter. For fewer stages, techniques are devised for optimizing the choice of weights with respect to the MSE. One unique optimal choice of weights is found, which will lead to the minimum achievable MSE at the final stage. Simulation results show that a few stages are sufficient for near-MMSE performance.

Estimation in Gaussian Noise: Properties of the Minimum Mean-Square Error

Consider the minimum mean-square error (MMSE) of estimating an arbitrary random variable from its observation contaminated by Gaussian noise. The MMSE can be regarded as a function of the signal-to-noise ratio (SNR) as well as a functional of the input distribution (of the random variable to be estimated). It is shown that the MMSE is concave in the input distribution at any given SNR. For a given input distribution, the MMSE is found to be infinitely differentiable at all positive SNR, and in fact a real analytic function in SNR under mild conditions. The key to these regularity results is that the posterior distribution conditioned on the observation through Gaussian channels always decays at least as quickly as some Gaussian density. Furthermore, simple expressions for the first three derivatives of the MMSE with respect to the SNR are obtained. It is also shown that, as functions of the SNR, the curves for the MMSE of a Gaussian input and that of a non-Gaussian input cross at most once over all SNRs. These properties lead to simple proofs of the facts that Gaussian inputs achieve both the secrecy capacity of scalar Gaussian wiretap channels and the capacity of scalar Gaussian broadcast channels, as well as a simple proof of the entropy power inequality in the special case where one of the variables is Gaussian.

Multiuser Detection of Sparsely Spread CDMA

Code-division multiple access (CDMA) is the basis of a family of advanced air interfaces in current and future generation networks. The benefits promised by CDMA have not been fully realized partly due to the prohibitive complexity of optimal detection and decoding of many users communicating simultaneously using the same frequency band. From both theoretical and practical perspectives, this paper advocates a new paradigm of CDMA with sparse spreading sequences, which enables near-optimal multiuser detection using belief propagation (BP) with low-complexity. The scheme is in part inspired by capacity-approaching low-density parity-check (LDPC) codes and the success of iterative decoding techniques. Specifically, it is shown that BP-based detection is optimal in the large-system limit under many practical circumstances, which is a unique advantage of sparsely spread CDMA systems. Moreover, it is shown that, from the viewpoint of an individual user, the CDMA channel is asymptotically equivalent to a scalar Gaussian channel with some degradation in the signal-to-noise ratio (SNR). The degradation factor, known as the multiuser efficiency, can be determined from a fixed-point equation. The results in this paper apply to a broad class of sparse, semi-regular CDMA systems with arbitrary input and power distribution. Numerical results support the theoretical findings for systems of moderate size, which further demonstrate the appeal of sparse spreading in practical applications.

Asymptotic normality of linear multiuser receiver outputs

This paper proves large-system asymptotic normality of the output of a family of linear multiuser receivers that can be arbitrarily well approximated by polynomial receivers. This family of receivers encompasses the single-user matched filter, the decorrelator, the minimum mean square error (MMSE) receiver, the parallel interference cancelers, and many other linear receivers of interest. Both with and without the assumption of perfect power control, we show that the output decision statistic for each user converges to a Gaussian random variable in distribution as the number of users and the spreading factor both tend to infinity with their ratio fixed. Analysis reveals that the distribution conditioned on almost all spreading sequences converges to the same distribution, which is also the unconditional distribution. This normality principle allows the system performance, e.g., the multiuser efficiency, to be completely determined by the output signal-to-interference ratio (SIR) for large linear systems.

A single-letter characterization of optimal noisy compressed sensing

Compressed sensing deals with the reconstruction of a high-dimensional signal from far fewer linear measurements, where the signal is known to admit a sparse representation in a certain linear space. The asymptotic scaling of the number of measurements needed for reconstruction as the dimension of the signal increases has been studied extensively. This work takes a fundamental perspective on the problem of inferring about individual elements of the sparse signal given the measurements, where the dimensions of the system become increasingly large. Using the replica method, the outcome of inferring about any fixed collection of signal elements is shown to be asymptotically decoupled, i.e., those elements become independent conditioned on the measurements. Furthermore, the problem of inferring about each signal element admits a single-letter characterization in the sense that the posterior distribution of the element, which is a sufficient statistic, becomes asymptotically identical to the posterior of inferring about the same element in scalar Gaussian noise. The result leads to simple characterization of all other elemental metrics of the compressed sensing problem, such as the mean squared error and the error probability for reconstructing the support set of the sparse signal. Finally, the single-letter characterization is rigorously justified in the special case of sparse measurement matrices where belief propagation becomes asymptotically optimal.

A simple proof of the entropy-power inequality

This correspondence gives a simple proof of Shannons entropy-power inequality (EPI) using the relationship between mutual information and minimum mean-square error (MMSE) in Gaussian channels.

Vector Precoding for Wireless MIMO Systems and its Replica Analysis

This paper studies a nonlinear vector precoding scheme which inverts the wireless multiple-input multiple-output (MIMO) channel at the transmitter so that simple symbol-by-symbol detection can be used in lieu of sophisticated multiuser detection at the receiver. In particular, the transmit energy is minimized by relaxing the transmitted symbols to a larger alphabet for precoding, which preserves the minimum signaling distance. The so-called replica method is used to analyze the average energy savings with random MIMO channels in the large-system limit. It is found that significant gains can be achieved with complex-valued alphabets. The analysis applies to a very general class of MIMO channels, where the statistics of the channel matrix enter the result via the R-transform of the asymptotic empirical distribution of its eigenvalues. Moreover, we introduce polynomial-complexity precoding schemes for binary and quadrature phase-shift keying in complex channels by using convex rather than discrete relaxed alphabets. In case the number of transmit antennas is more than twice the number of receive antennas, we show that a convex precoding scheme, despite its polynomial complexity, outperforms NP-hard precoding using the popular Tomlinson-Harashima signaling.