Wasim Huleihel

Analysis of Mismatched Estimation Errors Using Gradients of Partition Functions

We consider the problem of signal estimation (denoising) from a statistical-mechanical perspective, in continuation to a recent work on the analysis of mean-square error (MSE) estimation using a direct relationship between optimum estimation and certain partition functions. This paper consists of essentially two parts. In the first part, using the aforementioned relationship, we derive single-letter expressions of the asymptotic mismatched MSE of a codeword (from a randomly selected code), corrupted by a Gaussian vector channel. In the second part, we provide several examples to demonstrate phase transitions in the behavior of the MSE. These examples enable us to understand more deeply and to gather intuition regarding the roles of the real and the mismatched probability measures in creating these phase transitions.

Erasure/List Random Coding Error Exponents Are Not Universally Achievable

We study the problem of universal decoding for unknown discrete memoryless channels in the presence of erasure/list option at the decoder, in the random coding regime. In particular, we harness a universal version of Forney’s classical erasure/list decoder developed in earlier studies, which is based on the competitive minimax methodology, and guarantees universal achievability of a certain fraction of the optimum random coding error exponents. In this paper, we derive an exact single-letter expression for the maximum achievable fraction. Examples are given in which the maximal achievable fraction is strictly less than unity, which imply that, in general, there is no universal erasure/list decoder, which achieves the same random coding error exponents as the optimal decoder for a known channel. This is in contrast to the situation in ordinary decoding (without the erasure/list option), where optimum exponents are universally achievable, as is well known. It is also demonstrated that previous lower bounds derived for the maximal achievable fraction are not tight in general. We then analyze a generalized random coding ensemble, which incorporate a training sequence, in conjunction with a suboptimal practical decoder (“plug-in” decoder), which first estimates the channel using the available training sequence, and then decodes the remaining symbols of the codeword using the estimated channel. One of the implications of our results is setting the stage for a reasonable criterion of optimal training. Finally, we compare the performance of the “plug-in” decoder and the universal decoder, in terms of the achievable error exponents, and show that the latter is noticeably better than the former.

Asymptotic MMSE analysis under sparse representation modeling

Compressed sensing is a signal processing technique in which data is acquired directly in a compressed form. There are two modeling approaches that can be considered: the worst-case (Hamming) approach and a statistical mechanism, in which the signals are modeled as random processes rather than as individual sequences. In this paper, the second approach is studied. Accordingly, we consider a model of the form Y = HX +W, where each component of X is given by Xi = SiUi, where {Ui} are i.i.d. Gaussian random variables, and {Si} are binary random variables independent of {Ui{, and not necessarily independent and identically distributed (i.i.d.), H ∈ ℝk×n is a random matrix with i.i.d. entries, and W is white Gaussian noise. Using a direct relationship between optimum estimation and certain partition functions, and by invoking methods from statistical mechanics and from random matrix theory, we derive an asymptotic formula for the minimum mean-square error (MMSE) of estimating the input vector X given Y and H, as k, n → ∞, keeping the measurement rate, R = k/n, fixed. In contrast to previous derivations, which are based on the replica method, the analysis carried in this paper is rigorous. In contrast to previous works in which only memoryless sources were considered, we consider a more general model which allows a certain structured dependency among the various components of the source.

Random Coding Error Exponents for the Two-User Interference Channel

This paper is about deriving lower bounds on the error exponents for the two-user interference channel under the random coding regime for several ensembles. Specifically, we first analyze the standard random coding ensemble, where the codebooks are comprised of independently and identically distributed (i.i.d.) codewords. For this ensemble, we focus on optimum decoding, which is in contrast to other, suboptimal decoding rules that have been used in the literature (e.g., joint typicality decoding, treating interference as noise, and so on). The fact that the interfering signal is a codeword, rather than an i.i.d. noise process, complicates the application of conventional techniques of performance analysis of the optimum decoder. In addition, unfortunately, these conventional techniques result in loose bounds. Using analytical tools rooted in statistical physics, as well as advanced union bounds, we derive single-letter formulas for the random coding error exponents. We compare our results with the best known lower bound on the error exponent, and show that our exponents can be strictly better. Then, in the second part of this paper, we consider more complicated coding ensembles and find a lower bound on the error exponent associated with the celebrated Han–Kobayashi random coding ensemble, which is based on superposition coding.

Universal Decoding for Gaussian Intersymbol Interference Channels

A universal decoding procedure is proposed for the intersymbol interference (ISI) Gaussian channels. The universality of the proposed decoder is in the sense of being independent of the various channel parameters, and at the same time, attaining the same random coding error exponent as the optimal maximum-likelihood (ML) decoder, which utilizes full knowledge of these unknown parameters. The proposed decoding rule can be regarded as a frequency domain version of the universal maximum mutual information (MMI) decoder. Contrary to previously suggested universal decoders for ISI channels, our proposed decoding metric can easily be evaluated.

Erasure/list random coding error exponents are not universally achievable

We study the problem of universal decoding for unknown discrete memoryless channels in the presence of erasure/list option at the decoder, in the random coding regime. In particular, we harness a universal version of Forneys classical erasure/list decoder developed in earlier studies, which is based on the competitive minimax methodology, and guarantees universal achievability of a certain fraction of the optimum random coding error exponents. In this paper, we derive an exact single-letter expression for the maximum achievable fraction. Examples are given in which the maximal achievable fraction is strictly less than unity, which imply that, in general, there is no universal erasure/list decoder, which achieves the same random coding error exponents as the optimal decoder for a known channel. This is in contrast to the situation in ordinary decoding (without the erasure/list option), where optimum exponents are universally achievable, as is well known. It is also demonstrated that previous lower bounds derived for the maximal achievable fraction are not tight in general. We then analyze a generalized random coding ensemble, which incorporate a training sequence, in conjunction with a suboptimal practical decoder (“plug-in” decoder), which first estimates the channel using the available training sequence, and then decodes the remaining symbols of the codeword using the estimated channel. One of the implications of our results is setting the stage for a reasonable criterion of optimal training. Finally, we compare the performance of the “plug-in” decoder and the universal decoder, in terms of the achievable error exponents, and show that the latter is noticeably better than the former.

On Compressive Sensing in Coding Problems: A Rigorous Approach

We take an information theoretic perspective on a classical sparse-sampling noisy linear model and present an analytical expression for the mutual information, which plays a central role in a variety of communications/signal processing problems. Such an expression was addressed previously by bounds, by simulations, and by the (nonrigorous) replica method. The expression of the mutual information is based on techniques used, addressing the minimum mean square error analysis. Using these expressions, we study specifically a variety of sparse linear communication models, which include coding in various settings, accounting also for multiple access channels, broadcast channels, and different wiretap problems. For those, we provide single-letter expressions and derive achievable rates, capturing the communications/signal processing features of these contemporary models.

Multiple access channel with unreliable cribbing

It is by now well-known that cooperation between users can lead to significant performance gains. A common assumption in past works is that all the users are aware of the resources available for cooperation, and know exactly to what extent these resources can be used. In this work, we consider the multiple access channel (MAC) with (strictly causal, causal, and non-causal) cribbing that may be absent. The derived achievable regions are based on universal coding scheme which exploit the cribbing link if it is present, and can still operate (although at reduced rates) if cribbing is absent. We derive also an outer bound, which for some special case is tight.

Universal decoding for Gaussian intersymbol interference channels

A universal decoding procedure is proposed for the intersymbol interference (ISI) Gaussian channels. The universality of the proposed decoder is in the sense of being independent of the channel parameters, and at the same time, attaining the same random coding error exponent as the optimal maximum-likelihood decoder, which utilizes full knowledge of these unknown parameters. The proposed decoding rule can be regarded as a frequency domain version of the universal maximum mutual information decoder. Contrary to previously suggested universal decoders for ISI channels, our proposed decoding metric can easily be evaluated.

The error exponent of the binary symmetric channel for asymmetric random codes

In this paper, we consider asymmetric binary codes from Shannons random code ensemble and also from the random linear code ensemble (LCE), used over the the binary symmetric channel (BSC). The asymmetry is in the sense that the codeword symbols are not necessarily chosen in an equiprobable manner. One possible motivation for such asymmetry is mismatch, where the user uses asymmetric codes over the BSC, leading to a degradation in performance. Accordingly, we derive the distance distribution and the error exponents of a typical random code (TRC) from the RCE, and of a typical linear code (TLC) from the LCE. The derivation is based on a fine large-deviation analysis of some distance enumerators, contrary to the usual bounding technique by Gallager. Later, we propose a “time-varying” BSC model, in which the crossover probability of the BSC is time-dependent, and use our results for providing a lower bound on the error exponent of this channel model.