Fady Alajaji

Hybrid Digital–Analog Source–Channel Coding for Bandwidth Compression/Expansion

An approach to hybrid digital-analog (HDA) source-channel coding for the communication of analog sources over memoryless Gaussian channels is introduced. The HDA system, which exploits the advantages of both digital and analog systems, generalizes a scheme previously presented by the authors, and can operate for any bandwidth ratio (bandwidth compression and expansion). It is based on vector quantization and features turbo coding in its digital component and linear/nonlinear processing in its analog part. Simulations illustrate that, under both bandwidth compression and expansion modes of operation, the HDA system provides a robust and graceful performance with good reproduction fidelity for a wide range of channel conditions

The Kullback-Leibler divergence rate between Markov sources

In this work, we provide a computable expression for the Kullback-Leibler divergence rate lim/sub n/spl rarr//spl infin//1/nD(p/sup (n)//spl par/q/sup (n)/) between two time-invariant finite-alphabet Markov sources of arbitrary order and arbitrary initial distributions described by the probability distributions p/sup (n)/ and q/sup (n)/, respectively. We illustrate it numerically and examine its rate of convergence. The main tools used to obtain the Kullback-Leibler divergence rate and its rate of convergence are the theory of nonnegative matrices and Perron-Frobenius theory. Similarly, we provide a formula for the Shannon entropy rate lim/sub n/spl rarr//spl infin//1/nH(p/sup (n)/) of Markov sources and examine its rate of convergence.

Tight error bounds for nonuniform signaling over AWGN channels

We consider a Bonferroni-type lower bound due to Kounias (1968) on the probability of a finite union. The bound is expressed in terms of only the individual and pairwise event probabilities; however, it suffers from requiring an exponentially complex search for its direct implementation. We address this problem by presenting a practical algorithm for its evaluation. This bound is applied together with two other bounds, a recent lower bound (the KAT bound) and a greedy algorithm implementation of an upper bound due to Hunter (1976), to examine the symbol error (P/sub a/) and bit error (P/sub b/) probabilities of an uncoded communication system used in conjunction with M-ary phase-shift keying (PSK)/quadrature amplitude (QAM) (PSK/QAM) modulations and maximum a posteriori (MAP) decoding over additive white Gaussian noise (AWGN) channels. It is shown that the bounds-which can be efficiently computed-provide an excellent estimate of the error probabilities over the entire range of the signal-to-noise ratio (SNR) E/sub b//N/sub 0/. The new algorithmic bound and the greedy bound are particularly impressive as they agree with the simulation results even during very severe channel conditions.

Transmission of nonuniform memoryless sources via nonsystematic turbo codes

We investigate the joint source-channel coding problem of transmitting nonuniform memoryless sources over binary phase-shift keying-modulated additive white Gaussian noise and Rayleigh fading channels via turbo codes. In contrast to previous work, recursive nonsystematic convolutional encoders are proposed as the constituent encoders for heavily biased sources. We prove that under certain conditions, and when the length of the input source sequence tends to infinity, the encoder state distribution and the marginal output distribution of each constituent recursive convolutional encoder become asymptotically uniform, regardless of the degree of source nonuniformity. We also give a conjecture (which is empirically validated) on the condition for the higher order distribution of the encoder output to be asymptotically uniform, irrespective of the source distribution. Consequently, these conditions serve as design criteria for the choice of good encoder structures. As a result, the outputs of our selected nonsystematic turbo codes are suitably matched to the channel input, since a uniformly distributed input maximizes the channel mutual information, and hence, achieves capacity. Simulation results show substantial gains by the nonsystematic codes over previously designed systematic turbo codes; furthermore, their performance is within 0.74-1.17 dB from the Shannon limit. Finally, we compare our joint source-channel coding system with two tandem schemes which employ a fourth-order Huffman code (performing near-optimal data compression) and a turbo code that either gives excellent waterfall bit-error rate (BER) performance or good error-floor performance. At the same overall transmission rate, our system offers robust and superior performance at low BERs (< 10/sup -4/), while its complexity is lower.

Quantization of memoryless and Gauss-Markov sources over binary Markov channels

Joint source-channel coding for stationary memoryless and Gauss-Markov sources and binary Markov channels is considered. The channel is an additive-noise channel where the noise process is an Mth-order Markov chain. Two joint source-channel coding schemes are considered. The first is a channel-optimized vector quantizer-optimized for both source and channel. The second scheme consists of a scalar quantizer and a maximum a posteriori detector. In this scheme, it is assumed that the scalar quantizer output has residual redundancy that can be exploited by the maximum a posteriori detector to combat the correlated channel noise. These two schemes are then compared against two schemes which use channel interleaving. Numerical results show that the proposed schemes outperform the interleaving schemes. For very noisy channels with high noise correlation, gains of 4-5 dB in signal-to-noise ratio are possible.

Joint source-channel turbo coding for binary Markov sources

We investigate the construction of joint source-channel (JSC) turbo codes for the reliable communication of binary Markov sources over additive white Gaussian noise and Rayleigh fading channels. To exploit the source Markovian redundancy, the first constituent turbo decoder is designed according to a modified version of Berrous original decoding algorithm that employs the Gaussian assumption for the extrinsic information. Due to interleaving, the second constituent decoder is unable to adopt the same decoding method; so its extrinsic information is appropriately adjusted via a weighted correction term. The turbo encoder is also optimized according to the Markovian source statistics and by allowing different or asymmetric constituent encoders. Simulation results demonstrate substantial gains over the original (unoptimized) Turbo codes, hence significantly reducing the performance gap to the Shannon limit. Finally, we show that our JSC coding system considerably outperforms tandem coding schemes for bit error rates smaller than 10/sup -4/, while enjoying a lower system complexity.

On the joint source-channel coding error exponent for discrete memoryless systems

We investigate the computation of Csisza/spl acute/rs bounds for the joint source-channel coding (JSCC) error exponent E/sub J/ of a communication system consisting of a discrete memoryless source and a discrete memoryless channel. We provide equivalent expressions for these bounds and derive explicit formulas for the rates where the bounds are attained. These equivalent representations can be readily computed for arbitrary source-channel pairs via Arimotos algorithm. When the channels distribution satisfies a symmetry property, the bounds admit closed-form parametric expressions. We then use our results to provide a systematic comparison between the JSCC error exponent E/sub J/ and the tandem coding error exponent E/sub T/, which applies if the source and channel are separately coded. It is shown that E/sub T//spl les/E/sub J//spl les/2E/sub T/. We establish conditions for which E/sub J/>E/sub T/ and for which E/sub J/=2E/sub T/. Numerical examples indicate that E/sub J/ is close to 2E/sub T/ for many source-channel pairs. This gain translates into a power saving larger than 2 dB for a binary source transmitted over additive white Gaussian noise (AWGN) channels and Rayleigh-fading channels with finite output quantization. Finally, we study the computation of the lossy JSCC error exponent under the Hamming distortion measure.

Turbo codes for nonuniform memoryless sources over noisy channels

This work addresses the problem of designing turbo codes for nonuniform binary memoryless or independent and identically distributed (i.i.d.) sources over noisy channels. The extrinsic information in the decoder is modified to exploit the source redundancy in the form of nonuniformity; furthermore, the constituent encoder structure is optimized for the considered nonuniform i.i.d. source to further enhance the system performance. Some constituent encoders are found to substantially outperform Berrous (1996) (37, 21) encoder. Indeed, it is shown that the bit error rate (BER) performance of the newly designed turbo codes is greatly improved as significant coding gains are obtained.

Soft-decision demodulation design for COVQ over white, colored, and ISI Gaussian channels

In this work, the design of a q-bit (scalar and vector) soft-decision demodulator for Gaussian channels with binary phase-shift keying modulation is investigated. The demodulator is used in conjunction with a soft-decision channel-optimized vector quantization (COVQ) system. The COVQ is constructed for an expanded (q>1) discrete channel consisting of the concatenation of the modulator, the Gaussian channel, and the demodulator. It is found that as the demodulator resolution q increases, the capacity of the expanded channel increases, resulting in an improvement of the COVQ performance. Consequently, the soft-decision demodulator is designed to maximize the capacity of the expanded channel. Three Gaussian channel models are considered as follows: (1) additive white Gaussian noise channels; (2) additive colored Gaussian noise channels; and (3) Gaussian channels with intersymbol interference. Comparisons are made with (a) hard-decision COVQ systems, (b) COVQ systems which utilize interleaving, and (c) an unquantized (q=/spl infin/) soft-decision decoder proposed by Skoglund and Hedelin (1999). It is shown that substantial improvements can be achieved over COVQ systems which utilize hard decision demodulation and/or channel interleaving. The performance of the proposed COVQ system is comparable with the system by Skoglund and Hedelin-though its computational complexity is substantially less.

A lower bound on the probability of a finite union of events

Abstract A new lower bound on the probability P(A 1 ∪⋯∪A N ) is established in terms of only the individual event probabilities P(A i ) s and the pairwise event probabilities P(A i ∩A j ) s. This bound is shown to be always at least as good as two similar lower bounds: one by de Caen (1997) and the other by Dawson and Sankoff (1967). Numerical examples for the computation of this inequality are also provided.