Yangfan Zhong

On the joint source-channel coding error exponent for discrete memoryless systems

We investigate the computation of Csisza/spl acute/rs bounds for the joint source-channel coding (JSCC) error exponent E/sub J/ of a communication system consisting of a discrete memoryless source and a discrete memoryless channel. We provide equivalent expressions for these bounds and derive explicit formulas for the rates where the bounds are attained. These equivalent representations can be readily computed for arbitrary source-channel pairs via Arimotos algorithm. When the channels distribution satisfies a symmetry property, the bounds admit closed-form parametric expressions. We then use our results to provide a systematic comparison between the JSCC error exponent E/sub J/ and the tandem coding error exponent E/sub T/, which applies if the source and channel are separately coded. It is shown that E/sub T//spl les/E/sub J//spl les/2E/sub T/. We establish conditions for which E/sub J/>E/sub T/ and for which E/sub J/=2E/sub T/. Numerical examples indicate that E/sub J/ is close to 2E/sub T/ for many source-channel pairs. This gain translates into a power saving larger than 2 dB for a binary source transmitted over additive white Gaussian noise (AWGN) channels and Rayleigh-fading channels with finite output quantization. Finally, we study the computation of the lossy JSCC error exponent under the Hamming distortion measure.

Joint Source–Channel Coding Error Exponent for Discrete Communication Systems With Markovian Memory

We study the error exponent, E_J, for reliably transmitting a discrete stationary ergodic Markov (SEM) source Q over a discrete channel W with additive SEM noise via a joint source-channel (JSC) code. We first establish an upper bound for E_J in terms of the Renyi entropy rates of the source and noise processes. We next investigate the analytical computation of E_J by comparing our bound with Gallagers lower bound (1968) when the latter one is specialized to the SEM source-channel system. We also note that both bounds can be represented in Csiszars form (1980), as the minimum of the sum of the source and channel error exponents. Our results provide us with the tools to systematically compare E_J with the tandem (separate) coding exponent E_J. We show that as in the case of memoryless source-channel pairs E_J les 2E_r and we provide explicit conditions for which E_J > E_T. Numerical results indicate that E_J ap 2E_T for many SEM source-channel pairs, hence illustrating a substantial advantage of JSC coding over tandem coding for systems with Markovian memory.

Error Exponents for Asymmetric Two-User Discrete Memoryless Source-Channel Systems

Consider transmitting two discrete memoryless correlated sources, consisting of a common and a private source, over a discrete memoryless multi-terminal channel with two transmitters and two receivers. At the transmitter side, the common source is observed by both encoders but the private source can only be accessed by one encoder. At the receiver side, both decoders need to reconstruct the common source, but only one decoder needs to reconstruct the private source. We hence refer to this system by the asymmetric 2-user source-channel system. In this work, we derive a universally achievable joint source-channel coding (JSCC) error exponent pair for the 2-user system by using a technique which generalizes Csiszars method (1980) for the point- to-point (single-user) discrete memoryless source-channel system. We next investigate the largest convergence rate of asymptotic exponential decay of the system (overall) probability of erroneous transmission, i.e., the system JSCC error exponent. We obtain lower and upper bounds for the exponent. As a consequence, we establish the JSCC theorem with single letter characterization.

A Type Covering Lemma and the Excess Distortion Exponent for Coding Memoryless Laplacian Sources

In this work, we introduce the notion of Laplacian-type class and derive a type covering lemma for the memoryless Laplacian source (MLS) under the magnitude-error distortion measure. We then present an application of the type covering lemma to the lossy coding of the MLS. We establish a simple analytical lower bound for the excess distortion exponent, namely, the exponent of the probability of representing the source beyond a given distortion threshold. It is noted that, by introducing the Laplacian-type class, one can employ the classical method of types to solve source coding and source-channel coding problems regarding the MLS

Error Exponents for Asymmetric Two-User Discrete Memoryless Source-Channel Coding Systems

We study the transmission of two discrete memoryless correlated sources, consisting of a common and a private source, over a discrete memoryless multiterminal channel with two transmitters and two receivers. At the transmitter side, the common source is observed by both encoders but the private source can only be accessed by one encoder. At the receiver side, both decoders need to reconstruct the common source, but only one decoder needs to reconstruct the private source. We hence refer to this system by the asymmetric two-user source-channel coding system. We derive a universally achievable lossless joint source-channel coding (JSCC) error exponent pair for the two-user system by using a technique which generalizes Csiszars type-packing lemma (1980) for the point-to-point (single-user) discrete memoryless source-channel system. We next investigate the largest convergence rate of asymptotic exponential decay of the system (overall) probability of erroneous transmission, i.e., the system JSCC error exponent. We obtain lower and upper bounds for the exponent. As a consequence, we establish a JSCC theorem with single-letter characterization and we show that the separation principle holds for the asymmetric two-user scenario. By introducing common randomization, we also provide a formula for the tandem (separate) source-channel coding error exponent. Numerical examples show that for a large class of systems consisting of two correlated sources and an asymmetric multiple-access channel with additive noise, the JSCC error exponent considerably outperforms the corresponding tandem coding error exponent.

On the Excess Distortion Exponent for Memoryless Gaussian Source-Channel Pairs

For a memoryless Gaussian source under the squared-error distortion fidelity criterion and a memoryless additive Gaussian noise channel with a quadratic power constraint at the channel input, upper and lower bounds for the joint source-channel coding excess distortion exponent (which is the exponent of the probability of excess distortion) are established. A necessary and sufficient condition for which the two bounds coincide is provided, thus exactly determining the exponent. This condition is observed to hold for a wide range of source-channel parameters

Joint Source–Channel Coding Excess Distortion Exponent for Some Memoryless Continuous-Alphabet Systems

We investigate the joint source-channel coding (JSCC) excess distortion exponent EJ (the exponent of the probability of exceeding a prescribed distortion level) for some memoryless communication systems with continuous alphabets. We first establish upper and lower bounds for EJ for systems consisting of a memoryless Gaussian source under the squared-error distortion fidelity criterion and a memoryless additive Gaussian noise channel with a quadratic power constraint at the channel input. A necessary and sufficient condition for which the two bounds coincide is provided, thus exactly determining the exponent. This condition is observed to hold for a wide range of source-channel parameters. As an application, we study the advantage in terms of the excess distortion exponent of JSCC over traditional tandem (separate) coding for Gaussian systems. A formula for the tandem exponent is derived in terms of the Gaussian source and Gaussian channel exponents, and numerical results show that JSCC often substantially outperforms tandem coding. The problem of transmitting memoryless Laplacian sources over the Gaussian channel under the magnitude-error distortion is also carried out. Finally, we establish a lower bound for EJ for a certain class of continuous source-channel pairs when the distortion measure is a metric.

Source Side Information Can Increase the Joint Source-Channel Coding Error Exponent

We study the joint source-channel coding (JSCC) error exponent for discrete memoryless source-channel systems with side information which is correlated to the transmitted source. Two cases are considered: (1) the side information is available only at the decoder; (2) the side information is available at both the encoder and decoder. We employ the method of types to establish a lower bound for the JSCC error exponent for each case. As a consequence, a JSCC theorem on the reliable transmissibility of the source over the channel is obtained. It is noted that the same JSCC theorem applies for both cases. For binary sources and symmetric channels, we derive a sufficient condition for which the side information at the decoder can strictly improve the JSCC error exponent. Numerical results show that side information can enlarge the region for reliable transmissibility and increase the JSCC error exponent for a wide class of source-channel parameters.

On the joint source-channel coding error exponent for systems with memory

We establish an upper bound for the joint source-channel coding (JSCC) error exponent EJ(Q, W) for a discrete stationary ergodic Markov (SEM) source Q and a discrete channel W with additive SEM noise. This bound, which is expressed in terms of the Renyi entropy rates of the source and noise processes, admits an identical form to Csiszars sphere-packing upper bound for the JSCC error exponent for memoryless systems (I. Csiszar, Nov. 1982). In this regard, our result is a natural extension of Csiszars upper bound of the JSCC error exponent from the case of memoryless systems to the case of SEM systems. We also investigate the analytical computation of EJ(Q,W) by comparing our bound with Gallagers random-coding lower bound (R. G. Gallager, 1968), when the latter one is specialized to the SEM source-channel system

On the computation of the joint source-channel error exponent for memoryless system

We study the analytical computation of Csiszars [1980] random-coding lower bound and sphere-packing upper bound for the lossless joint source-channel (JSC) error exponent, E/sub J/(Q, W), for a discrete memoryless source (DMS) Q and a discrete memoryless channel (DMC) W. We provide equivalent expressions for these bounds, which can be readily calculated for arbitrary (Q,W) pairs. We also establish explicit conditions under which the bounds coincide, thereby exactly determining E/sub J/(Q,W).