L. Lorne Campbell

The Kullback-Leibler divergence rate between Markov sources

In this work, we provide a computable expression for the Kullback-Leibler divergence rate lim/sub n/spl rarr//spl infin//1/nD(p/sup (n)//spl par/q/sup (n)/) between two time-invariant finite-alphabet Markov sources of arbitrary order and arbitrary initial distributions described by the probability distributions p/sup (n)/ and q/sup (n)/, respectively. We illustrate it numerically and examine its rate of convergence. The main tools used to obtain the Kullback-Leibler divergence rate and its rate of convergence are the theory of nonnegative matrices and Perron-Frobenius theory. Similarly, we provide a formula for the Shannon entropy rate lim/sub n/spl rarr//spl infin//1/nH(p/sup (n)/) of Markov sources and examine its rate of convergence.

On the joint source-channel coding error exponent for discrete memoryless systems

We investigate the computation of Csisza/spl acute/rs bounds for the joint source-channel coding (JSCC) error exponent E/sub J/ of a communication system consisting of a discrete memoryless source and a discrete memoryless channel. We provide equivalent expressions for these bounds and derive explicit formulas for the rates where the bounds are attained. These equivalent representations can be readily computed for arbitrary source-channel pairs via Arimotos algorithm. When the channels distribution satisfies a symmetry property, the bounds admit closed-form parametric expressions. We then use our results to provide a systematic comparison between the JSCC error exponent E/sub J/ and the tandem coding error exponent E/sub T/, which applies if the source and channel are separately coded. It is shown that E/sub T//spl les/E/sub J//spl les/2E/sub T/. We establish conditions for which E/sub J/>E/sub T/ and for which E/sub J/=2E/sub T/. Numerical examples indicate that E/sub J/ is close to 2E/sub T/ for many source-channel pairs. This gain translates into a power saving larger than 2 dB for a binary source transmitted over additive white Gaussian noise (AWGN) channels and Rayleigh-fading channels with finite output quantization. Finally, we study the computation of the lossy JSCC error exponent under the Hamming distortion measure.

Renyi's divergence and entropy rates for finite alphabet Markov sources

In this work, we examine the existence and the computation of the Renyi divergence rate, lim/sub n/spl rarr//spl infin// 1/n D/sub /spl alpha//(p/sup (n)//spl par/q/sup (n)/), between two time-invariant finite-alphabet Markov sources of arbitrary order and arbitrary initial distributions described by the probability distributions p/sup (n)/ and q/sup (n)/, respectively. This yields a generalization of a result of Nemetz (1974) where he assumed that the initial probabilities under p/sup (n)/ and q/sup (n)/ are strictly positive. The main tools used to obtain the Renyi divergence rate are the theory of nonnegative matrices and Perron-Frobenius theory. We also provide numerical examples and investigate the limits of the Renyi divergence rate as /spl alpha//spl rarr/1 and as /spl alpha//spl darr/0. Similarly, we provide a formula for the Renyi entropy rate lim/sub n/spl rarr//spl infin// 1/n H/sub /spl alpha//(p/sup (n)/) of Markov sources and examine its limits as /spl alpha//spl rarr/1 and as /spl alpha//spl darr/0. Finally, we briefly provide an application to source coding.

Infinite series of interference variables with Cantor-type distributions

The sum of an infinite series of weighted binary random variables arises in communications problems involving intersymbol and adjacent-channel interference. If the weighting decays asymptotically at least exponentially and if the decay is not too slow, the sum has an unusual distribution which has neither a density nor a discrete mass function, and therefore cannot be manipulated with usual techniques. The distribution of the sum is given, and the calculus for dealing with the distribution is presented. It is shown that these Cantor-type random variables arise in a range of digital communications models, and exact explicit expressions for performance measures, such as the probability of error, may be obtained. Several examples are given. >

Equilibrium probability calculations for a discrete-time bulk queue model

Many problems in management science and telecommunications can be solved by the analysis of aDX/Dm/1 queueing model. In this paper, we use the zeros, both inside and outside the unit circle, of the denominator of the generating function of the model to obtain an explicit closed-form solution for the equilibrium probabilities of the number of customers in the system. The moments of the number of customers in the queue or in the system are also studied. When there are infinitely many zeros outside the unit circle, we propose an approximation method using polynomials. This method yields correct values for a finite number of the probabilities, the number depending on the degree of the polynomial approximation.

Entropy as a measure

A probability space of a special type is put into correspondence with a measure space. Under this correspondence, sets in the measure space correspond to partitions of the probability space and the measure of a set equals the entropy of the corresponding partition.

Joint Source–Channel Coding Error Exponent for Discrete Communication Systems With Markovian Memory

We study the error exponent, E_J, for reliably transmitting a discrete stationary ergodic Markov (SEM) source Q over a discrete channel W with additive SEM noise via a joint source-channel (JSC) code. We first establish an upper bound for E_J in terms of the Renyi entropy rates of the source and noise processes. We next investigate the analytical computation of E_J by comparing our bound with Gallagers lower bound (1968) when the latter one is specialized to the SEM source-channel system. We also note that both bounds can be represented in Csiszars form (1980), as the minimum of the sum of the source and channel error exponents. Our results provide us with the tools to systematically compare E_J with the tandem (separate) coding exponent E_J. We show that as in the case of memoryless source-channel pairs E_J les 2E_r and we provide explicit conditions for which E_J > E_T. Numerical results indicate that E_J ap 2E_T for many SEM source-channel pairs, hence illustrating a substantial advantage of JSC coding over tandem coding for systems with Markovian memory.

Performance analysis of a multibeam packet satellite system using random access techniques

Abstract A multibeam satellite system with on-board processing and memory is studied here. In this system, multiple slotted ALOHA up-links carry the traffic from spatially disjoint earth zones to the satellite. Packets are accepted at the satellite if memory is available and are routed to their destination zones. The model considered here will allow more than one transponder to serve a destination zone in each time slot. This is different from the model considered by Chlamtac and Ganz (1986), in which each zone is served by at most a single transponder. When the restriction of Chlamtac and Ganz to conflict-free scheduling is relaxed, the maximum throughput is increased by as much as 40%.

Error Exponents for Asymmetric Two-User Discrete Memoryless Source-Channel Systems

Consider transmitting two discrete memoryless correlated sources, consisting of a common and a private source, over a discrete memoryless multi-terminal channel with two transmitters and two receivers. At the transmitter side, the common source is observed by both encoders but the private source can only be accessed by one encoder. At the receiver side, both decoders need to reconstruct the common source, but only one decoder needs to reconstruct the private source. We hence refer to this system by the asymmetric 2-user source-channel system. In this work, we derive a universally achievable joint source-channel coding (JSCC) error exponent pair for the 2-user system by using a technique which generalizes Csiszars method (1980) for the point- to-point (single-user) discrete memoryless source-channel system. We next investigate the largest convergence rate of asymptotic exponential decay of the system (overall) probability of erroneous transmission, i.e., the system JSCC error exponent. We obtain lower and upper bounds for the exponent. As a consequence, we establish the JSCC theorem with single letter characterization.

Error probabilities on fading channels with intersymbol interference and noise

Sums of infinite sequences of weighted binary random variables arise in communications problems involving signal-dependent interferences. In many cases of practical importance, the distribution functions of these sums are singular and often of Cantor type; they are continuous but do not have a density function. For this reason, special methods of calculating expectations are needed. Results of this type are derived. The method is used to compute error probabilities for differential detection of minimum shift keying, and for noncoherent detection of frequency shift keying. In each case the model assumed is a Rician fading channel. >