Joy A. Thomas

Information theoretic inequalities

The role of inequalities in information theory is reviewed, and the relationship of these inequalities to inequalities in other branches of mathematics is developed. The simple inequalities for differential entropy are applied to the standard multivariate normal to furnish new and simpler proofs of the major determinant inequalities in classical mathematics. The authors discuss differential entropy inequalities for random subsets of samples. These inequalities when specialized to multivariate normal variables provide the determinant inequalities that are presented. The authors focus on the entropy power inequality (including the related Brunn-Minkowski, Youngs, and Fisher information inequalities) and address various uncertainty principles and their interrelations. >

Effective bandwidth in high-speed digital networks

The theory of large deviations provides a simple unified basis for statistical mechanics, information theory and queueing theory. The objective of this paper is to use large deviation theory and the Laplace method of integration to provide an simple intuitive overview of the theory of effective bandwidth for high-speed digital networks, especially ATM networks. This includes (1) identification of the appropriate energy function, entropy function and effective bandwidth function of a source, (2) the calculus of the effective bandwidth functions, (3) bandwidth allocation and buffer management, (4) traffic descriptors, and (5) envelope processes and conjugate processes for fast simulation and bounds. >

Feedback can at most double Gaussian multiple access channel capacity (Corresp.)

The converse for the discrete memoryless multiple access channel is generalized and is used to derive strong bounds on the total capacity (sum of the rates of all the senders) of an m -user Gaussian multiple access channel in terms of the input covariance matrix. These bounds are used to show that the total capacity of the channel with feedback is less than twice the total capacity without feedback. The converse for the general multiple access channel is also used to show that for any m -user multiple access channel, feedback cannot increase the total capacity by more than a factor of m .

Parallel compression with cooperative dictionary construction

It is often desirable to compress or decompress relatively small blocks of data at high bandwidth and low latency (for example, for data fetches across a high speed network). Sequential compression may not satisfy the speed requirement, while simply splitting the block into smaller subblocks for parallel compression yields poor compression performance due to small dictionary sizes. We consider an intermediate approach, where multiple compressors jointly construct a dictionary. The result is parallel speedup, with compression performance similar to the sequential case.

Determinant inequalities via information theory

Simple inequalities from information theory prove Hadamards inequality and some of its gen- eralizations. It is also proven that the determinant ofa positive definite matrix is log-concave and that the ratio ofthe determinant ofthe matrix to the determinant of its principal minor g, I/Ig,- 1 is concave, establishing the concavity of minimum mean squared error in linear prediction. For Toeplitz matrices, the normalized determinant g, TM is shown to decrease with n.

On the stability of open networks: A unified approach by stochastic dominance

Using stochastic dominance, in this paper we provide a new characterization of point processes. This characterization leads to a unified proof for various stability results of open Jackson networks where service times are i.i.d. with a general distribution, external interarrivai times are i.i.d. with a general distribution and the routing is Bernoulli. We show that if the traffic condition is satisfied, i.e., the input rate is smaller than the service rate at each queue, then the queue length process (the number of customers at each queue) is tight. Under the traffic condition, the pth moment of the queue length process is bounded for allt if the p+lth moment of the service times at all queues are finite. If, furthermore, the moment generating functions of the service times at all queues exist, then all the moments of the queue length process are bounded for allt. When the interarrivai times are unbounded and non-lattice (resp. spreadout), the queue lengths and the remaining service times converge in distribution (resp. in total variation) to a steady state. Also, the moments converge if the corresponding moment conditions are satisfied.

On the Shannon capacity of discrete time queues

Motivated by the analysis of high speed packet switched networks like ATM networks and the work on bits through queues, we analyze the information theoretic capacity of a discrete time queue. We assume that time is divided into slots and a random number of identical packets arrive during each slot. A server serves a random number of packets in a slot. We consider various cases of this channel, some of which are analogous to the continuous time case considered by Ananthram and Verdu (see IEEE Trans. Inform. Theory, vol.IT-42, p.4-18, 1996) and others which show some interesting new features.

Context allocation with application to data compression

Given a sequence of correlated symbols of a finite alphabet, the context or history of a substring (the sequence of symbols preceding the substring) provides information about likely future symbols for applications such as universal data compression or prediction. In this paper, we describe an approach to calculating optimal context and provide an application to a generalization of Lempel-Ziv coding.<<ETX>>

Universal Source Coding

2 Arithmetic Codes 3 2.1 Arithmetic Coding . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.3 Interval division . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.4 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.5 Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.6 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

On the properties of capacity regions in multi-user networks

Motivated by work by Yeung (1994) on the convexity of capacity regions for a multi-user channel, we formulate the general properties of the capacity region for any network of users. In particular, in addition to the usual properties of dominance and convexity, we also consider the bit masking property that a message could be sent to a larger number of receivers than necessary. We derive a general inner bound on the capacity region of a multiuser network without feedback using these properties.