Mladen Kovačević

Zero-Error Capacity of a Class of Timing Channels

We analyze the problem of zero-error communication through timing channels that can be interpreted as discrete-time queues with bounded waiting times. The channel model includes the following assumptions: 1) time is slotted; 2) at most N particles are sent in each time slot; 3) every particle is delayed in the channel for a number of slots chosen randomly from the set {0, 1, ... , K}; and 4) the particles are identical. It is shown that the zero-error capacity of this channel is log r, where r is the unique positive real root of the polynomial xK+1-xK-N. Capacity-achieving codes are explicitly constructed, and a linear-time decoding algorithm for these codes devised. In the particular case N = 1, K = 1, the capacity is equal to φ, where φ = (1 + √5)/2 is the golden ratio, and constructed codes give another interpretation of the Fibonacci sequence.

On the entropy of couplings

Abstract In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be NP-hard, and their special cases are found to be essentially information-theoretic restatements of well-known computational problems, such as the Subset sum and the 3-Partition . The notion of minimum entropy coupling is introduced and its relevance is demonstrated in information-theoretic, computational, and statistical contexts. Finally, a family of pseudometrics (on the space of discrete probability distributions) defined by these couplings is studied, in particular their relation to the total variation distance, and a new characterization of the conditional entropy is given.

On the hardness of entropy minimization and related problems

We investigate certain optimization problems for Shannon information measures, namely, minimization of joint and conditional entropies H(X, Y), H(X|Y), H(Y|X), and maximization of mutual information I(X; Y), over convex regions. When restricted to the so-called transportation polytopes (sets of distributions with fixed marginals), very simple proofs of NP-hardness are obtained for these problems because in that case they are all equivalent, and their connection to the well-known SUBSET SUM and PARTITION problems is revealed. The computational intractability of the more general problems over arbitrary polytopes is then a simple consequence. Further, a simple class of polytopes is shown over which the above problems are not equivalent and their complexity differs sharply, namely, minimization of H(X, Y) and H(Y|X) is trivial, while minimization of H(X|Y) and maximization of I(X; Y) are strongly NP-hard problems. Finally, two new (pseudo)metrics on the space of discrete probability distributions are introduced, based on the so-called variation of information quantity, and NP-hardness of their computation is shown.

Subset Codes for Packet Networks

In this paper, we present a coding-theoretic framework for message transmission over packet-switched networks. Network is modeled as a channel which can induce packet errors, deletions, insertions, and out of order delivery of packets. The proposed approach can be viewed as an extension of the one introduced by Kotter and Kschischang for networks based on random linear network coding. Namely, while their framework is based on subspace codes and designed for networks in which network nodes perform random linear combining of the packets, ours is based on the so-called subset codes, and is designed for networks employing routing in network nodes.

Perfect codes in the discrete simplex

We study the problem of existence of (nontrivial) perfect codes in the discrete

A Note on Parallel Asynchronous Channels With Arbitrary Skews

n

Coding for the permutation channel with insertions, deletions, substitutions, and erasures

n-simplex