Shashi Borade

Euclidean Information Theory

Many problems in information theory involve optimizing the Kullback-Leibler (KL) divergence between probability distributions. Since KL divergence is difficult to analyze, these optimizations are often intractable. We simplify these problems by assuming the distributions of interest to be close to each other. Under this assumption, the KL divergence behaves like a squared Euclidean distance. With this simplification, we solve the open problem of broadcasting with degraded message sets, as a canonical example of network information theory problems.

some fundamental limits of unequal error protection

Various formulations are considered where some information is more important than other and needs better protection. Our information theoretic framework in terms of exponential error bounds provides some fundamental limits and optimal strategies for such problems of unequal error protection. Even for data-rates approaching the channel capacity, it shows how a crucial part of information can be protected with exponential reliability. Channels without feedback are analyzed first, which is useful later in analyzing channels with feedback. A new channel parameter, called the Red-Alert Exponent, is fundamentally important in such problems.

Writing on Fading Paper, Dirty Tape With Little Ink: Wideband Limits for Causal Transmitter CSI

A wideband Rayleigh fading channel is considered with causal channel state information (CSI) at the transmitter and no receiver CSI. A simple orthogonal code with energy detection rule at the receiver (similar to pulse position modulation in IEEE Trans. Inf. Theory, vol. 46, no. 4, Apr. 2000 and IEEE Trans. Inf. Theory, vol. 52 no. 5, May 2006) is shown to achieve the capacity of this channel in the wideband limit. This strategy transmits energy only when the channel gain exceeds a threshold, hence only needs causal transmitter CSI. In the wideband limit, this capacity without any receiver CSI is the same as the capacity with full receiver CSI, which is proportional to the logarithm of the bandwidth. Similar threshold-based pulse position modulation is shown to achieve the capacity per unit cost of the dirty-tape channel (dirty paper channel with causal transmitter CSI and no receiver CSI), which equals its capacity per unit cost with full receiver CSI. Then, a general discrete channel with i.i.d. states is considered. Each input has an associated cost and a zero cost input “0” exists. The channel state is assumed to be known at the transmitter in a causal manner. Capacity per unit cost is found for this channel and a simple orthogonal code is shown to achieve this capacity. Later, a novel orthogonal coding scheme is proposed for the case of causal transmitter CSI and a condition for equivalence of capacity per unit cost for causal and noncausal transmitter CSI is derived.

Some fundamental coding theoretic limits of unequal error protection

This paper investigates asymptotic (in blocklength) tradeoffs between rate and minimum distance, for codes that provide unequal error protection (UEP). Two notions of UEP are analyzed: bit-wise, where a subset of bits is special and needs more protection, and message-wise, where a subset of the message-set is special. Both notions are analyzed for two cases: binary, and large-alphabet. In message-wise UEP for the binary channel alphabet, it turns out that the special messages and ordinary messages can simultaneously achieve the Gilbert-Varshamov bound at their respective rates. Similar “successive refinement” of the Singleton bound is shown to hold for large non-binary alphabets. We also analyze the situation when there is only one special message. In bit-wise UEP, it is shown that when the ordinary bits are achieving the Singleton bound, even a single special bit cannot achieve any larger distance. For the binary case, an upper bound is provided on the protection of the single special bit. These coding theoretic limits in terms of Hamming distances are close analogues of the information theoretic limits [1], [2] in terms of error exponents.