Baris Nakiboglu

Variations on a Theme by Schalkwijk and Kailath

Schalkwijk and Kailath (1966) developed a class of block codes for Gaussian channels with ideal feedback for which the probability of decoding error decreases as a second-order exponent in block length for rates below capacity. This well-known but surprising result is explained and simply derived here in terms of a result by Elias (1956) concerning the minimum mean-square distortion achievable in transmitting a single Gaussian random variable over multiple uses of the same Gaussian channel. A simple modification of the Schalkwijk-Kailath scheme is then shown to have an error probability that decreases with an exponential order which is linearly increasing with block length. In the infinite bandwidth limit, this scheme produces zero error probability using bounded expected energy at all rates below capacity. A lower bound on error probability for the finite bandwidth case is then derived in which the error probability decreases with an exponential order which is linearly increasing in block length at the same rate as the upper bound.

A Simple Converse of Burnashev's Reliability Function

In a remarkable paper published in 1976, Burnashev determined the reliability function of variable-length block codes over discrete memoryless channels (DMCs) with feedback. Subsequently, an alternative achievability proof was obtained by Yamamoto and Itoh via a particularly simple and instructive scheme. Their idea is to alternate between a communication and a confirmation phase until the receiver detects the codeword used by the sender to acknowledge that the message is correct. We provide a converse that parallels the Yamamoto-Itoh achievability construction. Besides being simpler than the original, the proposed converse suggests that a communication and a confirmation phase are implicit in any scheme for which the probability of error decreases with the largest possible exponent. The proposed converse also makes it intuitively clear why the terms that appear in Burnashevs exponent are necessary.

Error Exponents for Variable-Length Block Codes With Feedback and Cost Constraints

Variable-length block-coding schemes are investigated for discrete memoryless channels with ideal feedback under cost constraints. Upper and lower bounds are found for the minimum achievable probability of decoding error Pe,min as a function of constraints R, P, and tau on the transmission rate, average cost, and average block length, respectively. For given R and P, the lower and upper bounds to the exponent -( ln Pe,min )/tau are asymptotically equal as tau rarr infin. The resulting reliability function,limtaurarrinfin(-In Pe,min)/tau as a function of R and V, is concave in the pair (R,P) and generalizes the linear reliability function of Burnashev to include cost constraints. The results are generalized to a class of discrete-time memoryless channels with arbitrary alphabets, including additive Gaussian noise channels with amplitude and power constraints.

some fundamental limits of unequal error protection

Various formulations are considered where some information is more important than other and needs better protection. Our information theoretic framework in terms of exponential error bounds provides some fundamental limits and optimal strategies for such problems of unequal error protection. Even for data-rates approaching the channel capacity, it shows how a crucial part of information can be protected with exponential reliability. Channels without feedback are analyzed first, which is useful later in analyzing channels with feedback. A new channel parameter, called the Red-Alert Exponent, is fundamentally important in such problems.

Errors-and-Erasures Decoding for Block Codes With Feedback

Inner and outer bounds are derived on the optimal performance of fixed-length block codes on discrete memoryless channels with feedback and errors-and-erasures decoding. First, an inner bound is derived using a two-phase encoding scheme with communication and control phases together with the optimal decoding rule for the given encoding scheme, among decoding rules that can be represented in terms of pairwise comparisons between the messages. Then, an outer bound is derived using a generalization of the straight-line bound to errors-and-erasures decoders and the optimal error-exponent tradeoff of a feedback encoder with two messages. In addition, upper and lower bounds are derived, for the optimal erasure exponent of error-free block codes in terms of the rate. Finally, a proof is provided for the fact that the optimal tradeoff between error exponents of a two-message code does not improve with feedback on discrete memoryless channels (DMCs).

Error Exponents for Variable-length block codes with feedback and cost constraints

Variable-length block-coding schemes are investigated for discrete memoryless channels (DMC) with perfect feedback under cost constraints. Upper and lower bounds are found for the minimum achievable probability of decoding error Pepsi,min as a function of transmission rate R, cost constraint P, and expected block length taumacr. For given P and R, the lower and upper bounds to the exponent -(InPepsi,min)/taumacr are asymptotically equal as taumacr rarrinfin. The reliability function, limtau rarrinfin(-ln Pepsi,min)/taumacr, as a function of P and R, is concave in the pair (P, R) and generalizes the linear reliability function of Burnashev (M.V. Burnashev, 1976) to include cost constraints

Bit-wise unequal error protection for variable length blockcodes with feedback

Bit-wise unequal error protection problem with two layers is considered for variable length block-codes with feedback. Inner and outer bounds are derived for achievable performance for finite expected decoding time. These bounds completely characterize the error exponent of the special bits as a function of overall rate R, overall error exponent E and the rate of the special bits R s . Single message Message-wise unequal protection problem is also solved as a step on the way.

Errors-and-erasures decoding for block codes with feedback

Fixed length block codes on discrete memoryless channels with feedback are considered for errors and erasures decoding. Upper and lower bounds are derived for the error exponent in terms of the rate and the erasure exponents. In addition the converse result of Burnashev for variable length block codes is extended to include list decoding.

Bit-Wise Unequal Error Protection for Variable-Length Block Codes With Feedback

The bit-wise unequal error protection problem, for the case when the number of groups of bits l is fixed, is considered for variable-length block codes with feedback. An encoding scheme based on fixed-length block codes with erasures is used to establish inner bounds to the achievable performance for finite expected decoding time. A new technique for bounding the performance of variable-length block codes is used to establish outer bounds to the performance for a given expected decoding time. The inner and the outer bounds match one another asymptotically and characterize the achievable region of rate-exponent vectors, completely. The single-message message-wise unequal error protection problem for variable-length block codes with feedback is also solved as a necessary step on the way.

The Augustin center and the sphere packing bound for memoryless channels

For any channel with a convex constraint set and finite Augustin capacity, existence of a unique Augustin center and associated Erven-Harremoes bound are established. Augustin-Legendre capacity, center, and radius are introduced and proved to be equal to the corresponding Renyi-Gallager entities. Sphere packing bounds with polynomial prefactors are derived for codes on two families of channels: (possibly non-stationary) memoryless channels with multiple additive cost constraints and stationary memoryless channels with convex constraints on the empirical distribution of the input codewords.