Nir Weinberger

Codeword or Noise? Exact Random Coding Exponents for Joint Detection and Decoding

We consider the problem of coded communication, where in each time frame, the transmitter is either silent or transmits a codeword from a given (randomly selected) codebook. The task of the decoder is to decide whether transmission has taken place, and if so, to decode the message. We derive the optimum detection/decoding rule in the sense of the best tradeoff among the probabilities of decoding error, false alarm, and misdetection. For this detection/decoding rule, we then derive single-letter characterizations of the exact exponential rates of these probabilities for the average code in the ensemble. It is shown that previously proposed decoders are in general strictly suboptimal.

Optimum Tradeoffs Between the Error Exponent and the Excess-Rate Exponent of Variable-Rate Slepian–Wolf Coding

We analyze the optimal tradeoff between the error exponent and the excess-rate exponent for variable-rate Slepian-Wolf codes. In particular, we first derive upper (converse) bounds on the optimal error and excess-rate exponents, and then lower (achievable) bounds, via a simple class of variable-rate codes which assign the same rate to all source blocks of the same type class. Then, using the exponent bounds, we derive bounds on the optimal rate functions, namely, the minimal rate assigned to each type class, needed in order to achieve a given target error exponent. The resulting excess-rate exponent is then evaluated. Iterative algorithms are provided for the computation of both bounds on the optimal rate functions and their excess-rate exponents. The resulting Slepian-Wolf codes bridge between the two extremes of fixed-rate coding, which has minimal error exponent and maximal excess-rate exponent, and average-rate coding, which has maximal error exponent and minimal excess-rate exponent.

Erasure/List Random Coding Error Exponents Are Not Universally Achievable

We study the problem of universal decoding for unknown discrete memoryless channels in the presence of erasure/list option at the decoder, in the random coding regime. In particular, we harness a universal version of Forney’s classical erasure/list decoder developed in earlier studies, which is based on the competitive minimax methodology, and guarantees universal achievability of a certain fraction of the optimum random coding error exponents. In this paper, we derive an exact single-letter expression for the maximum achievable fraction. Examples are given in which the maximal achievable fraction is strictly less than unity, which imply that, in general, there is no universal erasure/list decoder, which achieves the same random coding error exponents as the optimal decoder for a known channel. This is in contrast to the situation in ordinary decoding (without the erasure/list option), where optimum exponents are universally achievable, as is well known. It is also demonstrated that previous lower bounds derived for the maximal achievable fraction are not tight in general. We then analyze a generalized random coding ensemble, which incorporate a training sequence, in conjunction with a suboptimal practical decoder (“plug-in” decoder), which first estimates the channel using the available training sequence, and then decodes the remaining symbols of the codeword using the estimated channel. One of the implications of our results is setting the stage for a reasonable criterion of optimal training. Finally, we compare the performance of the “plug-in” decoder and the universal decoder, in terms of the achievable error exponents, and show that the latter is noticeably better than the former.

A large deviations approach to secure lossy compression

A Shannon cipher system for memoryless sources is considered, in which distortion is allowed at the legitimate decoder. The source is compressed using a rate distortion code secured by a shared key, which satisfies a constraint on the compression rate, as well as a constraint on the exponential rate of the excess-distortion probability at the legitimate decoder. Secrecy is measured by the exponential rate of the exiguous-distortion probability at the eavesdropper, rather than by the traditional measure of equivocation. The perfect secrecy exponent is defined as the maximal exiguous-distortion exponent achievable when the key rate is unlimited. Under limited key rate, it is proved that the maximal achievable exiguous-distortion exponent is equal to the minimum between the average key rate and the perfect secrecy exponent, for a fairly general class of variable key rate codes.

A Large Deviations Approach to Secure Lossy Compression

A Shannon cipher system for memoryless sources in which distortion is allowed at the legitimate decoder is considered. The source is compressed using a secured rate distortion code, which satisfies a constraint on the compression rate, as well as a constraint on the exponential rate of the excess-distortion probability at the legitimate decoder. Secrecy is measured by the exponential rate of the exiguous-distortion probability at the eavesdropper, rather than by the traditional measure of equivocation. The perfect-secrecy exponent is defined as the maximal exiguous-distortion exponent achievable when the key rate is unlimited. The reproduction-based estimate exponent is defined as the maximal exiguous-distortion exponent achievable for a genie-aided eavesdropper, which knows the secret key. Under limited key rate, it is proved that the maximal achievable exiguous-distortion exponent is equal to the minimum between the key rate plus the reproduction-based estimate exponent, and the perfect-secrecy exponent. The result is generalized to a fairly general class of variable key-rate and coding-rate codes.

Codeword or noise? Exact random coding exponents for slotted asynchronism

We consider the problem of slotted asynchronous coded communication, where in each time frame (slot), the transmitter is either silent or transmits a codeword from a given (randomly selected) codebook. The task of the decoder is to decide whether transmission has taken place, and if so, to decode the message. We derive the optimum detection/decoding rule in the sense of the best trade-off among the probabilities of decoding error, false alarm, and misdetection. For this detection/decoding rule, we then derive single-letter characterizations of the exact exponential rates of these three probabilities for the average code in the ensemble. It is shown that previously suggested decoders care in general strictly sub-optimal.

Universal decoding for linear Gaussian fading channels in the competitive minimax sense

We address the problem of communicating over an unknown linear fading channel with additive white Gaussian noise, in the high SNR regime. A block fading model is adopted where the channel fading vector is unknown, yet assumed constant during the block. For a given codebook, a competitive minimax criterion is used to find a decoder ignorant of the specific channel fading prevailing, yet its performance, relative to the Maximum Likelihood decoder, has the best worst case. For a codebook with two codewords, the decoder is found explicitly, and a numerical method is described to find its performance.

Channel Detection in Coded Communication

The problem of block-coded communication where in each block the channel law belongs to one of two disjoint sets is considered. The decoder is aimed to decode only messages that have undergone a channel from one of the sets, and thus has to detect the set which contains the underlying channel. The simplified case where each of the sets is a singleton is studied first. The decoding error, false alarm, and misdetection probabilities of a given code are defined, and the optimum detection/decoding rule in a generalized Neyman-Pearson sense is derived. Sub-optimal detection/decoding rules are also introduced which are simpler to implement. Then, various achievable bounds on the error exponents are derived, including the exact single-letter characterization of the random coding exponents for the optimal detector/decoder. The random coding analysis is then extended to general sets of channels, and an asymptotically optimal detector/decoder under a worst case formulation of the error probabilities is derived, as well as its random coding exponents. The case of a pair of binary symmetric channels is discussed in detail.

Simplified erasure/list decoding

We consider the problem of erasure/list decoding using certain classes of simplified decoders. Specifically, we assume a class of erasure/list decoders, such that a codeword is in the list if its likelihood is larger than a threshold. This class of decoders both approximates the optimal decoder of Forney, and also includes the following simplified subclasses of decoding rules: The first is a function of the output vector only, but not the codebook (which is most suitable for high rates), and the second is a scaled version of the maximum likelihood decoder (which is most suitable for low rates). We provide singleletter expressions for the exact random coding exponents of any decoder in these classes, operating over a discrete memoryless channel. For each class of decoders, we find the optimal decoder within the class, in the sense that it maximizes the erasure/list exponent, under a given constraint on the error exponent. We establish the optimality of the simplified decoders of the first and second kind for low and high rates, respectively.

Erasure/list random coding error exponents are not universally achievable

We study the problem of universal decoding for unknown discrete memoryless channels in the presence of erasure/list option at the decoder, in the random coding regime. In particular, we harness a universal version of Forneys classical erasure/list decoder developed in earlier studies, which is based on the competitive minimax methodology, and guarantees universal achievability of a certain fraction of the optimum random coding error exponents. In this paper, we derive an exact single-letter expression for the maximum achievable fraction. Examples are given in which the maximal achievable fraction is strictly less than unity, which imply that, in general, there is no universal erasure/list decoder, which achieves the same random coding error exponents as the optimal decoder for a known channel. This is in contrast to the situation in ordinary decoding (without the erasure/list option), where optimum exponents are universally achievable, as is well known. It is also demonstrated that previous lower bounds derived for the maximal achievable fraction are not tight in general. We then analyze a generalized random coding ensemble, which incorporate a training sequence, in conjunction with a suboptimal practical decoder (“plug-in” decoder), which first estimates the channel using the available training sequence, and then decodes the remaining symbols of the codeword using the estimated channel. One of the implications of our results is setting the stage for a reasonable criterion of optimal training. Finally, we compare the performance of the “plug-in” decoder and the universal decoder, in terms of the achievable error exponents, and show that the latter is noticeably better than the former.