Marcelo Weinberger

Channel decoding of systematically encoded unknown redundant sources

This paper describes the channel decoding of systematically encoded unknown redundant sources. The redundancy of the data is known at the decoder and the channel decoder incorporates the statistics of the data to enhance the performance. The practical decoders are designed which takes the advantage of the source redundancy of systematically encoded for transmission over a discrete memoryless channel (DMC). The performance is achieved by operating discrete universal denoiser (DUDE) and the experiments involving Reed-Solomon codes show that DUDE-enhanced decoding is very effective at high rates.

On modeling and ordering for embedded image coding

We present an information-theoretic framework for the optimization of the order in which embedded bit-plane coders encode image data.

Optimal Algorithms for Universal Random Number Generation From Finite Memory Sources

We study random number generators (RNGs), both in the fixed to variable-length (FVR) and the variable to fixed-length (VFR) regimes, in a universal setting in which the input is a finite memory source of arbitrary order and unknown parameters, with arbitrary input and output (finite) alphabet sizes. Applying the method of types, we characterize essentially unique optimal universal RNGs that maximize the expected output (respectively, minimize the expected input) length in the FVR (respectively, VFR) case. For the FVR case, the RNG studied is a generalization of Eliass scheme, while in the VFR case the general scheme is new. We precisely characterize, up to an additive constant, the corresponding expected lengths, which include second-order terms similar to those encountered in universal data compression and universal simulation. Furthermore, in the FVR case, we consider also a twice-universal setting, in which the Markov order k of the input source is also unknown.

One-to-one lossless codes in the variable input-length regime: Back to Kraft's inequality

Unique decodability in the “one-shot” lossless coding scenario, where a single block of source samples is compressed, requires the assignment of distinct codewords to different blocks (one-to-one mapping), without the prefix constraint. As a result, for fixed-length blocks, the corresponding block entropy is not a lower bound on the expected code length, a fact that has recently attracted renewed interest. In this note, we consider an alternative scenario, where the encoder is fed with blocks of arbitrary length, which we argue better reflects the conditions under which one-shot codes may be of any interest. Elaborating on an argument by Rissanen, we first show that the block-entropy is still a fundamental performance bound for one-to-one codes. We then design a code that essentially achieves this bound and satisfies Krafts inequality for each block length. This code can be implemented with a modification to the termination procedure of the popular Shannon-Fano-Elias code. We conclude that Krafts inequality is relevant also in the one-shot coding scenario.

On the use of randomized experts in sequential strategies for loss functions with memory

The sequential decision problem for loss functions with memory is extended to cover the infinite class of randomized finite-state reference strategies. After showing the necessity of this extension, an on-line strategy is devised for which the normalized regret over an arbitrary sequence of observations of length n is O([(ln n)/n]/sup 1/3/).