Giuseppe O. Longo

An application of informational divergence to Huffman codes

A classification of all probability distributions over the finite alphabet of an information source is given, where the classes are the sets of distributions sharing the same binary Huffman code. Such a classification can be used in noiseless coding, when the distribution of the finite memoryless source varies in time or becomes gradually known. Instead of applying the Huffman algorithm to each new estimate of the probability distribution, if a simple test based on the above classification is passed, then the Huffman code used previously is optimal also for the new distribution.

The source coding theorem revisited: A combinatorial approach

A combinatorial approach is proposed for proving the classical source coding theorems for a finite memoryless stationary source (giving achievable rates and the error probability exponent). This approach provides a sound heuristic justification for the widespread appearence of entropy and divergence (Kullbacks discrimination) in source coding. The results are based on the notion of composition class -- a set made up of all the distinct source sequences of a given length which are permutations of one another. The asymptotic growth rate of any composition class is precisely an entropy. For a finite memoryless constant source all members of a composition class have equal probability; the probability of any given class therefore is equal to the number of sequences in the class times the probability of an individual sequence in the class. The number of different composition classes is algebraic in block length, whereas the probability of a composition class is exponential, and the probability exponent is a divergence. Thus if a codeword is assigned to all sequences whose composition classes have rate less than some rate R , the probability of error is asymptotically the probability of the must probable composition class of rate greater than R . This is expressed in terms of a divergence. No use is made either of the law of large numbers or of Chebyshevs inequality.

An Introduction to Algebraic Coding Theory

The following is a short introduction to some aspects of that part of coding and decoding theory which uses algebraic tools, and is therefore called algebraic coding theory. If the channel over which information is to be transmitted from source to user is noisy, and if it is not possible to modify the channel itself, then coding and decoding can help the user receive a better (i.e. less corrupted) reproduction of the channel input. Our basic assumption will be that inform ation consists of a (potentially infinite) sequence of symbols (or digits) belonging to a finite alphabet (often binary). The input alphabet of a channel need not coincide with the output alphabet, but this is often the case. The effect of the noise is then of transforming, with some positive probability, every input symbol into a different symbol, and the mathematical description of the channel consists precisely of the set of all conditional probabilities of an output given an input. If the channel is memoryless, i.e. if past history has no influence on the current noise effect, the channel description is simplified enormously.

Adaptive signal processing

The four chapters of this volume, written by prominent workers in the field of adaptive processing and linear prediction, address a variety of problems, ranging from adaptive source coding to autoregressive spectral estimation. The first chapter, by T.C. Butash and L.D. Davisson, formulates the performance of an adaptive linear predictor in a series of theorems, with and without the Gaussian assumption, under the hypothesis that its coefficients are derived from either the (single) observation sequence to be predicted (dependent case) or a second, statistically independent realisation (independent case). The contribution by H.V. Poor reviews three recently developed general methodologies for designing signal predictors under nonclassical operating conditions, namely the robust predictor, the high-speed Levinson modeling, and the approximate conditional mean nonlinear predictor. W. Wax presents the key concepts and techniques for detecting, localizing and beamforming multiple narrowband sources by passive sensor arrays. Special coding algorithms and techniques based on the use of linear prediction now permit high-quality voice reproduction at remorably low bit rates. The paper by A. Gersho reviews some of the main ideas underlying the algorithms of major interest today.

The Dynamics of Beauty

The debate on the “two cultures” is regaining strength as we approach the 50th anniversary of Charles P. Snow’s famous lecture entitled precisely The Two Cultures. The opposition between scientific culture and humanistic culture has sometimes assumed overheated and almost grotesque tones. Actually, the relationships between the so-called two cultures are very tight and their scopes and aims are closer than one can suppose. First, they meet in, and are produced by, human beings; and, second, they share the goal of seeking adequate and meaningful representations and models of the world and of man. Although they speak different languages and one is driven by tendentially objective and communicative goals and the other by subjective and expressive goals, finally both turn out to be intersubjective. I would like to put forward some considerations on beauty in the hope that it turns out to be a bridge between the two cultures.

A Pragmatic Way Out of the Maze of Uncertainty Measures

We argue that the construction of a general (non-specific) theory of pragmatic uncertainty measures is a dubious undertaking. To make our point we discuss fractional entropy, a measure introduced in the specific context of cryptography. Two new properties of fractional entropy are proved.

Calculation of Channel Capacity

The calculation of the capacity of a given channel is a rather difficult task in general, and we shall limit the discussion to some special cases.

Source Coding Theory

In what follows we shall refer to a communication system as schematized by the block-diagram of Fig. 1.1. Such a block-diagram is actually an over-simplification for most of real communication system, but its usefulness has been largely recognized since the pioneering work of Shannon.

Sull'invarianza dei codici lineari rispetto a certe trasformazioni

In this article a code is considered as a linear subspace of the spaceVn, formed by all of then-tuples built up by means of elements fromG(A) (Galois field of orderA). Certain linear transformations are studied, which do not affect a given code: a class of tranformations of this type is looked for and characterized among the automorphisms ofVn.Then the projection operators are considered, and a possible simplification in the decoding scheme for linear codes is pointed out.SommarioIn questo lavoro si considerano alcuni tipi di trasformazioni lineari che lasciano invariato un codice, inteso come sottospazio lineare dello spazioVn di tutte len-uple costruite con elementi diG(A). Si considerano dapprima gli automorfismi diVn che lasciano invariato un codice sistematico assegnato e si trova la forma di tali trasformazioni. Si considerino poi gli operatori di proiezione, che possono portare ad una semplificazione dello schema di decodifica dei codici lineari.

Some properties of binaryn-Tuples with reference to cyclic codes

This article gives a geometric interpretation of a product (already introduced in literature) betweenn-tuples, based on the use of «associate, polynomials». In the second part of the work, a number of algebraic considerations are made, which give a characterization of the ringsRn of all binaryn-tuples.