Glen G. Langdon

An introduction to arithmetic coding

Arithmetic coding is a data compression technique that encodes data (the data string) by creating a code string which represents a fractional value on the number line between 0 and 1. The coding algorithm is symbolwise recursive; i.e., it operates upon and encodes (decodes) one data symbol per iteration or recursion. On each recursion, the algorithm successively partitions an interval of the number line between 0 and 1, and retains one of the partitions as the new interval. Thus, the algorithm successively deals with smaller intervals, and the code string, viewed as a magnitude, lies in each of the nested intervals. The data string is recovered by using magnitude comparisons on the code string to recreate how the encoder must have successively partitioned and retained each nested subinterval. Arithmetic coding differs considerably from the more familiar compression coding techniques, such as prefix (Huffman) codes. Also, it should not be confused with error control coding, whose object is to detect and correct errors in computer operations. This paper presents the key notions of arithmetic compression coding by means of simple examples.

Universal modeling and coding

The problems arising in the modeling and coding of strings for compression purposes are discussed. The notion of an information source that simplifies and sharpens the traditional one is axiomatized, and adaptive and nonadaptive models are defined. With a measure of complexity assigned to the models, a fundamental theorem is proved which states that models that use any kind of alphabet extension are inferior to the best models using no alphabet extensions at all. A general class of so-called first-in first-out (FIFO) arithmetic codes is described which require no alphabet extension devices and which therefore can be used in conjunction with the best models. Because the coding parameters are the probabilities that define the model, their design is easy, and the application of the code is straightforward even with adaptively changing source models.

A simple general binary source code (Corresp.)

A source code for binary strings, admitting a simple and fast hardware implementation, is described. The code is an arithmetic code, and it is capable of encoding strings modeled by stationary or nonstationary sources alike without use of alphabet extension. In particular, in the case with a stationary independent information source, the code degenerates to a bitwise implementation of Golombs run-length code.

Parameter reduction and context selection for compression of gray-scale images

In the compression of multilevel (color or gray) image data, effective compression is obtained economically by judicial selection of the predictor and the conditioning states or contexts which determine what probability distribution to use for the prediction error. We provide a cost-effective approach to the following two problems: (1) to reduce the number of coding parameters to describe a distribution when several contexts are involved, and (2) to choose contexts for which variations in prediction error distributions are expected. We solve Problem 1 (distribution description) by a partition of the range of values of the outcomes into equivalence classes, called buckets. The result is a special decomposition of the error range. Cost-effectiveness is achieved by using the many contexts only to predict the bucket (equivalence class) probabilities. The probabilities of the value within the bucket are assumed to be independent of the context, thus enormously reducing the number of coding parameters involved. We solve Problem 2 (economical contexts) by using the buckets of the surrounding pixels as components of the conditioning class. The bucket values have the desirable properties needed for the error distributions.

A note on the Ziv - Lempel model for compressing individual sequences (Corresp.)

The Ziv-Lempel compression algorithm is a string matching and parsing approach to data compression. The symbolwise equivalent for parsing models has been defined by Rissanen and Langdon and gives the same ideal codelength at the same cost in coding parameters. By describing the context and coding parameter for each symbol an insight is provided into how the Ziv-Lempel method achieves compression. This treatment does not employ a probabilistic source for the data string. The Ziv-Lempel method effectively counts symbol instances within parsed phrases. The coding parameter for each symbolwise context is determined by cumulative count ratios. The code string length increase for a symbol y following substring s , under the symbolwise equivalent, is the log of the ratio of node counts in subtrees s and s\cdot y of the Ziv-Lempel parsing tree. To demonstrate the symbolwise equivalent of the Ziv-Lempel algorithm, we extend the work of Rissanen and Langdon to incomplete parse trees. The result requires the proper handling of the comma when one phrase is the prefix of another phrase.

A note on associative processors for data management

Associative “logic-per-track” processors for data management are examined from a technological and engineering point of view. Architectural and design decisions are discussed. Some alternatives to the design of comparators, garbage collection, and domain extraction for architectures like the Relational Associative Processor (RAP) are offered.

An algorithm for generating permutations

An argorithm is described which under repeated application generates all permutations of K elements. Only the previously generated permutation, the constant K, and a temporary index are needed. Starting with a particular ordering of K elements (ab cd), repeated application of the algorithm will generate K--1 additional permutations by K--1 successive rotations. From the initial circular ordering of K objects, another circular ordering can be obtained by rotating the K--1 lowest elements. For each new K--1 circular ordering, another K--2 can be obtained by rotating the K-2 lowest elements. By continuing in this manner, applications of the algorithm will generate all (K--l)! circular orderings, or since each circular ordering yields K permutations the algorithm generates all K! permutations.

Arithmetic codes for constrained channels

Arithmetic codes have been studied in the context of compression coding, i.e., transformations to code strings which take up less storage space or require less transmission time over a communications link. Another application of coding theory is that of noiseless channel coding, where constraints on strings in the channel symbol alphabet prevent an obvious mapping of data strings to channel strings. An interesting duality exists between compression coding and channel coding. The source alphabet and code alphabet of a compression system correspond, respectively, to the channel alphabet and data alphabet of a constrained channel system. The decodability criterion of compression codes corresponds to the representability criterion of constrained channel codes, as the generalized Kraft Inequality has a dual inequality due to the senior author.

A general fixed rate arithmetic coding method for constrained channels

This paper extends the result of earlier work on the application of arithmetic codes to the constrained channel problem. We specifically present a general length-based fixed rate implementation technique which performs the arithmetic coding recursions during each channel time unit. This technique is superior to an earlier unpublished code for general constrained channels. The approach permits the design of codes for sophisticated channel constraints.

Reading up books: A building-block approach to digital design: An overview of digital design, advocating a hierarchical approach, is followed by a review and rating of five major texts

As a student, you may be asking yourself whether you wish to enter the digital systems field, an area that is growing in importance because of the ever-decreasing cost of powerful digital building blocks. Digital designers create digital systems such as computers, industrial controllers, and electronic games. They may also do programming, but our focus in this article is on design with digital circuits.