Richard L. Baker

Motion compensation for video compression using control grid interpolation

A new class of motion compensation methods that are based on control grid interpolation (CGI) for use in video compression is described. The predominant motion compensation method, block matching, is shown to be a special case of CGI. A new CGI method is presented that produces a smooth motion vector field, preserving continuity and connectivity in the prediction image. When contrasted with block matching, the new algorithm eliminates blocking artifacts while using the same or less side information for approximately the same mean square error. Search algorithms and coding methods developed previously for block matching can be used with this new technique.<<ETX>>

Global zoom/pan estimation and compensation for video compression

An algorithm is presented for estimating and compensating camera zooms and pans. It models the global motion in each frame with just two parameters: a zoom factor and a two-dimensional pan vector both based on local displacement vectors found by conventional means (such as block matching). Since motion by objects in the scene obscures global motion, the algorithm can iterate to refine its estimate. Simulations suggest the algorithm is robust and accurate, and can significantly reduce both the energy of the motion compensated residual image as well as the zeroth-order entropy of the local displacement vector field.<<ETX>>

Efficient quadtree coding of images and video

The authors describe the theory needed to construct quadtree data structures which optimally allocate rate, given a set of quantizers. A Lagrange multiplier method finds these optimal rate allocations with no monotonicity restrictions. The theory is used to derive a new quadtree construction method which uses a stepwise search to find the overall optimal quadtree structure. The search can be driven with either actual measured quantizer performance or ensemble average predicted performance. This theory is then applied to the design of an interframe hybrid video coding system using a quadtree with vector quantization.<<ETX>>

Fixed distortion subband coding of images for packet-switched networks

A subband image codec is presented that approximately attains a user-prescribed fidelity by allowing the encoders compression rate to vary. The fixed distortion subband coding (FDSBC) system is suitable for use with future of packet-switched networks. The codecs design is based on an algorithm that allocates distortion among the subbands to minimize channel entropy. By coupling this allocation procedure with judiciously selected subband quantizers, an elementary four-band codec was obtained. Additional four-band structures may be nested in a hierarchical configuration for improved performance. Each of the configurations tested attains mean square distortions within 2.0 dB of the user-specific value over a wide range of distortion for several standard test images. Rate versus mean-square distortion performance rivals that of fixed-rate systems having similar complexity. The encoders output is formatted to take advantage of prioritized packet networks. Simulations show that FDSBC is robust with respect to packet loss and may be used effectively for progressive transmission applications. >

Compression of color digital images using vector quantization in product codes

There is a growing interest in the use of vector quantization for coding digital images. A key issue to be resolved is how to achieve perceptually pleasing results while limiting encoding complexity to tolerable levels. In this paper, product codes are described which improve the quality of the encoded edges and textures for a given level of complexity. These product codes separate the mean and orientation information from each source vector and encode this information independently to allow the residual to be vector quantized more accurately. The color image coder also reduces the required bit rate by taking advantage of spectral redundancy. Experimental results indicate that an improvement of almost 1.4 dB in SNR can be achieved over a Discrete Cosine Transform block coder of comparable complexity, with negligible computational complexity added by the product structure.

An integrated circuit design for pruned tree-search vector quantization encoding with an off-chip controller

The design of an encoder for pruned tree-search vector quantization (VQ) is discussed. This allows near-optimal performance in a mean square error sense while keeping the hardware complexity low. The encoder is partitioned into a slave processor chip that computes the distance and performs minimizations and an off-chip controller that directs the search. Pointer addressing is exploited in the codebook memory to keep the controller hardware simple. Inputs to the slave processor include the source vectors, the code vectors; and external control signals. The slave processor outputs the index of the code vector that best approximates the input in a mean square error sense. The layout for the slave processor has been generated using a 1.2- mu m CMOS library and measures 5.76*6.6 mm/sup 2/. Critical path simulation with SPICE indicates a throughput of 89 million multiply-accumulates per second. This implies that real-time processing at MPEG rates can be achieved if the number of levels (N7) and the number of children at any node (M) obey the constraint M*N >

A finite-state vector quantizer for low-rate image sequence coding

Vector quantization (VQ) is an effective spatial domain image coding technique at under 1.0 bits per pixel (bpp). Its performance can be improved by incorporating block-to-block memory. Finite State VQs (FSVQ) do this by using a state variable to express characteristics of neighboring blocks and identify which subset of a supercodebook the FSVQ encoder and decoder are to use for the current vector. This paper describes a new method for image sequence coding that uses an interframe FSVQ In the spatial domain. Block-to-block and frame-to-frame memory is expressed through a state which is a function of block motion and the intensity gradients of a blocks neighbors. A scalar predictor adds additional interframe memory and reduces the energy of residuals encoded by the VQ, improving edge rendition and limiting blocking artifacts. The codec at 0.6 bpp outperforms a similar, non-finite state intraframe VQ at 1.0 bpp. No motion compensation or conditional replenishment is used.

Laplacian pyramid encoding: optimum rate and distortion allocations

The authors develop the optimum bit allocations for fixed rate (minimum distortion) and fixed distortion (minimum rate) hierarchical Laplacian pyramid image coding structures, using scalar quantizers that have negative exponential quantizer functions. The optimum MSE (mean square error) bit allocation is shown to agree closely with P.J. Burt and E.H. Adelsons (IEEE Trans. Acoust., Speech and Signal Proc., vol. ASSP-34, p.1278-88, Oct. 1987) empirically derived and perceptually based allocation. The slope of the distortion-rate function is found to vary with the number of levels, N, in the pyramid, becoming worse as N increases. It is also shown that since the amount of compression is a function of the distribution of energies among the Laplacian images, performance is strongly affected by the quality of the filters. The analysis permits comparison with other important schemes, such as subband coding, from a distortion-rate perspective.<<ETX>>

Bit-Serial Architecture For Real Time Motion Compensation

We describe a bit-serial VLSI architecture for a real time motion estimation chip. The chip can search windows of arbitrary size with integer displacement resolution. Using 3 micron CMOS, it is projected to perform up to 6 million matches per second. This would permit real time exhaustive motion estimation of 8 x 8 blocks on 16 x 16 windows at NTSC resolution and 20 frames/sec. A short design time, without silicon assemblers or compilers, for the high speed chip is made possible by its bit-serial architecture.

A finite state/frame difference interpolative vector quantizer for low rate image sequence coding

A finite-state vector quantizer (VQ) suitable for both intraframe and interframe coding is presented. Interblock and interframe memory is expressed through a state variable driven by an improved classifier which keys on the intensity gradients of a blocks neighbours and its own interframe motion. A bilinear interpolator reduces the energy of foreground residuals encoded by the quantization, improving edge rendition and limiting blocking artifacts. Background blocks are vector predictive coded using simple frame differencing and codebooks built with a gradient design algorithm. The codec at 0.4 bit/pixel outperforms a similar but non-finite-state intraframe VQ at 1.0 bit/pixel by 4 dB.<<ETX>>