James W. Modestino

Optimum quantizer performance for a class of non-Gaussian memoryless sources

The performance of optimum quantizers subject to an entropy constraint is studied for a wide class of memoryless sources. For a general distortion criterion, necessary conditions are developed for optimality and a recursive algorithm is described for obtaining the optimum quantizer. Under a mean-square error criterion, the performance of entropy encoded uniform quantization of memoryless Gaussian sources is well-known to be within 0.255 bits/sample of the rate-distortion bound at relatively high rates. Despite claims to the contrary, it is demonstrated that similar performance can be expected for a wide range of memoryless sources. Indeed, for the cases considered, the worst case performance is observed to be less than 0.3 bits/sample from the rate-distortion bound, and in most cases this disparity is less at Iow rates.

Maximum-likelihood parameter estimation for unsupervised stochastic model-based image segmentation

An unsupervised stochastic model-based approach to image segmentation is described, and some of its properties investigated. In this approach, the problem of model parameter estimation is formulated as a problem of parameter estimation from incomplete data, and the expectation-maximization (EM) algorithm is used to determine a maximum-likelihood (ML) estimate. Previously, the use of the EM algorithm in this application has encountered difficulties since an analytical expression for the conditional expectations required in the EM procedure is generally unavailable, except for the simplest models. In this paper, two solutions are proposed to solve this problem: a Monte Carlo scheme and a scheme related to Besags (1986) iterated conditional mode (ICM) method. Both schemes make use of Markov random-field modeling assumptions. Examples are provided to illustrate the implementation of the EM algorithm for several general classes of image models. Experimental results on both synthetic and real images are provided.

Combined Source-Channel Coding of Images

A combined source-channel coding approach is described for the encoding, transmission and remote reconstruction of image data. The source encoder employs two-dimensional (2-D) differential pulse code modulation (DPCM). This is a relatively efficient encoding scheme in the absence of channel errors. In the presence of channel errors, however, the performance degrades rapidly. By providing error control protection to those encoded bits which contribute most significantly to image reconstruction, it is possible to minimize this degradation without sacrificing transmission bandwidth. The result is a relatively robust design which is reasonably insensitive to channel errors and yet provides performance approaching the rate-distortion bound. Analytical results are provided for assumed 2-D autoregressive image models while simulation results are described for real-world images.

Combined Source-Channel Coding of Images Using the Block Cosine Transform

An approach is described for exploiting the tradeoffs between source and channel coding in the context of image transmission. The source encoder employs two-dimensional (2-D) block transform coding using the discrete cosine transform (DCT). This technique has proven to be an efficient and readily implementable source coding technique in the absence of channel errors. In the presence of channel errors, however, the performance degrades rapidly, requiring some form of error-control protection if high quality image reconstruction is to be achieved. This channel coding can be extremely wasteful of channel bandwidth if not applied judiciously. The approach described here provides a rationale for combined source-channel coding which provides improved quality image reconstruction without sacrificing transmission bandwidth. This approach is shown to result in a relatively robust design which is reasonably insensitive to channel errors and yet provides performance approaching theoretical performance limits. Analytical results are provided for assumed 2-D autoregressive image models, while simulation results are provided for real-world images.

New short constraint length convolutional code constructions for selected rational rates (Corresp.)

New short constraint length convolutional code constructions are tabulated for rates R=(n-k)/n, k=1,2, \cdots ,n-1 with n=2, 3,\cdots ,8 , and for constraint lengths K=3,4, \cdots,8 . These codes have been determined by iterative search based upon a criterion of optimizing the free distance profile. Specifically, these codes maximize the free distance d_{f} while minimizing the number of adversaries in the distance, or weight, spectrum. In several instances we demonstrate the superiority of these codes over previously published code constructions at the same rate and constraint length. These codes are expected to have a number of applications, including combined source-channel coding schemes as well as coding for burst or impulsive noise channels.

Combined source-channel coding schemes for video transmission over an additive white Gaussian noise channel

There has been an increased interest in the transmission of digital video over real-world transmission media, such as the direct broadcast satellite (DBS) channel. Video transmitted over such a channel is subject to degradation due, in part, to additive white Gaussian noise (AWGN). Some form of forward error-control (FEC) coding may be applied in order to reduce the effect of the noise on the transmitted bitstream; however, determination of the appropriate level of FEC coding is generally an unwieldy and computationally intensive problem, as it may depend upon a variety of parameters such as the type of video, the available bandwidth, and the channel SNR. More specifically, a combined source-channel coding approach is necessary in optimally allocating rate between source and channel coding subject to a fixed constraint on overall transmission bandwidth. In this paper we develop a method of optimal bit allocation under the assumption that the distortion is additive and independent on a frame-by-frame basis. A set of universal operational distortion-rate characteristics is developed which balances the tradeoff between source coding accuracy and channel error protection for a fixed overall transmission rate and provides the basis for the optimal bit allocation approach. The results for specific source and channel coding schemes show marked improvement over suboptimum choices of channel error protection. In addition, we show that our results approach information-theoretic performance bounds which are developed in this work.

Convolutional Code Performance in the Rician Fading Channel

The performance of short constraint length convolutional codes in conjunction with binary phase-shift keyed (BPSK) modulation and Viterbi maximum likelihood decoding on the classical Rician fading channel is examined in detail. Primary interest is in the bit error probability performance as a function of E_{b}/N_{0} parameterized by the fading channel parameters. Fairly general upper bounds on bit error probability performance in the presence of fading are obtained and compared with simulation results in the two extremes of zero channel memory and infinite channel memory. The efficacy of simple block interleaving in combating the memory of the channel is thoroughly explored. Results include the effects of fading on tracking loop performance and the subsequent impact on overall coded system performance. The approach is analytical where possible; otherwise resort is made to digital computer simulation.

Rate-distortion performance of DPCM schemes for autoregressive sources

An analysis of the rate-distortion performance of differential pulse code modulation (DPCM) schemes operating on discrete-time auto-regressive processes is presented. The approach uses an iterative algorithm for the design of the predictive quantizer subject to an entropy constraint on the output sequence. At each stage the iterative algorithm optimizes the quantizer structure, given the probability distribution of the prediction error, while simultaneously updating the distribution of the resulting prediction error. Different orthogonal expansions specifically matched to the source are used to express the prediction error density. A complete description of the algorithm, including convergence and uniqueness properties, is given. Results are presented for rate-distortion performance of the optimum DPCM scheme for first-order Gauss-Markov and Laplace-Markov sources, including comparisons with the corresponding rate-distortion bounds. Furthermore, asymptotic formulas indicating the high-rate performance of these schemes are developed for both first-order Gaussian and Laplacian autoregressive sources.

Adaptive entropy coded subband coding of images

The authors describe a design approach, called 2-D entropy-constrained subband coding (ECSBC), based upon recently developed 2-D entropy-constrained vector quantization (ECVQ) schemes. The output indexes of the embedded quantizers are further compressed by use of noiseless entropy coding schemes, such as Huffman or arithmetic codes, resulting in variable-rate outputs. Depending upon the specific configurations of the ECVQ and the ECPVQ over the subbands, many different types of SBC schemes can be derived within the generic 2-D ECSBC framework. Among these, the authors concentrate on three representative types of 2-D ECSBC schemes and provide relative performance evaluations. They also describe an adaptive buffer instrumented version of 2-D ECSBC, called 2-D ECSBC/AEC, for use with fixed-rate channels which completely eliminates buffer overflow/underflow problems. This adaptive scheme achieves performance quite close to the corresponding ideal 2-D ECSBC system.

Operational rate-distortion performance for joint source and channel coding of images

This paper describes a methodology for evaluating the operational rate-distortion behavior of combined source and channel coding schemes with particular application to images. In particular, we demonstrate use of the operational rate-distortion function to obtain the optimum tradeoff between source coding accuracy and channel error protection under the constraint of a fixed transmission bandwidth for the investigated transmission schemes. Furthermore, we develop information-theoretic bounds on performance for specific source and channel coding systems and demonstrate that our combined source-channel coding methodology applied to different schemes results in operational rate-distortion performance which closely approach these theoretical limits. We concentrate specifically on a wavelet-based subband source coding scheme and the use of binary rate-compatible punctured convolutional (RCPC) codes for transmission over the additive white Gaussian noise (AWGN) channel. Explicit results for real-world images demonstrate the efficacy of this approach.