James C. Brailean

Noise reduction filters for dynamic image sequences: a review

In this paper, a thorough review is presented of noise reduction filters for digital image sequences. Detailed descriptions of several spatiotemporal and temporal noise reduction algorithms are provided. To aid in comparing between these different algorithms, we classify them based on their support (i.e., 3-D or 1-D filter) and whether or not motion compensation is employed. Several algorithms from each of the four categories are implemented and tested on real sequences degraded to various signal-to-noise ratios. These experimental results are discussed and analyzed to determine the overall advantages and disadvantages of the four general classifications, as well as, the individual filters. >

Image and video compression algorithms based on recovery techniques using mean field annealing

Image and video coding algorithms have found a number of applications ranging from video telephony on the public switched telephone networks (PSTN) to HDTV. However, as the bit rate is lowered, most of the existing techniques, as well as current standards, such as JPEG, H. 261, and MPEG-1 produce highly visible degradations in the reconstructed images primarily due to the information loss caused by the quantization process. In this paper, we propose an iterative technique to reduce the unwanted degradations, such as blocking and mosquito artifacts while keeping the necessary detail present in the original image. The proposed technique makes use of a priori information about the original image through a nonstationary Gauss-Markov model. Utilizing this model, a maximum a posteriori (MAP) estimate is obtained iteratively using mean field annealing. The fidelity to the data is preserved by projecting the image onto a constraint set defined by the quantizer at each iteration. The proposed solution represents an implementation of a paradigm we advocate, according to which the decoder is not simply undoing the operations performed by the encoder, but instead it solves an estimation problem based on the available bitstream and any prior knowledge about the source image. The performance of the proposed algorithm was tested on a JPEG, as well as on an H.261-type video codec. It is shown to be effective in removing the coding artifacts present in low bit rate compression. >

Restoration of low bit rate compressed images using mean field annealing

Most of the transform-based image compression techniques produce visible artifacts in the reconstructed image, particularly at low bit rates. In this paper, we propose an iterative technique to reduce the unwanted degradations such as blocking artifacts while keeping the necessary detail present in the original image. The proposed technique makes use of a priori information about the original image by using a nonstationary Gauss-Markov model. A MAP estimate is obtained iteratively using mean field annealing. A additional a priori information about the transform of the original image is incorporated into the estimation process by projecting the image onto a set defined by the quantizer at each iteration. The performance of the proposed algorithm was tested on JPEG compressed images. It is shown to be effective in removing the coding artifacts present in the low bit rate compressed images.<<ETX>>

Video Coding Standards: Error Resilience and Concealment

The emergence of digital video compression as a means to enable visual communications has been driven largely by the efforts of voluntary international standards organizations. The standards developed by these organizations have substantially increased the use of video in many different applications. However, transmitting standardized video bitstreams in error prone environments still presents a particular challenge for multimedia system designers. This is mainly due to the high sensitivity of these bitstreams to channel errors. This chapter examines the existing standards for video compression and communication, and details the tools for providing error resilience and concealment within the scope and syntax of these standards.

Noise robust spatial gradient estimation for use in displacement estimation

An important component of any spatial temporal gradient motion estimation algorithm is the accuracy by which spatial gradients are calculated. When an image sequence is corrupted by noise, the problem of determining these spatial gradients becomes extremely difficult. This is immediately apparent, since the magnitude response of the derivative operator is |/spl omega/|. In other words, the components of an image are amplified upon differentiation in proportion to their frequency value. Thus, high-frequency noise terms will dominate any low-frequency features in the differentiated image. If this corrupted differentiated image is then used within a spatio-temporal gradient motion estimator, the noise will erroneously influence the estimated motion vector. The problem of estimating the spatial gradient is treated as an inverse problem with noise. Formulating the problem in this manner results in a recursive gradient estimator that suppresses the effects of noise.

Recursive map displacement field estimation and its applications

We briefly describe some of our work on the use of stochastic models to describe the displacement vector field (DVF) in an image sequence. Specifically, autoregressive models are used which describe the abrupt transitions in the DVF with the use of a line process, but also result in spatio-temporally recursive structures. The use of such models in developing maximum a posteriori estimators for the DVF and the line process is subsequently described. Finally, the extension and application of the resulting estimator to the problems of object tracking, video compression and restoration of video sequences is reviewed.

Occlusion and nonstationary displacement field estimation in quantum-limited image sequences

In this paper, we develop an algorithm for obtaining the maximum a posteriori (MAP) estimate of the displacement vector field (DVF) from two consecutive image frames of an image sequence acquired under quantum-limited conditions. The estimation of the DVF has applications in temporal filtering, object tracking and frame registration in low- light level image sequences as well as low-dose clinical x-ray image sequences. The quantum-limited effect is modeled as an undesirable, Poisson-distributed, signal-dependent noise artifact. The specification of priors for the DVF allows a smoothness constraint for the vector field. In addition, discontinuities and areas corresponding to occlusions which are present in the field are taken into account through the introduction of both a line process and an occlusion process for neighboring vectors. A Bayesian formulation is used in this paper to estimate the DVF and a block component algorithm is employed in obtaining a solution. Several experiments involving a phantom sequence show the effectiveness of this estimator in obtaining the DVF under severe quantum noise conditions.

Video coding algorithm based on recovery techniques using mean field annealing

Most of the existing video coding algorithms produce highly visible artifacts in the reconstructed images as the bit-rate is lowered. These artifacts are due to the information loss caused by the quantization process. Since these algorithms treat decoding as simply the inverse process of encoding, these artifacts are inevitable. In this paper, we propose an encoder/decoder paradigm in which both the encoder and decoder solve an estimation problem based on the available bitstream and prior knowledge about the source image and video. The proposed technique makes use of a priori information about the original image through a nonstationary Gauss-Markov model. Utilizing this mode, a maximum a posteriori (MAP) estimate is obtained iteratively using mean field annealing. The fidelity to the data is preserved by projecting the image onto a constraint set defined by the quantizer at each iteration. The performance of the proposed algorithm is demonstrated on an H.261-type video codec. It is shown to be effective in improving the reconstructed image quality considerably while reducing the bit-rate.