Stefan J. P. Westen

Perceptual image quality based on a multiple channel HVS model

We propose a new measure of perceptual image quality based on a multiple channel human visual system (HVS) model for use in digital image compression. The model incorporates the HVS light sensitivity, spatial frequency and orientation sensitivity, and masking effects. The model is based on the concept of local band-limited contrast (LBC) in oriented spatial frequency bands. This concept leads to a simple masking function. The model has the flexibility to account for the changes in frequency sensitivity as a function of local luminance and is consistent with masking experiments using gratings and edges. Numerical scaling experiments with a test panel and a set a test images that were coded using different coding algorithms showed that the proposed measure correlates better with perceptual image quality than the conventional SNR measure.

Noise reduction for image sequences using an oriented pyramid thresholding technique

Denoising images by means of thresholding transform coefficients has become popular. The discrete wavelet transform (DWT) seems to be very useful in this context. We propose a method for achieving a spatio-temporal decomposition of image sequences by combining the spacial Simoncelli (1992) pyramid with, in a temporal direction, wavelets. The directionally sensitive filter banks of the Simoncelli pyramid lead to better separation of signal and noise compared to a separable DWT. Including motion compensated temporal information further improves this separation. The directional sensitive filters also make it possible to locally estimate the amount of noise and the orientation of image structures. This information can be used to control the thresholding operation.

Optimization of JPEG color image coding using a human visual system model

We introduce a new model that can be used in the perceptual optimization of standard color image coding algorithms (JPEG/MPEG). The human visual system model is based on a set of oriented filters and incorporates background luminance dependencies, luminance and chrominance frequency sensitivities, and luminance and chrominance masking effects. The main problem in using oriented filter-based models for the optimization of coding algorithms is the difference between the orientation of the filters in the model domain and the DCT block transform in decoding domain. We propose a general method to combine these domains by calculating a local sensitivity for each DCT (color) block. This leads to a perceptual weighting factor for each DCT coefficient in each block. We show how these weighting factors allow us to use advanced techniques for optimal bit allocation in JPEG (e.g. custom quantization matrix design and adaptive thresholding). With the model we propose it is possible to calculate a perceptually weighted mean squared error (WMSE) directly in the DCT color domain, although the model itself is based on a directional frequency band decomposition.

Perceptual optimization of image coding algorithms

We show how a model of the human visual system (HVS) can be used for encoder based perceptual optimization of JPEG. The HVS model takes into account the effects of light sensitivity, frequency sensitivity, and masking effects and is based on a hierarchy of oriented band pass filters. The model can be used to calculate a local frequency sensitivity, which in turn can be used to calculate perceptual weighting factors for each DCT coefficient in every block in the image. The proposed method is not limited to DCT based coders but is applicable to other coding schemes as well. The weighting factors can be used for adaptive quantization matrix design and for adaptive thresholding. We introduce a refinement of the adaptive thresholding technique that we call soft thresholding.

Spatiotemporal model of human vision for digital video compression

We propose a new model for the prediction of distortion visibility in digital image sequences, which is aimed at use in digital video compression algorithms. The model is an extension of our spatial vision model with a spatio-temporal contrast sensitivity function and an eye movement estimation algorithm. Due to the importance of smooth pursuit eye movements when viewing image sequences, eye movements cannot be neglected in a spatio-temporal vision model. Although eye movements can be incorporated by motion compensation of the contrast sensitivity function, the requirements for this motion compensation are different than those for motion compensated prediction in video coding. We propose an algorithm for the estimation of smooth pursuit eye movements, under the worst-case assumption that the observer is capable of tracking all objects in the image.

A quality measure for compressed image sequences based on an eye-movement compensated spatio-temporal model

We propose a vision model for predicting the perceptibility of coding errors in digital image sequences. Eye movements have a significant effect on distortion visibility, and hence they form an important part of the model. A modified motion estimator is used to predict the eye movements of a viewer when watching an image sequence. The spatio-temporal frequency sensitivity and masking effects are included in the model. The model is based on a decomposition of the signal into frequency and orientation bands to allow accurate modeling of spatial masking by a contrast transducer. We describe how the proposed quality measure can be used to compress image sequences using a constrained MPEG-2 encoder.