Zhongkang Lu

Modeling visual attention's modulatory aftereffects on visual sensitivity and quality evaluation

With the fast development of visual noise-shaping related applications (visual compression, error resilience, watermarking, encryption, and display), there is an increasingly significant demand on incorporating perceptual characteristics into these applications for improved performance. In this paper, a very important mechanism of the human brain, visual attention, is introduced for visual sensitivity and visual quality evaluation. Based upon the analysis, a new numerical measure for visual attentions modulatory aftereffects, perceptual quality significance map (PQSM), is proposed. To a certain extent, the PQSM reflects the processing ability of the human brain on local visual contents statistically. The PQSM is generated with the integration of local perceptual stimuli from color contrast, texture contrast, motion, as well as cognitive features (skin color and face in this study). Experimental results with subjective viewing demonstrate the performance improvement on two PQSM-modulated visual sensitivity models and two PQSM-based visual quality metrics.

Rate control for videophone using local perceptual cues

We present a method for extracting local visual perceptual cues and its application for rate control of videophone, in order to ensure the scarce bits to be assigned for maximum perceptual coding quality. The optimum quantization step is determined with the rate-distortion model considering the local perceptual cues in the visual signal. For extraction of the perceptual cues, luminance adaptation and texture masking are used as the stimulus-driven factors, while skin color serves as the cognition-driven factor in the current implementation. Both objective and subjective quality evaluations are given by evaluating the proposed perceptual rate control (PRC) scheme in the H.263 platform, and the evaluations show that the proposed PRC scheme achieves significant quality improvement in block-based coding for bandwidth-hungry applications.

A locally adaptive algorithm for measuring blocking artifacts in images and videos

Abstract Block transform coding is the most popular approach for image and video compression. The objective measurement of blocking artifacts plays an important role in the design, optimization, and assessment of image and video coding systems. This paper presents a new algorithm for measuring image quality of a BDCT coded images or videos. It exhibits unique and useful features: (1) it examines the blocks individually so that it can measure the severity of blocking artifacts locally; (2) it is a one-pass algorithm in the sense that the image needs to be accessed only once; (3) it takes into account the blocking artifacts for high bit rate images and the flatness for the very low bit rate images; (4) the quality measure is well defined in the range of 0–10. Experiments on various still images and videos show that the new quality measure is very efficient in terms of computational complexity and memory usage, and can produce consistent blocking artifacts measurement.

A locally-adaptive algorithm for measuring blocking artifacts in images and videos

Block transform coding is the most popular approach for image and video compression. The objective measurement of blocking artifacts plays an important role in the design, optimization, and assessment of image and video coding systems. This paper presents a new algorithm for measuring blocking artifacts in images and videos. It exhibits unique and useful features: 1) it examines the blocks individually so that it can measure the severity of blocking artifacts locally; 2) it is a one-pass algorithm in the sense that the image needs to be accessed only once; 3) it takes into account the blocking artifacts for high bit rate images and the flatness for the very low bit rate images; 4) the blocking artifacts measure is well-defined in the range of 0-10. Experiments on various still images and videos show that this blockiness measure is very efficient in terms of computational complexity and memory usage, and can produce consistent blocking artifacts measurement.

Just-noticeable-distortion profile with nonlinear additivity model for perceptual masking in color images

We propose a new spatial just noticeable distortion (JND) profile for color image processing. The JND threshold depends on various masking effects underlying existing in the human vision system (HVS). How to efficiently integrate different masking effects together is the key issue of modelling the JND profile. Based on recent vision research results, we model the masking effects in different stimulus dimensions as a nonlinear additivity model for masking (NAMM). It applies to all color components and accounts for the compound impact of luminance masking and texture masking to estimate the JND threshold in images. In our PSNR and subjective comparison to the related work, the proposed NAMM scheme provides a more accurate JND profile towards the actual JND bound in the HVS.

Measuring the negative impact of frame dropping on perceptual visual quality

Work presented in the paper includes two parts: first we measured the detectability and annoyance of frame droppings effect on perceptual visual quality evaluation under different motion and framesize conditions. Then, a new logistics function and an effective yet simple motion content representation are selected to model the relationship among motion, framerate and negative impact of frame-dropping on visual quality, in one formula. The high Pearson and Spearman correlation results between the MOS and predicted MOSp, as well as the results of other two error metrics, confirm the success of the selected logistic function and motion content representation.

Visual distortion assessment with emphasis on spatially transitional regions

It is known that the human visual system (HVS) does not pay equal attention to each error and even region in judging picture quality. In this paper, we combine a perceptual model with an integrated detection of the spatially transitional regions in visual distortion evaluation to better match the HVS perception to visual quality. For decompressed images or video, the spatially transitional regions are the regions where major perceptually disturbing artefacts caused by edge impairments (mainly due to blurring and locations where the edge information is not adequately represented) and the presence of false edges (mainly due to blockiness and the presence of strong rippling effects of ringing) usually occur. Such regions are efficiently detected based on a single two-dimensional spatial high-pass filter in our work. Good correlation between the proposed method and the human perception has been demonstrated with the full set of 50-Hz video quality expert group test data.

Blind blur assessment for vision-based applications

In this paper, a criterion for objective defocus blur measurement is theoretically derived from one image. The essential idea is to estimate the point spread function (PSF) from the line spread function (LSF), whereas the LSF is constructed from edge information. It is proven that an edge point corresponds to the local maximal gradient in a blurred image, and therefore edges can be extracted from blurred images by conventional edge detectors. To achieve high accuracy, local Radon transform is implemented and a number of LSFs are extracted from each edge. The experimental results on a variety of synthetic and real blurred images validate the proposed method. The algorithm can be implemented for image quality evaluation in vision-based applications as no reference images are needed.

Perceptual Quality Metric for H.264 Low Bit Rate Videos

This paper proposed an objective video quality metric designed for automatically assessing the perceived quality of digitally compressed multimedia videos using H.264 video compression. The rationale in proposing perceptual-based metric is because traditional measure, peak signal-to-noise ratio (PSNR), has been found to correlate poorly with subjective quality ratings, particularly at much lower bit rates. In this paper, computational models have been applied to emulate human visual perception based on a combination of local and global modulating factors. The proposed video quality metric has been tested on GIF and QCIF video sequences compressed using H.264 video compression technique at various bit rates (24-384 Kbps) and frame rates (7.5-30 Hz). Performance of the proposed metric with respect to subjective scores will be reported and a comparison with PSNR and also the video structural similarity method (being one of the best video quality metric for high bit rate videos recently reported in the literature) will also be provided in this paper

Video quality metric for low bitrate compressed videos

Traditionally, peak signal-to-noise ratio (PSNR) has been used to represent the quality of a compressed video sequence. However, PSNR has been found to correlate poorly with subjective quality ratings, particularly at much lower bit rates and frame rates. This paper proposes an objective video quality metric to automatically measure the perceived quality of a stream of video images based on a combined measure of distortion-invisibility, block-fidelity, and content richness fidelity. The proposed method has been tested on CIF and QCIF video sequences compressed at low bit rates and frame rates and it is shown to give significantly better correlations to human perception than peak signal-to-noise ratio (PSNR).