Demin Wang

Video Quality Metric for Bit Rate Control via Joint Adjustment of Quantization and Frame Rate

The purpose of this study is to propose a quality metric of video encoded with variable frame rates and quantization parameters suitable for mobile video broadcasting applications. As a first step, experiments are conducted to assess the subjective quality of video sequences encoded with variable frame rates and quantization parameters. Resulting experimental data show that for the purpose of video rate control, optimization using the classical PSNR does not match up to that of subjective quality data. The second step bridges this gap between PSNR and subjective quality data by constructing a new quality metric that accounts for both encoding parameters (quantization and frame rate), and intrinsic video sequence characteristics (motion speed). The average correlation coefficient for five video sequences tested is as high as 0.93 with the proposed metric, in contrast with the PSNRs 0.70

Stereoscopic image generation based on depth images

A depth-image-based rendering system for generating new views is proposed. One important aspect of the proposed system is that the depth maps are pre-processed using an asymmetric filter to smoothen the sharp changes in depth at object boundaries. In addition to ameliorating the effects of blocky artifacts and other distortions contained in the depth maps, the smoothing reduces or completely removes disocclusion areas where potential artifacts can arise from image warping which is needed to generate images from new viewpoints. The asymmetric nature of the filter reduces the amount of geometric distortion that might be perceived otherwise. We present some results to show that the proposed system provides an improvement in image quality of stereoscopic virtual views while maintaining reasonably good depth quality.

Hybrid de-interlacing algorithm based on motion vector reliability

This paper presents a hybrid de-interlacing algorithm that converts video from interlaced format to progressive format. This hybrid algorithm effectively combines two existing de-interlacing techniques: one that offers high vertical resolution and the other that is robust to erroneous motion vectors. The combination is based on a new measurement of motion vector reliability, so that the hybrid algorithm is dominated by the high-resolution technique if motion vectors are reliable; otherwise, it approaches the robust technique. Motion vector reliability is measured using the a posteriori probability of motion vectors. Experimental results show that this is an effective measure of motion vector reliability. The hybrid algorithm offers high spatial resolution without artifacts caused by erroneous motion vectors, and outperforms four of the best existing techniques in terms of peak signal-to-noise ratio.

Toward optimal rate control: a study of the impact of spatial resolution, frame rate, and quantization on subjective video quality and bit rate

Multi-dimensional rate control schemes, which jointly adjust two or three coding parameters, have been recently proposed to achieve a target bit rate while maximizing some objective measures of video quality. The objective measures used in these schemes are the peak signal-to-noise ratio (PSNR) or the sum of absolute errors (SAE) of the decoded video. These objective measures of quality may differ substantially from subjective quality, especially when changes of spatial resolution and frame rate are involved. The proposed schemes are, therefore, not optimal in terms of human visual perception. We have investigated the impact on subjective video quality of the three coding parameters: spatial resolution, frame rate, and quantization parameter (QP). To this end, we have conducted two experiments using the H.263+ codec and five video sequences. In Experiment 1, we evaluated the impact of jointly adjusting QP and frame rate on subjective quality and bit rate. In Experiment 2, we evaluated the impact of jointly adjusting QP and spatial resolution. From these experiments, we suggest several general rules and guidelines that can be useful in the design of an optimal multi-dimensional rate control scheme. The experiments also show that PSNR and SAE do not adequately reflect perceived video quality when changes in spatial resolution and frame rate are involved, and are therefore not adequate for assessing quality in a multi-dimensional rate control scheme. This paper describes the method and results of the investigation.

Curved wavelet transform for image coding

The conventional two-dimensional wavelet transform used in existing image coders is usually performed through one-dimensional (1-D) filtering in the vertical and horizontal directions, which cannot efficiently represent edges and lines in images. The curved wavelet transform presented in this paper is carried out by applying 1-D filters along curves, rather than being restricted to vertical and horizontal straight lines. The curves are determined based on image content and are usually parallel to edges and lines in the image to be coded. The pixels along these curves can be well represented by a small number of wavelet coefficients. The curved wavelet transform is used to construct a new image coder. The code-stream syntax of the new coder is the same as that of JPEG2000, except that a new marker segment is added to the tile headers. Results of image coding and subjective quality assessment show that the new image coder performs better than, or as well as, JPEG2000. It is particularly efficient for images that contain sharp edges and can provide a PSNR gain of up to 1.67 dB for natural images compared with JPEG2000.

Quality metric for video sequences with temporal scalability

For the purpose of temporally scalable video coding and multi-dimensional bit rate control, we conducted experiments of subjective quality assessment for video sequences encoding with variable frame rates and quantization parameters. The experimental results show that the correlation between the PSNR and the subjective quality becomes very low when the two coding parameters vary simultaneously. Then, we propose a new quality metric that takes into account the quantization errors, frame rates, and motion speeds. This metric has a higher correlation with the subjective quality. The average correlation coefficient for five video sequences is as high as 0.93, whereas that of the PSNR is only 0.70.

Improvement of JPEG2000 using curved wavelet transform

The wavelet transform in JPEG2000 is performed using one-dimensional (1D) filtering in the vertical and horizontal directions. This conventional wavelet transform is not effective to represent edges and lines in images. In this paper we present a curved wavelet transform that improves the performance of JPEG200. The curved wavelet transform is performed using 1D filtering along curves that are usually parallel to edges and lines in images. The pixels along these curves can be well represented by a small number of wavelet coefficients. A simple algorithm is proposed in this paper to determine the curves according to image content. Experimental results show that the curved wavelet transform can significantly improve the compression efficiency of JPEG2000, especially for images that contain sharp edges and lines. The coding gain can be up to 1.6 dB in the terms of PSNR.

Reliability measurement of disparity estimates for intermediate view reconstruction

Proposes an algorithm for improving the image quality of disparity-based intermediate view reconstruction by introducing a reliability measurement for disparity estimates. The reliability of disparity estimates is measured with a criterion based on a-posteriori probability of disparity estimates, in which the displaced image intensity difference and the variation of disparity estimates are taken into account. This reliability measurement is then integrated into an intermediate view reconstruction approach, which guides the recovery of disparity values in occluded areas and the reconstruction of the intermediate views. Experimental results with natural stereoscopic sequences show that the disparity reliability measurement is quite effective. The proposed algorithm improves the image quality of reconstructed intermediate views.

Curved wavelet transform and overlapped extension for image coding

The conventional 2D wavelet transform for image coding is performed using a symmetric extension and 1D filtering in the vertical and horizontal directions. In this paper, we present a curved wavelet transform and a method called overlapped extension. The curved wavelet transform is performed using 1D filtering along curves that are usually parallel to edges and lines in images. The pixels along these curves can be well represented using a small number of wavelet coefficients. The overlapped extension is proposed to prevent coding artifacts around the ends of the curves. Experimental results show that, compared with the conventional wavelet transform, the curved wavelet transform with overlapped extension significantly improves the subjective quality of decoded images, and the coding gain in PSNR can be up to 1.5 dB.

An adaptive object-based reconstruction of intermediate views from stereoscopic images

An intermediate view reconstruction is required in many applications. In this contribution, an adaptive object-based algorithm for intermediate view reconstruction of stereoscopic images is proposed. The disparity fields between the left-eye and the right-eye images are first estimated and then post-processed for recovering the disparity values in occluded areas. After that, the intermediate view is adaptively interpolated from the stereoscopic images using object-based disparity compensation. Tests with natural stereoscopic sequences show that the reconstructed views have high image quality and create very clean and stable depth when viewed stereoscopically.