Fuzheng Yang

Model-Based Joint Bit Allocation Between Texture Videos and Depth Maps for 3-D Video Coding

In 3-D video coding, texture videos and depth maps need to be jointly coded. The distortion of texture videos and depth maps can be propagated to the synthesized virtual views. Besides coding efficiency of texture videos and depth maps, joint bit allocation between texture videos and depth maps is also an important research issue in 3-D video coding. First, we present comprehensive analyses on the impacts of the compression distortion of texture videos and depth maps on the quality of the virtual views, and then derive a concise distortion model for the synthesized virtual views. Based on this model, the joint bit allocation problem is formulated as a constrained optimization problem, and is solved by using the Lagrangian multiplier method. Experimental results demonstrate the high accuracy of the derived distortion model. Meanwhile, the rate-distortion (R-D) performance of the proposed algorithm is close to those of search-based algorithms which can give the best R-D performance, while the complexity of the proposed algorithm is lower than that of search-based algorithms. Moreover, compared with the bit allocation method using fixed texture and depth bits ratio (5:1), a maximum 1.2 dB gain can be achieved by the proposed algorithm.

No-Reference Quality Assessment for Networked Video via Primary Analysis of Bit Stream

A no-reference (NR) quality measure for networked video is introduced using information extracted from the compressed bit stream without resorting to complete video decoding. This NR video quality assessment measure accounts for three key factors which affect the overall perceived picture quality of networked video, namely, picture distortion caused by quantization, quality degradation due to packet loss and error propagation, and temporal effects of the human visual system. First, the picture quality in the spatial domain is measured, for each frame, relative to quantization under an error-free transmission condition. Second, picture quality is evaluated with respect to packet loss and the subsequent error propagation. The video frame quality in the spatial domain is, therefore, jointly determined by coding distortion and packet loss. Third, a pooling scheme is devised as the last step of the proposed quality measure to capture the perceived quality degradation in the temporal domain. The results obtained by performance evaluations using MPEG-4 coded video streams have demonstrated the effectiveness of the proposed NR video quality metric.

A novel objective no-reference metric for digital video quality assessment

A novel objective no-reference metric is proposed for video quality assessment of digitally coded videos containing natural scenes. Taking account of the temporal dependency between adjacent images of the videos and characteristics of the human visual system, the spatial distortion of an image is predicted using the differences between the corresponding translational regions of high spatial complexity in two adjacent images, which are weighted according to temporal activities of the video. The overall video quality is measured by pooling the spatial distortions of all images in the video. Experiments using reconstructed video sequences indicate that the objective scores obtained by the proposed metric agree well with the subjective assessment scores.

Bitstream-based quality assessment for networked video: a review

Bitstream-based methods are intuitively suited for quality assessment of networked video services, and are currently under intense investigations in terms of research and standardization activities. This article examines the factors that may affect the quality of the networked video, and reviews the state-of-the-art techniques as well as standardization progress in bitstream-based video quality assessment. According to different levels of access to the bitstream, three types of models are described and compared: the parametric, packet layer, and bitstream-layer models.

Content-Adaptive Packet-Layer Model for Quality Assessment of Networked Video Services

Packet-layer models are designed to use only the information provided by packet headers for real-time and non-intrusive quality monitoring of networked video services. This paper proposes a content-adaptive packet-layer (CAPL) model for networked video quality assessment. Considering the fact that the quality degradation of a networked video significantly relies on the temporal as well as the spatial characteristics of the video content, temporal complexity is incorporated in the proposed model. Due to very limited information directly available from packet headers, a simple and adaptive method for frame type detection is adopted in the CAPL model. The temporal complexity is estimated using the ratio of the number of bits for coding P and I frames. The estimated temporal complexity and frame type are incorporated in the CAPL model together with the information about the number of bits and positions of lost packets to obtain the quality estimate for each frame, by evaluating the distortions induced by both compression and packet loss. A two-level temporal pooling is employed to obtain the video quality given the frame quality. Using content related information, the proposed model is able to adapt to different video contents. Experimental results show that the CAPL model significantly outperforms the G.1070 model and the DT model in terms of widely used performance criteria, including the Root-Mean-Squared Error (RMSE), the Pearson Correlation Coefficient (PCC), the Spearman Rank Order Correlation Coefficient (SCC), and the Outlier Ratio (OR).

Frame layer rate control for H.264/AVC with hierarchical B-frames

Hierarchical B-frames contribute to improvement of coding performance when introduced into H.264/AVC. However, the existing rate control schemes for H.264/AVC, which are mainly applied to IPPP and IBBP coding structures, cannot work efficiently for the coding structures with hierarchical B-frames. In this paper, a frame layer rate control scheme for hierarchical B-frames is proposed. Firstly, an adaptive starting quantization parameter (QP) determination method is implemented to derive the QP for the first coding frame based on the available channel bit rate and the content of the current video sequence. Then, the target bit budget for a group of pictures (GOP) is calculated based on the target bit rate and the buffer status. Afterwards, a temporal level (TL) layer rate control phase is introduced, and the GOP layer target bit budget is allocated to each TL. In the frame layer rate control phase, a method based on a rate-distortion model and the coding properties of the previous coded key frames is derived to determine the QP for the current key frame. For hierarchical B-frames, we introduce a typical weighting factor in the determination of their target bit budgets to address the features of the hierarchical coding structures. This weighting factor is calculated according to the target bit budget of the GOP layer and the knowledge obtained from the previous coded B-frames in each TL. Subsequently, the QP for coding the current B-frame is computed by a quadratic model with different model parameters for different TLs, and the computed QP is further adaptively adjusted according to the usage of the target bit budgets. After coding the current frame, an update stage, in which a threshold-based method is integrated to avoid model degradation, is invoked to update the parameters for rate control. Experimental results demonstrate that when the proposed rate control scheme is applied to the coding structure with hierarchical B-frames in H.264/AVC, the actual coding bit rates can match the target bit rates very well, and the encoding performance is also improved.

A cross-layer algorithm of packet scheduling and resource allocation for multi-user wireless video transmission

This paper investigates the problem of packet scheduling and resource allocation for multi-user wireless video transmission. The problem is first formulated as an optimization problem with the objective of minimizing the reconstructed video distortion of each user under the radio resource constraints. In order to solve the optimization problem, a crosslayer algorithm with low complexity is proposed by taking the radio resource constraints and the fairness across multi-user into account. Simulation results show that the proposed crosslayer algorithm outperforms the state-of-the-art packet scheduling and resource allocation algorithms by up to 1.2 dB in terms of the quality of the reconstructed video, and the fairness across different users is also maintained.

Gradient-threshold edge detection based on the human visual system

We present an improved method that is suitable for gradient-threshold edge detectors. The method takes into account the basic characteristics of the human visual system and masks the gradient image with the luminance and the activity of local image before edge labeling. An implementation of this method on a Canny detector is described as an example. The results show that the edge images obtained by our algorithm are more consistent with the perceptive edge images.

A novel hierarchical QAM-based unequal error protection scheme for H.264/AVC video over frequency-selective fading channels

A novel unequal error protection (UEP) scheme based on the hierarchical quadrature amplitude modulation (QAM) is proposed for H.264/AVC video transmission over frequency selective fading channels. In order to avoid the situation that data with higher priority in the H.264/AVC video streams to be mapped onto subcarriers in deep fading, an OFDM subcarrier classification strategy with two signal-to-noise ratio (SNR) thresholds is presented. Simulation results show that the proposed UEP scheme outperforms the state-of-the-art UEP method by up to 1.6 dB in terms of the quality of the reconstructed video.

QoE Evaluation of Multimedia Services Based on Audiovisual Quality and User Interest

Quality of experience (QoE) has significant influence on whether or not a user will choose a service or product in the competitive era. For multimedia services, there are various factors in a communication ecosystem working together on users, which stimulate their different senses inducing multidimensional perceptions of the services, and inevitably increase the difficulty in measurement and estimation of the users QoE. In this paper, a user-centric objective QoE evaluation model (QAVIC model for short) is proposed to estimate the users overall QoE for audiovisual services, which takes account of perceptual audiovisual quality (QAV) and user interest in audiovisual content (IC) amongst influencing factors on QoE such as technology, content, context, and user in the communication ecosystem. To predict the user interest, a number of general viewing behaviors are considered to formulate the IC evaluation model. Subjective tests have been conducted for training and validation of the QAVIC model. The experimental results show that the proposed QAVIC model can estimate the users QoE reasonably accurately using a 5-point scale absolute category rating scheme.