Yanwei Liu

Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model

Joint video/depth rate allocation is an important optimization problem in 3D video coding. To address this problem, this paper proposes a distortion model to evaluate the synthesized view without access to the captured original view. The proposed distortion model is an additive model that accounts for the video-coding-induced distortion and the depth-quantization-induced distortion, as well as the inherent geometry distortion. Depth-quantization-induced distortion not only considers the warping error distortion, which is described by a piecewise linear model with the video power spectral property, but also takes into account the warping error correlation distortion between two sources reference views. Geometry distortion is approximated from that of the adjacent view synthesis. Based on the proposed distortion model, a joint rate allocation method is proposed to seek the optimal trade-off between video bit-rate and depth bit-rate for maximizing the view synthesis quality. Experimental results show that the proposed distortion model is capable of approximately estimating the actual distortion for the synthesized view, and that the proposed rate allocation method can almost achieve the identical rate allocation performance as the full-search method at less computational cost. Moreover, the proposed rate allocation method consumes less computational cost than the hierarchical-search method at high bit-rates while providing almost the equivalent rate allocation performance.

A Novel Rate Control Technique for Multiview Video Plus Depth Based 3D Video Coding

This paper presents a novel rate control technique for multiview video plus depth (MVD) based 3D video coding. In the proposed rate control technique, an image-stitching method is first utilized to simultaneously encode video and depth, and then a joint rate control algorithm for MVD is presented. The joint rate control algorithm is performed on three levels, namely view level, video/depth level and frame level. In the view level, different proportions of rates are allocated for different types of views according to the pre-statistical rate allocation. In the video/depth level, the target rates for video and depth are discriminatorily assigned to guarantee the high quality of video for the backward-compatible display. In the frame level, the hierarchical rate allocation is used to regulate the target bits for each frame. In addition to the above mentioned rate control strategies, according to the special characteristics of multiview hypothetical reference decoder (HRD), the buffer-related rate control is also considered to prevent the decoder buffer from overflow or underflow even outputting multiple views. Experimental results show that the proposed rate control technique can accurately control the bit-rate to satisfy the requirements of 3D video systems.

Wireless Video Surveillance: A Survey

A wireless video surveillance system consists of three major components: 1) the video capture and preprocessing; 2) the video compression and transmission in wireless sensor networks; and 3) the video analysis at the receiving end. A myriad of research works have been dedicated to this field due to its increasing popularity in surveillance applications. This survey provides a comprehensive overview of existing state-of-the-art technologies developed for wireless video surveillance, based on the in-depth analysis of the requirements and challenges in current systems. Specifically, the physical network infrastructure for video transmission over wireless channel is analyzed. The representative technologies for video capture and preliminary vision tasks are summarized. For video compression and transmission over the wireless networks, the ultimate goal is to maximize the received video quality under the resource limitation. This is also the main focus of this survey. We classify different schemes into categories including unequal error protection, error resilience, scalable video coding, distributed video coding, and cross-layer control. Cross-layer control proves to be a desirable measure for system-level optimal resource allocation. At the receivers end, the received video is further processed for higher-level vision tasks, and the security and privacy issues in surveillance applications are also discussed.

RD-optimized interactive streaming of multiview video with multiple encodings

This paper presents a rate-distortion (RD) optimized interactive streaming method for multiview video pre-compressed by H.264 Joint Multiview Video Model (JMVM). In the proposed method, multiple encodings are first used to facilitate the flexible server-client interaction. Second, a RD-optimized scheduling strategy is provided to guarantee the optimal view-dependent delivery of multiview video. In the RD-optimized scheduling strategy, a distortion model is proposed to estimate the expected end-to-end distortion by accounting for both coding and packet-loss-induced distortions, as well as rendering-induced distortion. With the end-to-end distortion model, the server can select the optimal encoding combination for transmission. Experimental results demonstrate that the proposed method can achieve a significant end-to-end RD performance improvement over the selective streaming methods with simulcast coding or scalable multiview coding. In addition, it has better error-resilience performance to combat with packet-losses over the Internet protocol (IP) networks.

QoE-oriented 3D video transcoding for mobile streaming

With advance in mobile 3D display, mobile 3D video is already enabled by the wireless multimedia networking, and it will be gradually popular since it can make people enjoy the natural 3D experience anywhere and anytime. In current stage, mobile 3D video is generally delivered over the heterogeneous network combined by wired and wireless channels. How to guarantee the optimal 3D visual quality of experience (QoE) for the mobile 3D video streaming is one of the important topics concerned by the service provider. In this article, we propose a QoE-oriented transcoding approach to enhance the quality of mobile 3D video service. By learning the pre-controlled QoE patterns of 3D contents, the proposed 3D visual QoE inferring model can be utilized to regulate the transcoding configurations in real-time according to the feedbacks of network and user-end device information. In the learning stage, we propose a piecewise linear mean opinion score (MOS) interpolation method to further reduce the cumbersome manual work of preparing QoE patterns. Experimental results show that the proposed transcoding approach can provide the adapted 3D stream to the heterogeneous network, and further provide superior QoE performance to the fixed quantization parameter (QP) transcoding and mean squared error (MSE) optimized transcoding for mobile 3D video streaming.

Compression-Induced Rendering Distortion Analysis for Texture/Depth Rate Allocation in 3D Video Compression

In 3D video applications, the virtual view is generally rendered by the compressed texture and depth. The texture and depth compression with different bit-rate overheads can lead to different virtual view rendering qualities. In this paper, we analyze the compression-induced rendering distortion for the virtual view. Based on the 3D warping principle, we first address how the texture and depth compression affects the virtual view quality, and then derive an upper bound for the compression-induced rendering distortion. The derived distortion bound depends on the compression-induced depth error and texture intensity error. Simulation results demonstrate that the theoretical upper bound is an approximate indication of the rendering quality and can be used to guide sequence-level texture/depth rate allocation for 3D video compression.

Low-delay View Random Access for Multi-view Video Coding

Multi-view video coding is becoming a very active research topic, as multi-view video system provides the interactive feature which makes viewers experience the free viewpoint navigation within the range covered by the shooting cameras compared with the traditional single view video. Multi-view video coding aims at compressing the redundancy between views besides temporal correlations within each view. Rapid view random access is a basic requirement for multi-view video communication. Though inter-view prediction enhances the coding efficiency, it limits the rapid view random access capability. In this paper, we propose three approaches to provide low-delay view random access capability while keeping the high rate-distortion performance. The proposed techniques, including SP/SI frame coding, interleaved view coding and secondary representation coding, vastly reduce the decoding delays while view random access occurs and greatly improve the ability of switching between views.

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.

Low-complexity content-adaptive Lagrange multiplier decision for SSIM-based RD-optimized video coding

The SSIM-based rate distortion optimization (R-DO) has been proved to be an effective way to promote the perceptual video coding performance, and the Lagrange multiplier decision is the key to the SSIM-based RD-optimized video coding. Through extensively analyzing the characteristics of SSIM-based and SSE-based video distortions, this paper presents a low-complexity content-adaptive Lagrange multiplier decision method. The proposed method first estimates frame-level SSIM-based Lagrange multiplier by scaling the traditional SSE-based Lagrange multiplier with the ratio of SSE-based distortion to SSIM-based distortion. Via predicting the macroblocks perceptual importance in the whole frame, the macroblock-level Lagrange multiplier is further refined to promote the accuracy of the Lagrange multiplier decision. Experimental results show that the proposed method can obtain almost the same rate-SSIM performance and subjective quality as the state-of-the-art SSIM-based RD-optimized video coding methods with lower computation overheads.

Perceptual experience oriented transmission scheduling for scalable video streaming over cognitive radio networks

Cognitive radio (CR) promotes the utilization of the wireless spectrum through admitting the secondary users to access the primary channels in an opportunistic manner. Therefore, the real-time video streaming over the CR network is challenging due to the time-varying channels. In this paper, we propose an adaptive scalable video transmission scheduling method for video streaming over time-varying channels in CR network, in which the end-to-end perceptual visual experience is optimized systematically by considering various experience-influencing factors of the scalable video source, channel, and receiving buffer. Moreover, an enhanced content-based adaptive video playout scheme is developed to further optimize the end-to-end perceptual visual experience by decreasing the receiving buffer underflow probability. The efficiency and effectiveness of the proposed method has been validated by both analytical study and extensive experimental results.