Ruixiao Yao

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.

Low-complexity content-adaptive Lagrange multiplier decision for SSIM-based RD-optimized video coding

The SSIM-based rate distortion optimization (R-DO) has been proved to be an effective way to promote the perceptual video coding performance, and the Lagrange multiplier decision is the key to the SSIM-based RD-optimized video coding. Through extensively analyzing the characteristics of SSIM-based and SSE-based video distortions, this paper presents a low-complexity content-adaptive Lagrange multiplier decision method. The proposed method first estimates frame-level SSIM-based Lagrange multiplier by scaling the traditional SSE-based Lagrange multiplier with the ratio of SSE-based distortion to SSIM-based distortion. Via predicting the macroblocks perceptual importance in the whole frame, the macroblock-level Lagrange multiplier is further refined to promote the accuracy of the Lagrange multiplier decision. Experimental results show that the proposed method can obtain almost the same rate-SSIM performance and subjective quality as the state-of-the-art SSIM-based RD-optimized video coding methods with lower computation overheads.

Perceptual experience oriented transmission scheduling for scalable video streaming over cognitive radio networks

Cognitive radio (CR) promotes the utilization of the wireless spectrum through admitting the secondary users to access the primary channels in an opportunistic manner. Therefore, the real-time video streaming over the CR network is challenging due to the time-varying channels. In this paper, we propose an adaptive scalable video transmission scheduling method for video streaming over time-varying channels in CR network, in which the end-to-end perceptual visual experience is optimized systematically by considering various experience-influencing factors of the scalable video source, channel, and receiving buffer. Moreover, an enhanced content-based adaptive video playout scheme is developed to further optimize the end-to-end perceptual visual experience by decreasing the receiving buffer underflow probability. The efficiency and effectiveness of the proposed method has been validated by both analytical study and extensive experimental results.

Utility-Based H.264/SVC Video Streaming Over Multi-Channel Cognitive Radio Networks

In a cognitive radio (CR) network, a secondary user (SU) with multiple interfaces is capable of accessing multiple CR channels in an opportunistic fashion. Therefore, the available channel resources may change dramatically, and the reliabilities of the multiple accessed CR channels are also time-varying. Video streaming in such a multi-channel CR network faces great challenges in guaranteeing the quality of the received video. To deal with these challenges, we adopt H.264/SVC encoded video as the source and firstly optimize the video streaming from the perspective of exploiting more channel resources for the SU by developing a flexible sensing-transmission scheme for opportunistic spectrum access (OSA). In this scheme, the primary user activity, channel sensing result, and channel sensing accuracy are all considered in reducing the unnecessary channel sensings and correspondingly extending the transmission duration for the SU. Based on the flexible sensing-transmission scheme, we then propose a utility-based H.264/SVC video transmission scheme to further improve the expected video quality at the receiver. Specifically , the network abstraction layer units (NALUs) in the SVC video are assigned utilities which accurately reflect their contributions to the video quality, and the total effective utility of expected received video is maximized through perfectly dispatching the NALUs over the multiple CR channels. Both analytical studies and experimental results validate the effectiveness and efficiency of the proposed method.

Perceptual quality driven cross-layer optimization for wireless video streaming

The traditional cross-layer model for video streaming can achieve significant improvement to the end-to-end video quality. However, the video quality measurement in terms of sum of squared error (SSE) in the model does not always correlate well with the perception of the human visual system. In this paper, we propose a perceptual quality driven cross-layer optimization scheme based on the structural similarity (SSIM) index for wireless video streaming. The H.264/AVC encoding parameters on the application layer and the modulation and channel coding modes on the physical layer are mainly considered to optimize the end-to-end perceptual video quality. In order to decrease the computation complexity, a low complexity optimization algorithm based on the rate-quantization (R-Q) model is proposed to reduce the range of the candidate optimization parameters. Experimental results show that the proposed SSIM-based cross-layer optimization scheme achieves better perceptual quality than the SSE-based cross-layer optimization scheme, and the proposed low complexity optimization algorithm can achieve about 62% computation decrease compared to the conventional exhaustive searching algorithm with little loss in perceptual quality.

SSIM-based cross-layer optimized video streaming over LTE downlink

Many research efforts have been done to guarantee the quality of service for video streaming over LTE. Among them, cross-layer optimized video delivery is one of the most commonly adopted approaches. However, most existing schemes of cross-layer optimized video delivery adopt PSNR as the optimization target, but do not well consider human vision characteristics. In this paper, we adopt a new metric - Structural Similarity (SSIM) in video quality evaluation and develop a SSIM-based cross-layer optimization scheme with the suppression of propagated error for video delivery over LTE downlink. The modulation and coding scheme at the physical layer of LTE downlink is selected by jointly considering the characteristics of video packets and time-varying channel states. Correspondingly, the quantization parameter of video codec at the application layer is adjusted to make the bit rate of the video stream adapt to the varying throughput of the physical link. Moreover, the error-resilient rate-distortion optimization is adopted in the proposed cross-layer optimization to suppress the effect of error propagation on the video quality degradation. Experimental results show that the proposed cross-layer optimization scheme can maintain more structural information than the conventional schemes in the received video, which correspondingly improves the video quality perceived by the end user.

Perceptual rate-distortion optimization for H.264/AVC video coding from both signal and vision perspectives

Recent researches have shown that the Structural SIMilariy (SSIM)-based Rate-Distortion Optimization (RDO) can obtain more structural information than the traditional SSE-based RDO for video coding. Correspondingly, the perceptual video quality is improved at certain degree for the video stream encoded by the SSIM-based RDO. However, for the MB with significant luminance change (due to the flashlight or sunshine change, etc.) but little structure change compared to the co-located MB in the referred frame, the SSIM-based RDO may select improper encoding mode (SKIP) which leads to uncomfortable visual experience. In this paper, involving the surrounding pixels of the current coding MB into the measurement of spacial texture quality degradation, the RDO which jointly considers the SSIM-based distortion and SSE-based distortion is proposed to improve H.264/AVC perceptual coding performance. The Lagrange multiplier in the proposed RDO is firstly derived at the frame level. Then, to make the Lagrange multiplier more adaptive to the specific video content, the Lagrange multiplier for each individual MB is refined based on the visual information theory. Experimental results show that the proposed perceptual RDO can select the optimal encoding mode which preserves as much structural information as possible with as little SSE-based distortion as possible, and that the perceptual quality of the encoded video is improved.

3D visual experience oriented cross-layer optimized scalable texture plus depth based 3D video streaming over wireless networks

Abstract 3D video streaming over the mobile Internet generally incurs the inferior 3D visual experience due to the time-varying characteristics of wireless channel. The conventional video streaming optimization methods generally neglect the harmony among different networking protocol layers. This paper proposes a cross-layer optimized texture plus depth based scalable 3D video streaming method to improve the expected 3D visual experience of the user by systematically considering the application layer texture-video/depth/FEC bit-rate allocation, MAC layer multi-channel allocation, and physical layer modulation and channel coding scheme (MCS) selection. In the cross-layer optimization, a networking-related 3D visual experience model which fuses the overlapped retinal view visual quality and depth sensation with mimicking human vision system is established to predict the 3D visual experience under the specific parameter configurations of different protocol layers. The efficiency and effectiveness of the proposed cross-layer optimized 3D video streaming method has been validated by subjective and objective experimental results.

Hierarchical-matching based scalable video streaming over multi-channel cognitive radio networks

With its ability to promote the wireless spectrum utilization, cognitive radio (CR) is a promising technology for various broadband wireless video applications. A secondary user (SU) with multiple wireless interfaces is capable to sense and access multiple CR channels, each of which can only be accessed in an opportunistic fashion. Therefore, the amount of available channel resources may change dramatically, and the reliabilities of the multiple accessed CR channels are time-varying and different from each other. To improve the quality of the received video in such a multi-channel CR network, we first develop a flexible sensing-transmission structure for dynamic spectrum access according to primary user activity and channel sensing accuracy. Based on the proposed sensing-transmission structure, we then propose a hierarchical-matching scheme to adapt the scalable video stream to the multiple time-varying and reliability-different CR channels, where we consider the priorities and validity of the network abstraction layer units (NALUs) in the transmission scheduling. Under this scheme, the more important NALUs at the global Group Of Pictures (GOP) scope will be dispatched over the more reliable channels. Both analytical studies and experimental results show the effectiveness and efficiency of the proposed method.

Transmit power aware cross-layer optimization for LTE uplink video streaming

The rapid developments of advanced wireless communication technologies and mobile devices are boosting the uplink multimedia applications. In this paper, a transmit power aware cross-layer optimization scheme is proposed to achieve a good trade-off between the transmit power and the perceived video quality for Long Term Evolution uplink video streaming. Specifically, the video coding quantization parameter and encoding mode at the application layer, and the uplink transmit power as well as modulation and coding scheme at the physical layer are jointly adjusted in the cross-layer optimization. To further improve the perceptual video experience for the end user with limited transmission resources, unequal quality control is performed by enhancing the video quality of region of interest. Additionally, the structural similarity is adopted as the video quality measurement metric to make the optimized video properly preserve the structural information during the cross-layer optimization process. Experimental results show that significant performance improvements in terms of the transmit power reduction and the perceptual video quality are achieved for the proposed transmit power aware cross-layer optimization scheme.