Haohong Wang

Cross-layer optimization for video summary transmission over wireless networks

Video summarization has gained increased popularity in the emerging multimedia communication applications, however, very limited work has been conducted to address the transmission problem of video summary frames. In this paper, we propose a cross-layer optimization framework for delivering video summaries over wireless networks. Within a rate-distortion theoretical framework, the source coding, allowable retransmission, and adaptive modulation and coding have been jointly optimized, which reflects the joint selection of parameters at physical, data link and application layers. The goal is to achieve the best video quality and content coverage of the received summary frames and to meet the delay constraint. The problem is solved using Lagirangian relaxation and dynamic programming. Experimental results indicate the effectiveness and efficiency of the proposed optimization framework, especially when the delay budget imposed by the upper layer applications is small, where more than 10% distortion gain can be achieved.

4G Wireless Video Communications

A comprehensive presentation of the video communication techniques and systems, this book examines 4G wireless systems which are set to revolutionise ubiquitous multimedia communication.4G Wireless Video Communications covers the fundamental theory and looks at systems descriptions with a focus on digital video. It addresses the key topics associated with multimedia communication on 4G networks, including advanced video coding standards, error resilience and error concealment techniques, as well as advanced content-analysis and adaptation techniques for video communications, cross-layer design and optimization frameworks and methods. It also provides a high-level overview of the digital video compression standard MPEG-4 AVC/H.264 that is expected to play a key role in 4G networks. Material is presented logically allowing readers to turn directly to specific points of interest. The first half of the book covers fundamental theory and systems, while the second half moves onto advanced techniques and applications. This book is a timely reflection of the latest advances in video communications for 4G wireless systems. One of the first books to study the latest video communications developments for emerging 4G wireless systems Considers challenges and techniques in video delivery over 4G wireless systems Examines system architecture, key techniques and related standards of advanced wireless multimedia applications Written from both the perspective of industry and academia

Video Surveillance Over Wireless Sensor and Actuator Networks Using Active Cameras

Although there has been much work focused on the camera control issue on keeping tracking a target of interest, few has been done on jointly considering the video coding, video transmission, and camera control for effective and efficient video surveillance over wireless sensor and actuator networks (WSAN). In this work, we propose a framework for real-time video surveillance with pan-tilt cameras where the video coding and transmission as well as the automated camera control are jointly optimized by taking into account the surveillance video quality requirement and the resource constraint of WSANs. The main contributions of this work are: i) an automated camera control method is developed for moving target tracking based on the received surveillance video clip in consideration of the impact of video transmission delay on camera control decision making; ii) a content-aware video coding and transmission scheme is investigated to save network node resource and maximize the received video quality under the delay constraint of moving target monitoring. Both theoretical and experimental results demonstrate the superior performance of the proposed optimization framework over existing systems.

Application-Centric Routing for Video Streaming Over MultiHop Wireless Networks

Most existing works on routing for video transmission over multihop wireless networks only focus on how to satisfy the network-oriented quality-of-service (QoS), such as through put, delay, and packet loss rate rather than application-oriented QoS such as the user-perceived video quality. Although there are some research efforts which use application-centric video quality as the routing metric, they either calculate the video quality based on some predefined rate-distortion function or model without considering the impact of video coding and decoding (including error concealment) on routing, or use exhaustive search or heuristic methods to find the optimal path, leading to high computational complexity and/or suboptimal solutions. In this paper, we propose an application-centric routing framework for real-time video transmission in multihop wireless networks, where expected video distortion is adopted as the routing metric. The major contributions of this paper are: 1) the development of an efficient routing algorithm with the routing metric expressed in terms of the expected video distortion and being calculated on-the-fly, and 2) the development of a quality-driven cross-layer optimization framework to enhance the flexibility and robustness of routing by the joint optimization of routing path selection and video coding, thereby maximizing the user-perceived video quality under a given video playback delay constraint. Both theoretical and experimental results demonstrate that the proposed quality-driven application-centric routing approach can achieve a superior performance over existing network-centric routing approaches.

Power-Aware Mobile Multimedia: a Survey (Invited Paper)

With the fast-paced development of computing technologies, mobile devices have almost sufficient computation and communication capabilities to support mobile multimedia applications such as multimedia streaming, VoIP, and mobile TV. However, most existing mobile devices are powered by battery with limited energy resource. To support multimedia on battery-powered mobile devices, how to efficiently utilize the limited power source and other limited network resources has become one of the major challenges in mobile multimedia system design. It has been shown that achieving a satisfactory user experience needs a systematic consideration of both video source adaptation and network transmission adaptation, indicating that the core of mobile multimedia system design is how to achieve a good balance of the power consumption between computation usage and communication usage. In other words, it depends on how to jointly select video source parameters and channel parameters based on the video content characteristics, available network resources, and underlying network conditions, given the fact that the power management schemes as well as the nonlinear battery effects also affect the system power consumption of mobile devices. In this paper, we review the recent advances in power-aware mobile multimedia, especially the adaptation technologies applied in video coding and delivery. In addition, the major research challenges in the field are demonstrated and discussed, which include power-management for mobile devices, rate-distortion-complexity optimized video codec design, and computational complexity and power aware cross-layer design and optimization. At the end, we propose a number of future research directions for audiences to continue investigation in this field.

Cost-distortion optimized unequal error protection for object-based video communications

Object-based video coding is a relatively new technique to meet the fast growing demand for interactive multimedia applications. Compared with conventional frame-based video coding, it consists of two types of source data: shape information and texture information. Recently, joint source-channel coding for multimedia communications has gained increased popularity. However, very limited work has been conducted to address the problem of joint source-channel coding for object-based video. In this paper, we propose a cost-distortion optimal unequal error protection (UEP) scheme for object-based video communications. Our goal is to achieve the best video quality (minimum total expected distortion) with constraints on transmission cost and delay in a lossy network environment. The problem is solved using Lagrangian relaxation and dynamic programming. The performance of the proposed scheme is tested using simulations of a narrow-band block-fading wireless channel with additive white Gaussian noise and a simplified differentiated services Internet channel. Experimental results indicate that the proposed UEP scheme can significantly outperform equal error protection methods.

Joint source-channel coding for wireless object-based video communications utilizing data hiding

In recent years, joint source-channel coding for multimedia communications has gained increased popularity. However, very limited work has been conducted to address the problem of joint source-channel coding for object-based video. In this paper, we propose a data hiding scheme that improves the error resilience of object-based video by adaptively embedding the shape and motion information into the texture data. Within a rate-distortion theoretical framework, the source coding, channel coding, data embedding, and decoder error concealment are jointly optimized based on knowledge of the transmission channel conditions. Our goal is to achieve the best video quality as expressed by the minimum total expected distortion. The optimization problem is solved using Lagrangian relaxation and dynamic programming. The performance of the proposed scheme is tested using simulations of a Rayleigh-fading wireless channel, and the algorithm is implemented based on the MPEG-4 verification model. Experimental results indicate that the proposed hybrid source-channel coding scheme significantly outperforms methods without data hiding or unequal error protection

Rate-distortion optimal bit allocation for object-based video coding

In object-based video encoding, the encoding of the video data is decoupled into the encoding of shape, motion, and texture information, which enables certain functionalities, like content-based interactivity and content-based scalability. The fundamental problem, however, of how to jointly encode this separate information to reach the best coding efficiency has not been studied thoroughly. In this paper, we present an operational rate-distortion optimal scheme for the allocation of bits among shape, motion, and texture in object-based video encoding. Our approach is based on Lagrangian relaxation and dynamic programming. We implement our algorithm on the MPEG-4 video verification model, although it is applicable to any object-based video encoding scheme. The performance is accessed utilizing a proposed metric that jointly captures the distortion due to the encoding of the shape and texture. Experimental results demonstrate that the gains of lossy shape encoding depend on the percentage the shape bits occupy out of the total bit budget. This gain may be small or may be realized at very low bit rates for certain typical scenes.

Availability-Aware Multimedia Scheduling in Heterogeneous Wireless Networks

Scheduling multimedia applications over heterogeneous wireless networks is a challenging issue due to quality-of-service (QoS) requirements, different resource requests, and dynamically available network resources. Resource availability is an important constraint for the adaptive usage of limited network resources. However, it has not been well studied in the literature. In this paper, we propose a novel distributed availability-aware adaptive rate-allocation scheme for multimedia applications. A general distortion model is first proposed, which is highly dependent on the applications rate-distortion characteristics and the availability model. Then, a near-optimal rate-allocation approach is presented that jointly considers QoS, reliability, and availability. Numerical results indicate that the proposed scheme is able to achieve higher QoS under various environments compared with other reference approaches.

Application-Centric Routing for Video Streaming over Multi-hop Wireless Networks

Routing for video transmissions over multi-hop wireless networks has gained increasing research interest in recent years. However, most existing works only focus on how to satisfy the network-oriented QoS, such as, throughput, delay, and packet loss rate rather than the user perceived quality. Although there are some research efforts which use application-centric video quality as the routing metric, the calculation of video quality is based on some predefined rate-distortion function or model without exploring the impact of video coding and decoding (including error concealment) on network path selection and the resulting received video quality. Moreover, unlike network-centric routing metrics, such as, hop count, average delay or average success probability of packet transmission, video distortion cannot be calculated either additively or multiplicatively in a hop-by-hop fashion due to the dependency among packets introduced by error concealment. As a result, most existing works use either exhaustive search or heuristic methods to find the optimal path, which leads to high computational complexity or suboptimal solutions to the routing problem of video transmission. In this paper, we propose an application-centric routing framework for real-time video transmission over multi-hop wireless networks, where expected video distortion is used as the routing metric. The major contributions of this work are: 1) the development of an efficient routing algorithm with the routing metric in terms of the expected video distortion being calculated on-the-fly, and 2) the development of a quality-driven cross-layer optimization framework to enhance the flexibility and robustness of routing by the joint optimization of routing path selection and video coding, thereby maximizing the user perceived video quality under a given video playback delay constraint. Both theoretical and experimental results demonstrate that the proposed quality-driven application-centric routing approach can achieve a superior performance over existing network-centric routing approaches.