Wei Siong Lee

Complexity Scalable H.264/AVC Encoding

The H.264/AVC video coding standard encapsulates the most advanced video coding tools. Since the various techniques that lead to better coding efficiency of the coding standard also inevitably increase the complexity of the video encoder, real-time H.264 encoding of video streams is a challenging task. If available computational resource does not allow the entire encoding process to be carried out in time, a complexity scalable technique that ensures a graceful degradation of coding performance will be a valuable tool. We designed a video encoding scheme that allows the rate distortion (R-D) process to be carried out in a complexity scalable fashion. Our proposed singularly parameterized complexity scalable scheme allows the control of complexity-coding performance tradeoff when available resources are limited and the optimal R-D performance is unattainable.

Complexity scalable rate-distortion optimization for H.264/AVC

The H.264/AVC video coding standard encapsulates the most advanced video coding tools. The various techniques that lead to better coding efficiency of the coding standard also inevitably increase the complexity of the video encoder. Thus, real-time encoding of video streams with H.264 coding standard is a challenging task. If available computational resource does not allow the entire encoding process to be carried out in time, a complexity scalable technique that ensures a graceful degradation of coding performance will be a useful tool. This work proposes a singularly-parameterized complexity scalable rate-distortion framework for H.264/AVC encoders.

Complexity-Rate-Distortion Optimization for Real-Time H.264/AVC Encoding

In this paper, we introduce a singularly-parameterized complexity scalable H.264-compliant encoder. Through modeling its complexity-rate-distortion relationships, we derive optimized operating mode of the encoder (rate and complexity) and show through experiment that such optimization can help a video encoder operate within rate and time constraints. The design of the complexity scalable encoding scheme enables the encoder to perform optimization while taking into consideration the availability of computational resource. This extension of traditional rate-distortion optimization is necessary when time or power constraints do not allow a video encoder to achieve rate-distortion optimized coding performance. Our optimization scheme outputs parameters that allow the encoder to be as close to being rate-distortion optimized as possible, within rate and complexity constraints. Index Terms—H.264, power consumption, complexity

Complexity control and computational resource allocation during H.264/SVC encoding

In this paper, a singularly parameterized complexity-scalable scheme is proposed for designing power-aware and power-adaptive H.264 video encoder. The proposed scheme enables adaptive control of the trade-off between the encoders complexity and coding performance by dynamically updating a control parameter during encoding. This enables the encoder to perform optimally on a variety of computing platforms and runtime environments by adaption to instantaneous available computing resources and varying source video characteristics. The proposed scheme also allows computational resource allocation across layers in SVC encoders.

Pattern selection for error-resilient slice interleaving based on receiver error concealment technique

Video communication is often afflicted by packet loss over unreliable networks. To alleviate the quality degradation in the situation of packet loss, this paper proposes a novel error resilient slice interleaving method for compressed video packetization. Each packet is constructed by interleaving independently decodable slices of the coded video bit-stream in consecutive frames. The optimal interleaving pattern for minimizing the overall distortion of the decoded video, subject to a delay constraint, is determined based on the error concealment technique employed at the user end. Compared with traditional error-resilient methods, this scheme greatly improves the overall performance without requiring an increase in bit-rate. Experimental results on different video sequences demonstrate the effectiveness of the proposed approach.

Efficient rate-quantization model for frame level rate control in spatially scalable video coding

This paper addresses the quantization parameter (QP) selection problem in H.264 spatially scalable video coding (SVC). For frame level rate control in SVC, it is important to have an accurate QP selection scheme such that the target bit rate of each coding layer will be achieved. In this paper, we present an adaptive rate-quantization (R-Q) model to select the appropriate QP for each inter frame in spatial enhancement layers according to the target bit rate. The proposed algorithm introduces an efficient coding complexity estimation method by taking into consideration the inter-layer dependency between different spatial layers. Based on the coding complexity information, the R-Q model parameters can be adaptively updated. Experimental results demonstrate that compared to the traditional method, the proposed method provides better estimation accuracy for bit rate in terms of target bits mismatch error and thus it is very desirable for H.264/SVC rate control applications.

Slice error concealment based on size-adaptive SSIM matching and motion vector outlier rejection

Consecutive corrupted MBs or slice errors are commonly seen in modern video transmission systems. Temporal error concealment is an effective approach to reduce the impact of errors. Conventional temporal error concealment techniques recover slice errors on a MB basis. We propose a new novel temporal scheme for slice error concealment based on a size-adaptive region basis. Size-adaptive region boundary matching can significantly reduce the misalignments or side mismatches that are commonly seen in MB-based approaches. Instead of using MSE or MAE as the matching criteria, We propose a matching criterion based on structural similarity (SSIM). Our proposed matching criterion can achieve better structural alignments and better visual perceptual quality. We also propose a method to detect motion vector outliers at the last stage of the error concealment scheme, which can further improve the visual quality of the concealed videos. Our proposed scheme conceals slice errors with good accuracy as well as low computation complexity, and it can be applied for real-time applications.

A distance-based slice interleaving scheme for robust video transmission over error-prone networks

Video transmission over wireless networks suffers from packet loss due to either temporary packet drop or fading-induced bit errors. To ensure that the quality of the decoded video is not overly affected by the channel unreliability, the video applications have to provide sufficient robustness. In this paper, a novel slice interleaving algorithm is proposed for compressed video packetization. Each packet is constructed by interleaving independently decodable slices of the coded video bit-stream in a group of frames. We group slices into packets according to the maximum minimal distance of these slices. The scattered pattern guarantees that each lost slice has, as many as possible, spatial and temporal neighbors that belong to other correctly received packets. This scheme greatly strengthens the error concealment performance compared with traditional methods. Experimental results demonstrate the superiority of the proposed algorithm.

MP4 File Creator for SVC Adaptive Video Streaming

With the advances in the Internet and mobile networks, consumers now wish to enjoy their live and on-demand multimedia contents anytime, anywhere and across any device. Scalable Video Coding (SVC) provides a very attractive solution to achieve this goal. However, SVC streams themselves are not stream-able by a streaming server. The SVC streams must be stored in a proper file format to facilitate packet-based streaming. MP4 files are designed to contain timed media for streaming purposes. An MP4 file contains media sample data, sample timing information, and sample packetization information, among which sample times are the most important information. In this paper, we take advantage of the unique SVC hierarchical B-pictures coding structure and propose to use a fixed initial delay ((GOP/2)*duration) for SVC composition times instead of parsing the whole file to find out the proper initial delay. We also propose that sample times of the full SVC streams should be obtained firstly, and then the sample times for SVC sub-streams are simply matched from the sample times of the full SVC streams, instead of calculating sample times for each sub-stream, for both single-RTP-stream and multiple-RTP-stream packetization modes. This paper also highlights and addresses some other key implementation issues of the MP4 creator for SVC adaptive video streaming.

Real-time software MPEG-2 TO H.264 video transcoding

The paper describes a fast MPEG-2 to H.264 transcoder by leveraging the MPEG-2 metadata set to reduce the computational complexity of the H.264 encoding process. With the proposed fast transcoding algorithms, our software-based transcoder can perform real-time transcoding a MPEG-2 movie from a DVD disc (at D1 resolution and full frame-rate) and simultaneously performing RTP/UDP streaming of the H.264 video with AC-3 audio over IP networks using only a 1.6 GHz PC notebook. We can achieve about 20-30% faster than the full re-encoding method even at the same target bit-rate and without visible quality degradation. Benchmarking with existing software systems has shown similar results.