Chung-Neng Wang

A robust fine granularity scalability using trellis-based predictive leak

Recently, the MPEG-4 committee has approved the MPEG-4 fine granularity scalability (FGS) profile as a streaming video tool. We propose novel techniques to improve further the temporal prediction at the enhancement layer so that coding efficiency is superior to the existing FGS. Our approach utilizes two parameters, the number of bitplanes, /spl beta/ (0/spl les//spl beta//spl les/maximal number of bitplanes), and the amount of predictive leak, /spl alpha/ (0/spl les//spl alpha//spl les/1), to control the construction of the reference frame at the enhancement layer. Parameters /spl alpha/ and /spl beta/ can be selected for each frame to provide tradeoffs between coding efficiency and error drift. Our approach offers a general and flexible framework that allows further optimization. It also includes several well-known motion-compensated FGS techniques as special cases with particular sets of /spl alpha/ and /spl beta/. We analyze the theoretical advantages when /spl alpha/ and /spl beta/ are used, and provide an adaptive technique to select /spl alpha/ and /spl beta/, which yields an improved performance as compared to that of fixed parameters. An identical technique is applied to the base layer for further improvement. Our experimental results show over 4 dB improvements in coding efficiency using the MPEG-4 testing conditions. Removal of error propagation is demonstrated with several typical channel transmission scenarios.

A platform-based MPEG-4 advanced video coding (AVC) decoder with block level pipelining

We present a baseline MPEG-4 AVC (advanced video coding) decoder based on an optimized platform-based design methodology. With this methodology, we jointly optimize the software and hardware design of the decoder. Overall decoding throughput is increased by synchronizing the software and the dedicated co-processors. The synchronization is achieved at macroblock-level pipelining. In addition, we optimize the decoder software by enhancing the frame buffer management, boundary padding, and content aware inverse transform. To speed up motion compensation and inverse transform, which are the most computationally intensive modules, two dedicated acceleration modules are realized. For comparison, the proposed prototype decoder and MPEG-4 AVC reference decoder are evaluated on an ARM platform, which is one of most popular portable devices. Our experiments show that the throughput of the MPEG-4 reference decoder can be improved by 6 to 7 times. On an ARM966 board, the optimized software without hardware acceleration can achieve a decoding rate up to 5 frames per second (fps) for QCIF video sequences. With the dedicated accelerators, the overall throughput is increased by about 30% to reach 6.6 fps on the average and is up to 10.3 fps for slow motion video sequences.

A hierarchical N-Queen decimation lattice and hardware architecture for motion estimation

A subsampling structure, an N-Queen lattice, for spatially decimating a block of pixels is presented. Despite its use for many applications, we demonstrate that the N-Queen lattice can be used to speed up motion estimation with nominal loss of coding efficiency. With a simple construction, the N-Queen lattice characterizes the spatial features in the vertical, horizontal, and diagonal directions for both texture and edge areas. Especially in the 4-Queen case, every skipped pixel has the minimal and equal distance of unity to the selected pixel. It can be hierarchically organized for variable nonsquare block-size motion estimation. Despite the randomized lattice, we design compact data storage architecture for efficient memory access and simple hardware implementation. Our simulations show that the N-Queen lattice is superior to several existing sampling techniques with improvement in speed by about N times and small loss in peak SNR (PSNR). The loss in PSNR is negligible for slow-motion video sequences and is less than 0.45 dB at worst for high-motion estimation sequences.

A robust fine granularity scalability using trellis based predictive leak

Recently, the MPEG-4 committee has approved the MPEG-4 Streaming Video Profile (SVP) that includes the Fine Granularity Scalability (FGS) as a new coding tool. In this paper, we propose novel techniques to further improve the temporal prediction schemes at the enhancement layer so that both the coding efficiency and error resilience are superior to the existing FGS while the fine granularity scalability is preserved. Our approach utilizes two parameters, the number of bitplanes /spl beta/(0/spl les/ /spl beta//spl les/maximal number of bitplanes) and the amount of predictive leak /spl alpha/ (0/spl les//spl alpha//spl les/1), to control the construction of the reference frame at the enhancement layer. These parameters /spl alpha/ and /spl beta/ can be temporally selected to provide tradeoffs between coding efficiency, error propagation and the predictive drift. It also encompasses several well-known FGS techniques as special cases for particular sets of /spl alpha/ and /spl beta/. The experimental results show over 2 dB improvement in coding efficiency using the MPEG-4 testing conditions.

A Software-Hardware Co-Implementation of MPEG-4 Advanced Video Coding (AVC) Decoder with Block Level Pipelining

We present a baseline MPEG-4 Advanced Video Coding (AVC) decoder based on the methodology of joint optimization of software and hardware. The software is first optimized with algorithm improvements for frame buffer management, boundary padding, content-aware inverse transform and context-based entropy decoding. The overall decoding throughput is further enhanced by pipelining the software and the dedicated hardware at macroblock level. The decoder is partitioned into the software and hardware modules according to the target frame rate and complexity profiles. The hardware acceleration modules include motion compensation, inverse transform and loop filtering. By comparing the optimized decoder with the committee reference decoder of Joint Video Team (JVT), the experimental results show improvement on the decoding throughput by 7 to 8 times. On an ARM966 board, the optimized software without hardware acceleration can achieve a decoding rate up to 5.9 frames per second (fps) for QCIF video source. The overall throughput is improved by another 27% to 7.4 fps on the average and up to 11.5 fps for slow motion video sequences. Finally, we provide a theoretical analysis of the ideal performance of the proposed decoder.

Efficient FGS to single layer transcoding

This paper presents a novel architecture for the transcoding of FGS (fine granularity scalability) coded video to a single-layer format. Results indicate that the proposed architecture has almost the same performance as the cascaded transcoding method, but with much lower complexity.

A hierarchical decimation lattice based on N-queen with an application for motion estimation

We present a novel technique, N-queen lattice, to spatially subsample a block of pixels. Although this lattice is pertinent to many applications, we present an application to speed up motion estimation with minimal loss of coding efficiency. The N-queen lattice is constructed to characterize spatial features in all directions. It can be hierarchically organized for motion estimation with variable nonsquare block size. Despite the randomized lattice structure, we demonstrate that it is possible to achieve compact data storage architecture for efficient memory access and simple hardware implementation. Our simulations show that the N-queen lattice is superior to several existing sampling techniques with improvement in speed by about N times and small loss in peak SNR.

Fast Motion Estimation Using N-Queen Pixel Decimation

We present a technique to improve the speed of block motion estimation using only a subset of pixels from a block to evaluate the distortion with minimal loss of coding efficiency. To select such a subset we use a special sub-sampling structure, N-queen pattern. The N-queen pattern can characterize the spatial information in the vertical, horizontal and diagonal directions for both texture and edge features. In the 4-queen case, it has a special property that every skipped pixel has the minimal and equal distance of one to the selected pixel. Despite of the randomized pattern, our technique has compact data storage architecture. Our results show that the pixel decimation of N-queen patterns improves the speed by about N times with small loss in PSNR. The loss in PSNR is negligible for slow motion video sequence and has 0.45 dB loss in PSNR at worst for high motion video sequence.

A fast H.264-based picture-in-picture (PIP) transcoder

The work presents a novel and fast transcoding scheme with PIP (picture-in-picture) functionality, based on the H.264/AVC standard. We seamlessly achieve significant speedup and maintain identical video quality for the composite video content using two H.264 bitstreams. The proposed partial re-encoding transcoder (PRET) method, based on the intra-prediction refinement, inter-prediction refinement and rate-distortion optimization, has sped up by 8 times compared with rate-distortion (R-D) based optimization alone. In addition, the performance of the PRET is better than that of the R-D re-encoding algorithm.

A multiple-window video embedding transcoder based on H.264/AVC standard

This paper proposes a low-complexity multiple-window video embedding transcoder (MW-VET) based on H.264/AVC standard for various applications that require video embedding services including picture-in-picture (PIP), multichannel mosaic, screen-split, pay-per-view, channel browsing, commercials and logo insertion, and other visual information embedding services. The MW-VET embeds multiple foreground pictures at macroblock-aligned positions. It improves the transcoding speed with three block level adaptive techniques including slice group based transcoding (SGT), reduced frame memory transcoder (RFMT), and syntax level bypassing (SLB). The SGT utilizes prediction from the slice-aligned data partitions in the original bitstreams such that the transcoder simply merges the bitstreams by parsing. When the prediction comes from the newly covered area without slice-group data partitions, the pixels at the affected macroblocks are transcoded with the RFMT based on the concept of partial reencoding to minimize the number of refined blocks. The RFMT employs motion vector remapping (MVR) and intra mode switching (IMS) to handle intercoded blocks and intracoded blocks, respectively. The pixels outside the macroblocks that are affected by newly covered reference frame are transcoded by the SLB. Experimental results show that, as compared to the cascaded pixel domain transcoder (CPDT) with the highest complexity, our MW-VET can significantly reduce the processing complexity by 25 times and retain the rate-distortion performance close to the CPDT. At certain bit rates, the MW-VET can achieve up to 1.5 dB quality improvement in peak signal-to-noise-ratio (PSNR).