Yi-Shin Tung

An adaptive edge detection based colorization algorithm and its applications

Colorization is a computer-assisted process for adding colors to grayscale images or movies. It can be viewed as a process for assigning a three-dimensional color vector (YUV or RGB) to each pixel of a grayscale image. In previous works, with some color hints the resultant chrominance value varies linearly with that of the luminance. However, it is easy to find that existing methods may introduce obvious color bleeding, especially, around region boundaries. It then needs extra human-assistance to fix these artifacts, which limits its practicability. Facing such a challenging issue, we introduce a general and fast colorization methodology with the aid of an adaptive edge detection scheme. By extracting reliable edge information, the proposed approach may prevent the colorization process from bleeding over object boundaries. Next, integration of the proposed fast colorization scheme to a scribble-based colorization system, a modified color transferring system and a novel chrominance coding approach are investigated. In our experiments, each system exhibits obvious improvement as compared to those corresponding previous works.

Quality Enhancement of Frame Rate Up-Converted Video by Adaptive Frame Skip and Reliable Motion Extraction

Frame rate up-conversion is a postprocessing tool to convert the frame rate from a lower number to a higher one. It is a useful technique for a lot of practical applications, such as display format conversion, low bit rate video coding and slow motion playback. Unlike traditional approaches, such as frame repetition or linear frame interpolation, motion-compensated frame interpolation (MCFI) technique is more efficient since it takes block motion into account. In this paper, by considering the deficiencies of previous works, new criteria and coding schemes for improving motion derivation and interpolation processes are proposed. Next, for video coding applications, adaptive frame skip is executed at the encoder side to maximize the power of MCFI so that the quality of interpolated frames is guaranteed. Experimental results show that our proposal effectively enhances the overall quality of the frame rate up-converted video sequence, both subjectively and objectively.

MMX-based DCT and MC algorithms for real-time pure software MPEG decoding

To overcome the difficulties of computation-intensive multimedia applications, the development groups of major CPU manufacturers, such as Intel/sup TM/ and Digital/sup TM/, have decided to include new instruction sets into their CPU families to increase their multimedia handling ability. The newly introduced instruction set is basically in a Single Instruction Multiple Data (SIMD) Stream operation type. For practical purposes (e.g, the trade off between the complexity of hardware implementation and the so-obtained performance improvement), they use a reduced SIMD instruction set instead of the full one. Taking Intel as an example, the new instruction set is composed of 57 operations called the MultiMedia eXtension (MMX) instruction set. Nowadays, how to fully utilize the power of the embedded instruction set for providing various multimedia applications becomes an interesting and important issue. We demonstrate an efficient realization, based on the new MMX instruction set of the block Inverse Discrete Cosine Transform (IDCT) and Motion Compensation (MC) which are kernel components of the block based decoding standards, such as MPEG-1, MPEG-2, H.261 and H.263. The convincing results show that: with the addition of the proper SIMD instruction set, the pure software solution for complicated multimedia applications (such as real time MPEG video decoding) becomes feasible.

DSP-based multi-format video decoding engine for media adapter applications

There are four major components in a generic digital home framework: media center, set-top box, media adapter and portable viewer. In this paper, we present a DSP-based multi-format video decoding engine for realizing media adapter application. By choosing a powerful DSP core and designing an efficient and configurable software engine, the proposed solution is believed to possess low-cost and high-flexibility advantages at the same time. In our current implementation, the video decoding engine can decode most of the major video formats in the market, such as MPEG-1/2/4, H.264/AVC baseline/main, VC-I and its precedents, and some other MPEG-4-like formats. Simulation results demonstrate the efficiency of the decoding engine: videos up to SD resolution can be decoded seamlessly. The engine has been successfully integrated into an available media adapter module and the corresponding product. Our future work is to enhance the engine performance capable of handling HD contents.

The optimization of H.264/AVC baseline decoder on low-cost TriMedia DSP processor

The emerging video coding standard, H.264/AVC, exhibits the unprecedented coding performance. Comparing to traditional coders, e.g., MPEG-2 and MEPG-4 ASP, about half bitrate saving is shown in the official verification test. Such outstanding performance makes it become the video compression candidate for the upcoming HD-DVD. As a side effect, it was also blamed that H.264/AVC is much more logically complex and requires more computation power than any of the existing standards. A low-cost and efficient implementation of the international standard hence plays an important role of its success. In this paper, we realize an H.264/AVC baseline decoder by a low-cost DSP processor, i.e., Philips’ TriMedia TM-1300, and illustrate that less computation demand for H.264/AVC decoding becomes feasible by using effective software core. To this end, we first consider different approaches and take advantage of SIMD instruction set to optimize critical time-consuming coding modules, such as the fractional motion compensation, spatial prediction and inverse transform. Next, we also present some other optimization approaches for entropy decoding and in-loop deblocking filtering, even though they cannot get benefits from utilizing SIMD. In our experiments, by exploiting appropriate instruction level parallelism and efficient algorithms, the decoding speed can be improved by a factor of 8~10; a CIF video sequence can be decoded at up to 19.74~28.97 fps on a 166-MHz TriMedia TM-1300 processor compared to 2.40~2.98 fps by the standard reference software.

An efficient streaming and decoding architecture for stored FGS video

Fine granularity scalability (FGS) is the latest video-coding tool provided in Amendment 2 of the MPEG-4 standard. By taking advantage of bitplane coding of DCT residues, the compressed bitstream can tie truncated at any location to support the finest rate scalability in the enhancement layer. However, both frame buffer scanning several times in bitplane decoding and frame duplication simultaneously for base and enhancement layer decoding make FGS difficult in its implementation. We propose a corresponding pair of efficient streaming schedule and pipeline decoding architecture to deal with the prescribed problems. The design may be applied to the case of streaming stored FGS videos and benefit FGS-related applications.

DSRA: a block matching algorithm for near-real-time video encoding

The correlations of motion vectors between neighboring image blocks have long been applied to improve the efficiency of block matching algorithms. A block matching algorithm with a dynamic search range is presented. The search range is adjusted based on the correlations of the motion vectors between neighboring blocks. It is shown by simulation results that the proposed method can increase the speed of an MPEG-1 encoding system up to a ratio of 13.3 and the bit-rate by 0.025 bpp, in average. It is therefore beneficial to embed the proposed method into a motion-based encoding system.

Memory Efficient Hierarchical Lookup Tables for Mass Arbitrary-Side Growing Huffman Trees Decoding

This paper addresses the optimization problem of minimizing the number of memory access subject to a rate constraint for any Huffman decoding of various standard codecs. We propose a Lagrangian multiplier based penalty-resource metric to be the targeting cost function. To the best of our knowledge, there is few related discussion, in the literature, on providing a criterion to judge the approaches of entropy decoding under resource constraint. The existing approaches which dealt with the decoding of the single-side growing Huffman tree may not be memory-efficient for arbitrary-side growing Huffman trees adopted in current codecs. By grouping the common prefix part of a Huffman tree, in stead of the commonly used single-side growing Huffman tree, we provide a memory efficient hierarchical lookup table to speed up the Huffman decoding. Simulation results show that the proposed hierarchical table outperforms previous methods. A Viterbi-like algorithm is also proposed to efficiently find the optimal hierarchical table. More importantly, the Viterbi-like algorithm obtains the same results as that of the brute-force search algorithm.

An efficient encryption scheme for MPEG video

We propose an encryption scheme for MPEG video, which uses a permuted Huffman table to encode the input symbols, and then rotates and XORs (exclusive or) the encoded bit stream. This scheme adds very little computational overhead to the MPEG video compression process, and therefore the implementation is fast enough to meet the real-time requirement of MPEG video applications. In our analysis, it is also robust to both plaintext and ciphertext attacks. In addition, our scheme can be applied to all MPEG compression standards.

Boundary-energy sensitive visual de-blocking for H.264/AVC coder

Finding out the better parameter set (OffsetA and OffsetB) for conducting the de-blocking process of H.264/AVC, is capable of improving visual quality, said eliminating the resultant blocking artifact. Identifying which edges belong to blocking regions relies on the perceptual judging process of human beings. In fact, this subjective assessment may not exactly match existing objective assessing measurements, and the meaning of high PSNR does not always stand for less blocking artifacts. In this paper, we first introduce a new criterion for measuring the block boundary distortion by comparing the source video and the reconstructed video prior to the deblocking process. By jointly optimizing the objective picture quality and the blocky energy, the deblocking parameters decision process can find out a good balance between signal matching and blocky elimination, and therefore, maximize the effect of the built-in deblocking process. In our experiments, the proposed method can efficiently pick a better deblocking parameter set from all 169 possibilities for each coded frame and result in a better visual quality.