Chia-Chiang Ho

A user-attention based focus detection framework and its applications

In this paper, a generic user-attention based focus detection framework is developed to capture user focus points for video frames. The proposed framework considers both bottom-up and top-down attentions, and integrates both image-based and video-based visual features for saliency map computation. For efficiency purpose, the number of adopted features is kept as few as possible. The realized framework is extensible and flexible in integrating more features with a variety of fusion schemes. One application of the proposed framework, the user-assisted spatial resolution reduction, has also been addressed.

MMX-based DCT and MC algorithms for real-time pure software MPEG decoding

To overcome the difficulties of computation-intensive multimedia applications, the development groups of major CPU manufacturers, such as Intel/sup TM/ and Digital/sup TM/, have decided to include new instruction sets into their CPU families to increase their multimedia handling ability. The newly introduced instruction set is basically in a Single Instruction Multiple Data (SIMD) Stream operation type. For practical purposes (e.g, the trade off between the complexity of hardware implementation and the so-obtained performance improvement), they use a reduced SIMD instruction set instead of the full one. Taking Intel as an example, the new instruction set is composed of 57 operations called the MultiMedia eXtension (MMX) instruction set. Nowadays, how to fully utilize the power of the embedded instruction set for providing various multimedia applications becomes an interesting and important issue. We demonstrate an efficient realization, based on the new MMX instruction set of the block Inverse Discrete Cosine Transform (IDCT) and Motion Compensation (MC) which are kernel components of the block based decoding standards, such as MPEG-1, MPEG-2, H.261 and H.263. The convincing results show that: with the addition of the proper SIMD instruction set, the pure software solution for complicated multimedia applications (such as real time MPEG video decoding) becomes feasible.

A practical foveation-based rate-shaping mechanism for MPEG videos

Foveation is one of the nonuniform resolution properties of the human visual system. Recently, different foveation models are proposed and utilized for image and video coding, for the sake of bit-rate saving with no or minor perceptual quality distortion. In the first part of this paper, we propose an efficient and practical DCT-domain foveation model, which is deduced from existing experimental results. In the second part, we present a foveation-based rate-shaping mechanism for MPEG bitstreams, as an application example of the proposed foveation model. The rate shaper is based on eliminating DCT coefficients embedded in MPEG bitstreams. An efficient rate-shaping mechanism is developed to meet various bit-rate requirements. Our simulation confirmed that the proposed foveation model and the rate-shaping mechanism are practical for real-world usage.

A DCT-domain H.263 based video combiner for multipoint continuous presence video conferencing

This paper proposes an H.263 based DCT-domain video combiner which is suitable for a multipoint continuous presence videoconference system and supports up to six conferees. The main issues of the H.263 video combiner are discussed. A software-based combiner is implemented and tested for various test sequences. The combined videos have promising quality and the combiner is considered very efficient for practical usage.

Software-controlled cache architecture for energy efficiency

Power consumption is an important design issue of current multimedia embedded systems. Data caches consume a significant portion of total processor power for multimedia applications because they are data intensive. In an integrated multimedia system, the cache architecture cannot be tuned specifically for an application. Therefore, a significant amount of cache energy is actually wasted. In this paper, we propose the software-controlled cache architecture that improves the energy efficiency of the shared cache in an integrated multimedia system on an application-specific base. Data types in an application are allocated to different cache regions. On each access, only the allocated cache regions need to be activated. We test the effectiveness of the software-controlled cache of the MPEG-2 software decoder. The results show up to 40% of cache energy reduction on an ARM-like cache architecture without sacrificing performance.

A Foveation-Based Rate Shaping Mechanism for MPEG Videos

Considering human perceptual property and user specific need can help to reduce the bandwidth requirement for streaming compressed videos over networks. In this paper, we present a foveation-based rate shaping mechanism for MPEG bitstreams. The rate shaper is designed based on an experimental-proven foveation model, and achieve rate reduction by eliminating DCT coefficients embedded in MPEG bitstreams. An efficient rate control mechanism is developed to meet various bitrate requirements, and a real-time implementation is developed to confirm that the proposed scheme is practical for real world usage.

A low-cost media-processor based real-time MPEG-4 video decoder

In this paper by implementing a realistic MPEG-4 core profile decoder over the TriMedia/spl trade/ DSP chip, we investigate a series of important issues related to the use of a media-processor to satisfy multimedia compression and processing requirements. The realistic MPEG-4 decoder optimized implementation by this DSP chip can support resolutions up to 4 CIF (720/spl times/576) at 30 fps. The issues we investigated include the overview of characteristics of media-processor architecture, the speedup of the MPEG-4 decoder based on the TriMedia/spl trade/ DSP chip, multimedia co-processor or hardware accelerator based approaches, Very Long Instruction Word (VLIW) and Single Instruction Multiple Data (SIMD) programming technique and the media-processor based solution for set-top box applications.

Smart cache: an energy-efficient D-cache for a software MPEG-2 video decoder

Power consumption is an important design issue of current embedded systems. Data caches consume a significant portion of total processor power for data intensive applications. In this paper, we propose to utilize application-specific information for cache resource allocation to achieve energy saving, including cache bypassing, the mini-cache and way-partition. We use a software MPEG-2 video decoder as our first targeted application. Cache bypassing excludes data types that have little reuse from the L1 cache. The mini-cache stores data types with high access frequency and small memory footprints to a small on-chip memory area. The way-partition mechanism maps program data structures to different ways of caches and enables only the matching ways on each access. The results show up to 40% of cache energy reduction without sacrificing performance.

Building MPEG-7 transcoding hints from intrinsic characteristics of MPEG videos

Transcoding hints are defined in the MPEG-7 standard to provide better quality or reduced complexity of transcoding applications. The paper: 1) defines practical transcoding hint hierarchies from both structural and semantic perspectives; 2) addresses three application scenarios from which transcoding hints can be generated; 3) proposes architectures of transcoding hint generators for different application scenarios, based on retrieved low-level MPEG characteristics.

User-oriented approach in spatial and temporal domain video coding

Scalable coding techniques usually focus on enhancing the quality of the whole frame when more bandwidth is available. However, not all regions in a frame are equally important to user perception. This paper proposed a user-oriented approach to facilitate scalable coding in spatial and temporal domains. By combining user attention and foveation models, the regions of users focus in a frame (spatial domain) or the segment of a video clip with steady saliency values (temporal domain) are prioritized as far as perceived quality is concerned.