Chi-Sun Tang

Content-adaptive inverse tone mapping

Tone mapping is an important technique used for displaying high dynamic range (HDR) content on low dynamic range (LDR) devices. On the other hand, inverse tone mapping enables LDR content to appear with an HDR effect on HDR displays. The existing inverse tone mapping algorithms usually focus on enhancing the luminance in over-exposed regions with less (or even no) effort on the process of the wellexposed regions. In this paper, we propose an algorithm with not only enhancement in the over-exposed regions but also in the remaining well-exposed regions. This paper provides an ”histogram-based” method for inverse tone mapping. The proposed algorithm contains a content-adaptive inverse tone mapping operator, which has different responses with different scene characteristics. Scene classification is included in this algorithm to select the environment parameters. Lastly, enhancement of the over-exposed regions, which reconstructs the truncated information, is performed.

A multimedia semantic analysis SoC (SASoC) with machine-learning engine

Advances in semiconductors and developments in machine learning [1] have led to versatile multimedia applications with semantic processing abilities. Real-time applications, such as face detection, facial-expression recognition, scene analysis [2] and object recognition [3], have become indispensable functionality for Consumer Electronic (CE) products. To deal with complicated video-processing algorithms for multimedia content analysis, many powerful processors have been reported [2–5]. Although these processors can speed up video-processing tasks with massively parallel processing elements, they only focus on the feature-extraction parts, and there is no specialized hardware to support different kinds of advanced machine-learning algorithms, which require extensive computations. In this paper, a Semantic Analysis SoC (SASoC) that accelerates video processing and machine learning simultaneously, is developed to meet the demands of the near future.

A flexible fully hardwired CABAC encoder for UHDTV H.264/AVC high profile video

In this paper, a flexible CABAC encoder architecture for H.264/AVC encoder applications up to UHDTV (7680×4320) resolution is proposed. Stages of CABAC encoding are analyzed and a generalized CABAC architecture is designed. The parallel binarizer and context modeler (BCM) and multi-symbol binary arithmetic coder (MSBAC) is coupled together by a variable throughput buffer and packers (VTBAP) for throughput matching. Syntax elements (SEs) are analyzed thoroughly and various SEs processing engines are proposed to achieve parallelism for performance with high degree of flexibility for CABAC designers. Special attentions have been paid to the feeding of SEs into BCM that is not discussed in most other works. Without the bubble-free access control and the bubble-free feeding of SEs, the high throughput of BCM and MSBAC engine will not be possible to integrate with the rest of the encoder engine, otherwise external pre-processing has to be applied for SEs feeding. Flexibilities of architecture and level of parallelism are incorporated into a CABAC auto generating scheme that can produce the CABAC configurations according to user requirements. Towards a 0.13 μm CMOS technology, the highest performance design generated by the automatic generation scheme can encode 4.86 bins per cycle on the average, and it provides a throughput of 1234 Mbin/s. The proposed CABAC encoder architecture has been integrated into a H.264/AVC encoder of a multimedia SoC successfully.

Tera-Scale Performance Machine Learning SoC (MLSoC) With Dual Stream Processor Architecture for Multimedia Content Analysis

A new machine learning SoC (MLSoC) for multimedia content analysis is implemented with 16-mm2 area in 90-nm CMOS technology. Different from traditional VLSI architectures, it focuses on the coacceleration of computer vision and machine learning algorithms, and two stream processors with massively parallel processing elements are integrated to achieve tera-scale performance. In the dual stream processor (DSP) architecture, the data are transferred between processors and the high-bandwidth dual memory (HBDM) through the local media bus without consuming the AMBA AHB bandwidth. The image stream processor (ISP) of the MLSoC can handle common window-based operations for image processing, and the feature stream processor (FSP) can deal with machine learning algorithms with different dimensions. The power efficiency of the proposed MLSoC is 1.7 TOPS/W, and the area efficiency is 81.3 GOPS/mm 2.

Tera-scale performance machine learning SoC with dual stream processor architecture for multimedia content analysis

A new SoC architecture for multimedia content analysis is implemented with 16mm2 area in 90nm CMOS technology. It focuses on the co-acceleration of computer vision and machine learning algorithms, and two stream processors with massively parallel processing elements are integrated to achieve tera-scale performance. In the dual processor architecture, the data are transferred between processors and the high bandwidth dual memory through the local media bus, which reduces the power consumption in the AHB data access. The power efficiency of the proposed machine learning SoC is 1.7 TOPS/W, and the area efficiency is 81.3 GOPS/mm2.

Highly Efficient Face Detection in Color Images

A highly efficient algorithm for image-based face detection is proposed in this paper, whose features include a fast and regular search method, a local histogram equalization of face candidates, and a frontal face classifier. Experimental results show that the proposed algorithm works efficiently with different types of images. Besides, the average detection rate is 92%, and the required execution time is about 90ms/frame.

Coding Mode Analysis of MPEG-2 to H.264/AVC Transcoding for Digital TV Applications

MPEG-2 to H.264/AVC transcoding is an important module for video recoding in digital TV applications. For pixel domain transcoding, MPEG-2 bitstream is decoded and then re-encoded by H.264 encoder. Since the behavior of the decoded video is different from the original video, in this paper, the performance of each coding mode is analyzed to select the effective coding tools. It is shown from the analysis that transcoding with motion estimation with only one reference frame and 16times16 block size and deblocking filter can achieve almost the same video quality with only 19% of the computation. The analysis result could be an important reference for the implementation of MPEG-2 to H.264 transcoder.

Adaptive AC line current modulation for active power conditioners

The objective of this paper is to propose an adaptive ac line current modulation (ACM) for the active power conditioner (APC). The proposed ACM is able to determine the appropriate input current to stabilize the dc-bus voltage as soon as an abrupt power change of the APC occurred. By adopting the proposed ACM, the dc-bus voltage fluctuation can be minimized so that the required dc-bus capacitance can be reduced. On the other hand, in order to regulate the voltage variation under different power changing scenarios, the regulating slope of the dc-bus voltage under different operation conditions are discussed. Mathematical equations for the proposed ACM are also derived thoroughly in this paper. Finally, experimental measurements obtained from a 5kVA prototype circuit are presented to verify the performances of the proposed adaptive ac line current modulation.

Automatic high dynamic range hallucination in inverse tone mapping

Nowadays the dynamic range of displays has been higher and higher, which means that contents can be recorded and displayed with more detail. However, the original low dynamic range contents were recorded in a lower dynamic range. Such contents will be unsatisfying compared to high dynamic range contents, especially in the saturated, or overexposed region. This paper proposes an algorithm to compensate such exposed regions, which is called automatic high dynamic range image hallucination for inverse tone mapping. Inverse tone-mapping is the process of creating a high dynamic range image from a single low dynamic range image. In this work, high dynamic range image hallucination is used as the key method to reproduce the information which is lost in the low dynamic range image capturing. Previous methods require user interaction as a hallucination criteria, and is not practical in some applications where user interaction is not available. In this paper, the hallucination is performed automatically with the assistance of luminance and texture decoupling process. This scheme produces visually satisfying results and has the potential to be applied to video inverse tone-mapping with its automatic property.

High performance silicon intellectual property for K-Nearest Neighbor algorithm

K-Nearest Neighbor (K-NN) is a classification algorithm that is widely applied in pattern recognition and machine learning. Due to real-time requirements of multimedia content analysis in embedded systems for consumer electronics, it is necessary to accelerate K-NN algorithm by hardware implementations. A high performance silicon intellectual property for K-NN is proposed in this paper. The features include the distance calculator supporting both Euclidean distance and Manhattan distance, and a set of ranking processing elements with high computational efficiency. Experiments show that the proposed hardware has the maximum clock frequency 400MHz with TSMC 90nm technology.