Yu-Chi Su

A 52 mW Full HD 160-Degree Object Viewpoint Recognition SoC With Visual Vocabulary Processor for Wearable Vision Applications

A 1920 × 1080 160° object viewpoint recognition system-on-chip (SoC) is presented in this paper. The SoC design is dedicated to wearable vision applications, and we address several crucial issues including the low recognition accuracy due to the use of low resolution images and dramatic changes in object viewpoints, and the high power consumption caused by the complex computations in existing computer vision object recognition systems. The human-centered design (HCD) mechanism is proposed in order to maintain a high recognition rate in difficult situations. To overcome the degradation of accuracy when dramatic changes to the object viewpoint occur, the object viewpoint prediction (OVP) engine in the HCD provides 160° object viewpoint in- variance by synthesizing various object poses from predicted object viewpoints. To achieve low power consumption, the visual vocabulary processor (VVP), which is based on bag-of-words (BoW) matching algorithm, is used to advance the matching stage from the feature-level to the object-level and results in a 97% reduction in the required memory bandwidth compared to previous recognition systems. Moreover, the matching efficiency of the VVP enables the system to support real-time full HD (1920 × 1080) processing, thereby improving the recognition rate for detecting a traffic light at a distance of 50 m to 95% compared to the 29% recognition rate for VGA (640 × 480) processing. The real-time 1920 × 1080 visual recognition chip is realized on a 6.38 mm2 die with 65 nm CMOS technology. It achieves an average recognition rate of 94%, a power efficiency of 1.18 TOPS/W, and an area efficiency of 25.9 GOPS/mm2 while only dissipating 52 mW at 1.0 V.

An intelligent depth-based obstacle detection system for visually-impaired aid applications

In this paper, we present a robust depth-based obstacle detection system in computer vision. The system aims to assist the visually-impaired in detecting obstacles with distance information for safety. With analysis of the depth map, segmentation and noise elimination are adopted to distinguish different objects according to the related depth information. Obstacle extraction mechanism is proposed to capture obstacles by various object proprieties revealing in the depth map. The proposed system can also be applied to emerging vision-based mobile applications, such as robots, intelligent vehicle navigation, and dynamic surveillance systems. Experimental results demonstrate the proposed system achieves high accuracy. In the indoor environment, the average detection rate is above 96.1%. Even in the outdoor environment or in complete darkness, 93.7% detection rate is achieved on average.

Visual Vocabulary Processor Based on Binary Tree Architecture for Real-Time Object Recognition in Full-HD Resolution

Feature matching is an indispensable process for object recognition, which is an important issue for wearable devices with video analysis functionalities. To implement a low-power SoC for object recognition, the proposed visual vocabulary processor (VVP) is employed to accelerate the speed of feature matching. The VVP can transform hundreds of 128-D SIFT vectors into a 64-D histogram for object matching by using the binary-tree-based architecture, and 16 calculators for the computations of the Euclidean distances are designed for each of the two processors in each level. A total of 126 visual words can be saved in the six-level hierarchical memory, which instantly offers the data required for the matching process, and more than 5 times of bandwidth can be saved compared with the non-binary-tree-based architecture. As a part of the recognition SoC, the VVP is implemented with the 65-nm CMOS technology, and the experimental results show that the gate count and the average power consumption are 280 K and 5.6 mW, respectively.

Connected vehicle safety science, system, and framework

In this paper, we propose a framework to develop an M2M-based (machine-to-machine) proactive driver assistance system. Unlike traditional approaches, we take the benefits of M2M in intelligent transportation system (ITS): 1) expansion of sensor coverage, 2) increase of time allowed to react, and 3) mediation of bidding for right of way, to help driver avoiding potential traffic accidents. To develop such a system, we divide it into three main parts: 1) driver behavior modeling and prediction, which collects grand driving data to learn and predict the future behaviors of drivers; 2) M2M-based neighbor map building, which includes sensing, communication, and fusion technologies to build a neighbor map, where neighbor map mentions the locations of all neighboring vehicles; 3) design of passive information visualization and proactive warning mechanism, which researches on how to provide user-needed information and warning signals to drivers without interfering their driving activities.

An intelligent depth-based obstacle detection for mobile applications

In this paper, we present a depth-based obstacle detection system in computer vision. Segmentation and noise elimination are adopted to capture different obvious objects. High performance is achieved via obstacle extraction. Our framework can be used to help blind people prevent from dangers and applied to other dynamic surveillance applications. The detection rate is above 93.7% outdoors and indoors.

Mapping Scalable Video Coding decoder on multi-core stream processors

Scalable Video Coding (SVC) is an advanced video compression technique that can support temporal, spatial, and quality scalability to terminals with different network conditions. SVC adopts layered coding techniques to improve coding efficiency for spatial and quality scalability. Upsampling and inter-layer prediction are two important mechanisms to remove redundant information between different layers. However, upsampling occupying around 75% memory bandwidth of SVC decoder results in serious performance degradation, especially for applications with high resolutions. Moreover, inter-layer prediction with complex scheduling leads to difficulties when mapping the SVC decoder in parallel. In this paper, we propose a method to parallelize the SVC decoder on a multi-core stream processor platform in both efficiency and flexibility. We focus on mapping issues of spatial scalability supporting with various resolutions of decoded frames. The experiment result proves the proposed design for SVC decoder reduces 95% memory bandwidth of the upsampling module in JSVM, performed on a single general-purpose processor.

Robust moving object tracking and trajectory prediction for visual navigation in dynamic environments

We propose a robust and efficient moving object detection system with trajectory prediction mechanism for mobile visual applications in dynamic environments. The navigation system not only supports intelligent moving object detection and collision avoidance, but also provides automatic trajectory prediction mechanism to create a safe environment for the user. To take into account recognition challenges in practice, the system addresses problems of camera shake and image blurs caused by mobile camera and fast moving objects, respectively. Experimental results show our system supports 97.1% detection rate for fast moving objects and achieves high accuracy of trajectory prediction with error estimation less than 16.5 cm.

Accurate positioning system based on street view recognition

In this paper, an accurate and robust positioning system based on street view recognition is introduced. Vision-based technique is employed for dynamically recognizing shop or building signs on the GPS map. Two mechanisms including view-angle invariant distance estimation and path refinement are proposed for robust and accurate position estimation. Through the combination of visual recognition technique and GPS scale data, the real user location can be accurately inferred. Experimental results demonstrate that the proposed system is reliable and feasible. Compared with 20m error of position estimation provided by the GPS, our system only has 0.97m error estimation.

Intelligent document capturing and blending system based on robust feature matching with an active camera

We propose an intelligent document capturing and blending system based on robust feature matching for efficient document management. The proposed system not only supports handwritten text and figure extraction, but also provides image blending mechanism to automatically merge the extracted handwritten texts and figures into electronic documents for the user. The proposed system addresses camera shake and luminance variation problems caused by active cameras. Besides, we adopt robust feature matching techniques to improve the system accuracy. Experimental results show that our system supports 95.65% detection rate and achieves 88.3% compression ratio reduction compared with the previous work. Besides, we also compare system performance considering Scale Invariant Feature Transform (SIFT) [1] and Speeded-Up Robust Features (SURF) [2]. We derive 71.2% complexity reduction and 4.3% detection rate degradation with SURF feature matching.

A novel hybrid pipeline design methodology on a multi-cores streaming system for multimedia applications

With the rapid growth of media-processing technologies and the advancement of semiconductor process, more and more multimedia applications are integrated into consumer electronics. However, in such highly complex system, the design time for the circuit designers does not reduce much as the process advances. We propose a novel hybrid pipeline design methodology for multi-core streaming system from C-level to RTL design. Our methodology can optimize data communication for media-processing applications to achieve both flexibility and efficiency.