Chen-Kuo Chiang

Fast JND-Based Video Carving With GPU Acceleration for Real-Time Video Retargeting

A recently developed image resizing technique, seam carving, has been proved to be a useful tool for content-adaptive spatially nonuniform image resizing with the purpose of optimal display on a screen of reduced resolution or different aspect ratio. In this paper, we present a fast algorithm for real-time content-aware video retargeting based on the improved seam carving method proposed in this paper. The proposed algorithm is designed to be highly parallelizable and suitable for running on a multicore architecture. First, two novel operators, i.e., seam update and seam split, are introduced to analyze an image for detecting the local and global seams with minimum costs very efficiently. With these operators, parallel processing can be achieved to determine multiple seams simultaneously. In addition, the saliency measure is extended with a just-noticeable-distortion model which makes the resized video more consistent with human perception. We demonstrate the efficiency of the above new components with a graphics processing unit (GPU) implementation. In addition, the proposed fast seam carving algorithm is extended for video retargeting. To the best of our knowledge, this is the first paper based on the seam carving method to achieve real-time video retargeting on a GPU. Experimental results on video sequences of various characteristics are demonstrated to show the superior performance of the proposed algorithm in comparison with the existing content-adaptive image/video resizing methods.

Fast H.264 Encoding Based on Statistical Learning

H.264/AVC, the latest video coding standard of the Joint Video Team, greatly outperforms previous standards in terms of coding bitrate and video quality, because it adopts several new techniques. However, the computational complexity is also considerably increased due to these new components. In this paper, we propose fast algorithms based on statistical learning to reduce the computational cost involved in three main components in H.264 encoder, i.e., intermode decision, multi-reference motion estimation (ME), and intra-mode prediction. First, representative features are extracted to build the learning models. Then, an offline pre-classification approach is used to determine the best results from the extracted features, thus a significant amount of computation is reduced based on the classification strategy. The proposed statistical learning-based approach is applied to the aforementioned three main components in H.264 encoder to speed up the computation. Experimental results show that the ME time of the proposed system is significantly sped up with 12 times faster than the conventional fast ME algorithm of H.264, and the total encoding time of the proposed encoder is greatly reduced with about four times faster than the fast encoder EPZS in the H.264 reference code with negligible video quality degradation.

Face Verification With Local Sparse Representation

In this letter, a local sparse representation is proposed for face components to describe the local structure and characteristics of the face image for face verification. We first learn a dictionary from collected local patches of face images. Then, a novel local descriptor is presented by using sparse coefficients obtained by the learned dictionary and local face patches from face components to represent the entire human face. We demonstrate the performance of the proposed local sparse representation method on several publicly available datasets. Extensive experiments on both CMU PIE dataset and the challenging LFW database have shown the effectiveness of the proposed method.

Learning component-level sparse representation using histogram information for image classification

A novel component-level dictionary learning framework which exploits image group characteristics within sparse coding is introduced in this work. Unlike previous methods, which select the dictionaries that best reconstruct the data, we present an energy minimization formulation that jointly optimizes the learning of both sparse dictionary and component level importance within one unified framework to give a discriminative representation for image groups. The importance measures how well each feature component represents the image group property with the dictionary by using histogram information. Then, dictionaries are updated iteratively to reduce the influence of unimportant components, thus refining the sparse representation for each image group. In the end, by keeping the top K important components, a compact representation is derived for the sparse coding dictionary. Experimental results on several public datasets are shown to demonstrate the superior performance of the proposed algorithm compared to the-state-of-the-art methods.

Spatio-Temporally Consistent View Synthesis From Video-Plus-Depth Data With Global Optimization

We propose a novel algorithm to generate a virtual-view video from a video-plus-depth sequence. The proposed method enforces the spatial and temporal consistency in the disocclusion regions by formulating the problem as an energy minimization problem in a Markov random field (MRF) framework. At the system level, we first recover the depth images and the motion vector maps after the image warping with the preprocessed depth map. Then, we formulate the energy function for the MRF with additional shift variables for each node. To reduce the high computational complexity of applying belief propagation (BP) to this problem, we present a multilevel BPs by using BP with smaller numbers of label candidates for each level. Finally, the Poisson image reconstruction is applied to improve the color consistency along the boundary of the disocclusion region in the synthesized image. Experimental results demonstrate the performance of the proposed method on several publicly available datasets.

Learning Component-Level Sparse Representation for Image and Video Categorization

A novel component-level dictionary learning framework that exploits image/video group characteristics based on sparse representation is introduced in this paper. Unlike the previous methods that select the dictionaries to best reconstruct the data, we present an energy minimization formulation that jointly optimizes the learning of both sparse dictionary and component-level importance within one unified framework to provide a discriminative and sparse representation for image/video groups. The importance measures how well each feature component represents the group property with the dictionary. Then, the dictionary is updated iteratively to reduce the influence of unimportant components, thus refining the sparse representation for each group. In the end, by keeping the top K important components, a compact representation is obtained for the sparse coding dictionary. Experimental results on several public image and video data sets are shown to demonstrate the superior performance of the proposed algorithm compared with the-state-of-the-art methods.

Fast Multi-Reference Frame Motion Estimation via Downhill Simplex Search

Multi-reference frame motion estimation improves the accuracy of motion compensation in video compression, but it also dramatically increases computational complexity. Based on tracing motion vector trajectories, fast approximated motion estimation results can be obtained for multi-reference frames. In this paper, we extend the downhill simplex search to multiple reference frames and propose several enhanced schemes to improve its efficiency and accuracy. Experimental results show that the proposed algorithm outperforms several representative single-reference frame block matching methods

Multi-attribute sparse representation with group constraints for face recognition under different variations

A novel multi-attribute sparse representation enforced with group constraints is proposed in this paper. Data with multiple attributes can be represented by individual binary matrices to indicate the group properties for each data sample. Then, these attribute matrices are incorporated into the formulation of l1-minimization. The solution is obtained by jointly considering the data reconstruction error, the sparsity property as well as the group constraints, thus making the basis selection in sparse coding more efficient in term of accuracy. The proposed optimization formulation with group constraints is simple yet very efficient for classification problems with multiple attributes. In addition, it can be derived into a modified sparse coding form so that any l1-minimization solver can be employed in the corresponding optimization problem. We demonstrate the performance of the proposed multi-attribute sparse representation algorithm through experiments on face recognition with different kinds of variations. Experimental results show that the proposed method is very competitive compared to the state-of-the-art methods.

Multi-attributed Dictionary Learning for Sparse Coding

We present a multi-attributed dictionary learning algorithm for sparse coding. Considering training samples with multiple attributes, a new distance matrix is proposed by jointly incorporating data and attribute similarities. Then, an objective function is presented to learn category-dependent dictionaries that are compact (closeness of dictionary atoms based on data distance and attribute similarity), reconstructive (low reconstruction error with correct dictionary) and label-consistent (encouraging the labels of dictionary atoms to be similar). We have demonstrated our algorithm on action classification and face recognition tasks on several publicly available datasets. Experimental results with improved performance over previous dictionary learning methods are shown to validate the effectiveness of the proposed algorithm.

Single-shot person re-identification based on improved Random-Walk pedestrian segmentation

Single-shot person re-identification is to match pedestrian images captured from different cameras at different time under the condition of large illumination variations, different viewpoints, and inadequate information of single-shot case. To deal with these challenges, we propose a four-step single-shot person re-identification algorithm that consists of pedestrian segmentation, human region partitioning, feature extraction and human feature matching. Based on an improved Random Walks algorithm, human foreground is segmented by combining the shape prior information and the color seed constraint into the Random Walk formulation. Then color features of HSV histogram and 1-D RGB signal along with texture features from human body parts are used for the person re-identification. The correct match is then determined by the similarity scores of all features with appropriate weight selection. The experimental results demonstrate the superior performance by using the proposed algorithm compared to the previous representative methods.