Wei-Lun Chao

Facial age estimation based on label-sensitive learning and age-oriented regression

This paper provides a new age estimation approach, which distinguishes itself with the following three contributions. First, we combine distance metric learning and dimensionality reduction to better explore the connections between facial features and age labels. Second, to exploit the intrinsic ordinal relationship among human ages and overcome the potential data imbalance problem, a label-sensitive concept and several imbalance treatments are introduced in the system training phase. Finally, an age-oriented local regression is presented to capture the complicated facial aging process for age determination. The simulation results show that our approach achieves the lowest estimation error against existing methods.

Video Summarization with Long Short-Term Memory

We propose a novel supervised learning technique for summarizing videos by automatically selecting keyframes or key subshots. Casting the problem as a structured prediction problem on sequential data, our main idea is to use Long Short-Term Memory (LSTM), a special type of recurrent neural networks to model the variable-range dependencies entailed in the task of video summarization. Our learning models attain the state-of-the-art results on two benchmark video datasets. Detailed analysis justifies the design of the models. In particular, we show that it is crucial to take into consideration the sequential structures in videos and model them. Besides advances in modeling techniques, we introduce techniques to address the need of a large number of annotated data for training complex learning models. There, our main idea is to exploit the existence of auxiliary annotated video datasets, albeit heterogeneous in visual styles and contents. Specifically, we show domain adaptation techniques can improve summarization by reducing the discrepancies in statistical properties across those datasets.

An Empirical Study and Analysis of Generalized Zero-Shot Learning for Object Recognition in the Wild

We investigate the problem of generalized zero-shot learning (GZSL). GZSL relaxes the unrealistic assumption in conventional zero-shot learning (ZSL) that test data belong only to unseen novel classes. In GZSL, test data might also come from seen classes and the labeling space is the union of both types of classes. We show empirically that a straightforward application of classifiers provided by existing ZSL approaches does not perform well in the setting of GZSL. Motivated by this, we propose a surprisingly simple but effective method to adapt ZSL approaches for GZSL. The main idea is to introduce a calibration factor to calibrate the classifiers for both seen and unseen classes so as to balance two conflicting forces: recognizing data from seen classes and those from unseen ones. We develop a new performance metric called the Area Under Seen-Unseen accuracy Curve to characterize this trade-off. We demonstrate the utility of this metric by analyzing existing ZSL approaches applied to the generalized setting. Extensive empirical studies reveal strengths and weaknesses of those approaches on three well-studied benchmark datasets, including the large-scale ImageNet with more than 20,000 unseen categories. We complement our comparative studies in learning methods by further establishing an upper bound on the performance limit of GZSL. In particular, our idea is to use class-representative visual features as the idealized semantic embeddings. We show that there is a large gap between the performance of existing approaches and the performance limit, suggesting that improving the quality of class semantic embeddings is vital to improving ZSL.

Summary Transfer: Exemplar-Based Subset Selection for Video Summarization

Video summarization has unprecedented importance to help us digest, browse, and search todays ever-growing video collections. We propose a novel subset selection technique that leverages supervision in the form of humancreated summaries to perform automatic keyframe-based video summarization. The main idea is to nonparametrically transfer summary structures from annotated videos to unseen test videos. We show how to extend our method to exploit semantic side information about the videos category/ genre to guide the transfer process by those training videos semantically consistent with the test input. We also show how to generalize our method to subshot-based summarization, which not only reduces computational costs but also provides more flexible ways of defining visual similarity across subshots spanning several frames. We conduct extensive evaluation on several benchmarks and demonstrate promising results, outperforming existing methods in several settings.

Facial expression recognition based on improved local binary pattern and class-regularized locality preserving projection

Abstract This paper provides a novel method for facial expression recognition, which distinguishes itself with the following two main contributions. First, an improved facial feature, called the expression-specific local binary pattern (es-LBP), is presented by emphasizing the partial information of human faces on particular fiducial points. Second, to enhance the connection between facial features and expression classes, class-regularized locality preserving projection (cr-LPP) is proposed, which aims at maximizing the class independence and simultaneously preserving the local feature similarity via dimensionality reduction. Simulation results show that the proposed approach is very effective for facial expression recognition.

Coarse-to-fine temporal optimization for video retargeting based on seam carving

In this paper, a new video retargeting method based on temporal information and seam carving is presented. Two video energy functions, motion weight prediction and pixel-based optimization, are proposed to take the temporal information into account and make dynamic programming available during the process of retargeting. The motion weight prediction exploits both the block-based motion estimation and Gaussian masks to predict the coarse location of seams in the current frame and reduce the search range of dynamic programming. The pixel-based optimization then utilizes the concept of pixel-based optical flow to explore better temporal relations between the current frame and previous frames in the reduced search range. The experimental results show that combining these two video energy functions as well as dynamic programming, the proposed method could achieve content-aware and temporal smoothing retargeting results with less computational complexity.

Facial age estimation based on label-sensitive learning and age-specific local regression

In this paper, a new age estimation framework considering the intrinsic properties of human ages is proposed, which improves the dimensionality reduction techniques to learn the connections between facial features and aging labels. To enhance the performance of dimensionality reduction, a distance metric adjustment step is introduced in advance to achieve a suitable metric in the feature space. In addition, to further exploit the ordinal relationship of human ages, the “label-sensitive” concept is proposed, which regards the label similarity during the learning phase of distance metric and dimensionality reduction. Finally, an age-specific local regression algorithm is proposed to capture the complicated aging process for age determination. From the simulation results, the proposed framework achieves the lowest mean absolute error against the existing methods.

Color constancy by chromaticity neutralization

In this paper, a robust illuminant estimation algorithm for color constancy is proposed. Considering the drawback of the well-known max-RGB algorithm, which regards only pixels with the maximum image intensities, we explore the representative pixels from an image for illuminant estimation: The representative pixels are determined via the intensity bounds corresponding to a certain percentage value in the normalized accumulative histograms. To achieve the suitable percentage, an iterative algorithm is presented by simultaneously neutralizing the chromaticity distribution and preventing overcorrection. The experimental results on the benchmark databases provided by Simon Fraser University and Microsoft Research Cambridge, as well as several web images, demonstrate the effectiveness of our approach.

Muscle injury determination by image segmentation

Clinical examination of Congenital Muscular Torticollis (CMT) is often carried out by ultrasound equipments. However, a variety of subjective factors during diagnosis may result in wrong decision. Thus, we propose an image processing algorithm to derive the objective judgment on the healthiness of muscle in this paper. We first apply image segmentation technique, such as the fast scanning algorithm, for ultrasonic muscle image segmentation. Then, the proposed algorithms are applied to determine the healthiness of muscle fibers. We furthermore propose a score criterion to evaluate the degree of injury. The experimental results show that the injury score measured by the proposed methods can successfully determine whether the muscle is hurt and infer the extent of fibrosis.

Asymmetric fourier descriptor of non-closed segments

The Fourier descriptor is an efficient and effective way to describe a closed boundary. However, for a non-closed segment, since the two non-adjacent end points result in signal discontinuity, after eliminating the high-frequency part, the reconstructed segment has large error near the two ends. In this paper, we propose a warping method to connect the two ends and perform odd-symmetric extension to smooth the warped segment around them. With these modifications, the high-frequency components near the two ends can be much reduced and we can obtain the reconstructed segment with accurate end-point locations even when only the low frequency coefficients are preserved. This method could also be used for a closed boundary with a pre-segmentation process, and the experimental result shows that with the same boundary compression rate, our method has better reconstruction quality than directly extracting Fourier descriptors on the closed boundary.