Liyu Gong

Shape of Gaussians as feature descriptors

This paper introduces a feature descriptor called shape of Gaussian (SOG), which is based on a general feature descriptor design framework called shape of signal probability density function (SOSPDF). SOSPDF takes the shape of a signals probability density function (pdf) as its feature. Under such a view, both histogram and region covariance often used in computer vision are SOSPDF features. Histogram describes SOSPDF by a discrete approximation way. Region covariance describes SOSPDF as an incomplete parameterized multivariate Gaussian distribution. Our proposed SOG descriptor is a full parameterized Gaussian, so it has all the advantages of region covariance and is more effective. Furthermore, we identify that SOGs form a Lie group. Based on Lie group theory, we propose a distance metric for SOG. We test SOG features in tracking problem. Experiments show better tracking results compared with region covariance. Moreover, experiment results indicate that SOG features attempt to harvest more useful information and are less sensitive against noise.

Recognizing affect from non-stylized body motion using shape of Gaussian descriptors

In this paper, we address the problem of recognizing affect from non-stylized human body motion. We utilize a novel feature descriptor which is based on the shape of signal probability density function framework to represent the motion capture data. Combining the feature representation scheme with support vector machine classifier, we detect implicitly communicated affect in human body motion. We test our algorithm using a comprehensive database of affectively performed motion. Experiment results show state-of-the-art performance compared with the existing methods.

An effective head pose estimation approach using Lie Algebrized Gaussians based face representation

The accuracy of head pose estimation is significant for many computer vision applications such as face recognition, driver attention detection and human-computer interaction. Most appearance-based head pose estimation works typically extract the low-dimensional face appearance features in some statistic subspaces, where the subspaces represent the underlying geometry structure of the pose space. However, there is an open problem, namely, how to effectively represent appearance-based subspace face for the head pose estimation problem. To address the problem, this paper proposes a head pose estimation approach based on the Lie Algebrized Gaussians (LAG) feature to model the pose characteristic. LAG is built on Gaussian Mixture Models (GMM), which actually not only models the distribution of local appearance features, but also captures the Lie group manifold structure of the feature space. Moreover, to keep multi-resolution structure information, LAG is operated on many subregions of the image. As a result, these properties of LAG enable it to effectively model the structure of subspace face which can lead to powerful discriminative ability for head pose estimation. After representing subspace face using the LAG, we treat the head pose estimation as a classification problem. The within-class covariance normalization (WCCN) based Support Vector Machine (SVM) classifier is employed to achieve robust performance as WCCN could reduce the within-class variabilities of the same pose. Extensive experimental analysis and comparison with both traditional and state-of-the-art algorithms on two challenging benchmarks demonstrate the effectiveness of our approach.

A Lie group based spatiogram similarity measure

Spatiograms were generalization of histograms, which can harvest spatial information of images. The similarity measure is important when applying spatiograms to various computer vision problems such as tracking and image retrieval. The original proposed measures use Mahalanobis distance of coordinate mean to measure spatial information in spatiograms. However, spatial information which is described by spatiograms does not lie on vector space. Measures for vector space such as Mahalanobis distance are not effective measures for them. In this paper, We model spatial information as Gaussian approximation of coordinate distributions. Then we parameterize them as a Lie group. Based on Lie group theory, we analyze function space structure of Gaussian pdfs (probability density function) and propose an effective spatiogram similarity measure. We test our measure in object tracking scenarios. Experiments show better tracking results compared with previously proposed measures.

Action recognition using lie algebrized gaussians over dense local spatio-temporal features

This paper presents a novel framework for human action recognition based on a newly proposed mid-level feature representation method named Lie Algebrized Guassians (LAG). As an action sequence can be treated as a 3D object in space-time space, we address the action recognition problem by recognizing 3D objects and characterize 3D objects by the probability distributions of local spatio-temporal features. First, for each video, we densely sample local spatio-temporal features (e.g. HOG3D) at multiple scales confined in bounding boxes of human body. Moreover, normalized spatial coordinates are appended to local descriptor in order to capture spatial position information. Then the distribution of local features in each video is modeled by a Gaussian Mixture Model (GMM). To estimate the parameters of video-specific GMMs, a global GMM is trained using all training data and video-specific GMMs are adapted from the global GMM. Then the LAG is adopted to vectorize those video-specific GMMs. Finally, linear SVM is employed for classification. Experimental results on the KTH and UCF Sports dataset show that our method achieves state-of-the-art performance.

A Lie group based Gaussian Mixture Model distance measure for multimedia comparison

In this paper, we propose a novel method to measure the distance between two Gaussian Mixture Models. The proposed distance measure is based on the minimum cost that must paid to transform from one Gaussian Mixture Model into the other. We parameterize the components of a Gaussian Mixture Model which are Gaussian probability density functions (pdf) as positive definite lower triangular transformation matrices. Then we identify that Gaussian pdfs form a Lie group. Based on Lie group theory, the geodesic length can be used to measure the minimum cost that must paid to transform from one Gaussian pdf into the other. Combining geodesic length with the earth movers distance, we propose the Lie group earth movers distance for Gaussian Mixture Models. We test our distance measure in image retrieval. The experimental results indicate that our distance measure is more effective than other measures including the Kullback-Liebler divergence.

Effective human age estimation using a two-stage approach based on Lie Algebrized Gaussians feature

Automatically and effectively estimating human ages via facial images has lots of practical applications, such as security surveillance, electronic customer relationship management and entertainment. Motivated by the fact that feature representation and recognition are two key problems in facial image based human age estimation, in this paper, we propose to employ a novel discriminative feature called Lie Algebrized Gaussians (LAG) for the representation of age images and design a two-stage approach for learning and predicting human ages. LAG is built on Gaussian Mixture Models (GMM) and is able to capture the aging manifold of the age image by preserving the Lie group manifold structure information embedded in the feature space. Given the LAG feature for each image, we estimate the human age using a two-stage approach in a coarse-to-fine fashion. In the first stage, an adaptive age group for each input image is obtained by selecting a number of neighboring age labels around the output of a global regressor. In the second stage, a local classifier is learned from the selected age classes to determine the final age of the input image. The effectiveness of our approach is evaluated on both FG-NET and MORPH benchmarks, extensive experimental results and comparisons with the state-of-the-art algorithms demonstrate the superiority of our approach for the human age estimation task.

Modeling spatio-temporal layout with Lie Algebrized Gaussians for action recognition

We propose a novel approach to model spatio-temporal distribution of local features for action recognition in videos. The proposed approach is based on the Lie Algebrized Gaussians (LAG) which is a feature aggregation approach and yields high-dimensional video signature. In the framework of LAG, local features extracted from a video are aggregated to train a video-specific Gaussian Mixture Model (GMM). Then the video-specific GMM is encoded as a vector based on Lie group theory and this step is also referred to as GMM vectorization. As the video-specific GMM gives a soft partition of the feature space, for each cell of the feature space (i.e. each Gaussian component), we use a GMM to model the spatio-temporal locations of the local features assigned to the Gaussian component. The location GMMs are encoded as vectors just like the local feature GMM. We term those vectors of location GMMs spatio-temporal LAG (STLAG). In addition, although the LAG and the popular Fisher Vector (FV) are derived from distinct theory perspectives, we find that they are closely related. Hence the power and ℓ2 normalization proposed for the FV are also beneficial to the LAG. Experimental results show that STLAG is very effective to model spatio-temporal layout compared with other techniques such as spatio-temporal pyramid and feature augmentation. Using the state-of-the-art dense trajectory features, our approach achieves state-of-the-art performance on two challenging datasets: Hollywood2 and HMDB51.

Detecting Human in Still Images by Learning Multi-Scale Mid-Level Features

Detecting human in still images is one of the most challenging object detection problems. In this paper we apply the scale theory to human detection. By integrating Gaussian Pyramids multi-scale object representation approach we present a Learned Multi-scale Mid-level Feature (LMMF) based human detection algorithm. Firstly multiscale low-level features are extracted by Gaussian Pyramid decomposition and gradient computation. Then LMMFs are learned from multi-scale low-level features using AdaBoost algorithm. The final human/non-human decision is made by classification on the LMMFs. Using LMMF descriptors, our method attempts to harvest more information than using uni-scale feature descriptors. Experiments on INRIA person dataset demonstrate that our method outperforms the previous state of the art detector.