Yu Guan

On Reducing the Effect of Covariate Factors in Gait Recognition: A Classifier Ensemble Method

Robust human gait recognition is challenging because of the presence of covariate factors such as carrying condition, clothing, walking surface, etc. In this paper, we model the effect of covariates as an unknown partial feature corruption problem. Since the locations of corruptions may differ for different query gaits, relevant features may become irrelevant when walking condition changes. In this case, it is difficult to train one fixed classifier that is robust to a large number of different covariates. To tackle this problem, we propose a classifier ensemble method based on the random subspace Method (RSM) and majority voting (MV). Its theoretical basis suggests it is insensitive to locations of corrupted features, and thus can generalize well to a large number of covariates. We also extend this method by proposing two strategies, i.e, local enhancing (LE) and hybrid decision-level fusion (HDF) to suppress the ratio of false votes to true votes (before MV). The performance of our approach is competitive against the most challenging covariates like clothing, walking surface, and elapsed time. We evaluate our method on the USF dataset and OU-ISIR-B dataset, and it has much higher performance than other state-of-the-art algorithms.

A robust speed-invariant gait recognition system for walker and runner identification

In real-world scenarios, walking/running speed is one of the most common covariate factors that can affect the performance of gait recognition systems. By assuming the effect caused by the speed changes (from the query walker-s/runners) are intra-class variations that the training data (i.e., gallery) fails to capture, overfitting to the less representative training data may be the main problem that degrades the performance. In this work, we employ a general model based on random subspace method to solve this problem. More specifically, for query gaits in unknown speeds, we try to reduce the generalization errors by combining a large number of weak classifiers. We evaluate our method on two benchmark databases, i.e., Infrared CASIA-C dataset and Treadmill OU-ISIR-A dataset. For the cross-speed walking/running gait recognition experiments, nearly perfect results are achieved, significantly higher than other state-of-the-art algorithms. We also study the unknown- speed nrunner identification solely using the walking gait gallery, and the encouraging experimental results suggest the effectiveness of our method in such cross-mode gait recognition tasks.

Robust Clothing-Invariant Gait Recognition

Robust gait recognition is a challenging problem, due to the large intra-subject variations and small inter-subject variations. Out of the covariate factors like shoe type, carrying condition, elapsed time, it has been demonstrated that clothing is the most challenging covariate factor for appearance-based gait recognition. For example, long coat may cover a significant amount of gait features and make it difficult for individual recognition. In this paper, we proposed a random subspace method (RSM) framework for clothing-invariant gait recognition by combining multiple inductive biases for classification. Even for small size training set, this method can achieve promising performance. Experiments are conducted on the OU-ISIR Treadmill dataset B which includes 32 combinations of clothing types, and the average recognition accuracy is more than 80%, which indicates the effectiveness of our proposed method.

Random Subspace Method for Gait Recognition

Over fitting is a common problem for gait recognition algorithms when gait sequences in gallery for training are acquired under a single walking condition. In this paper, we propose an approach based on the random subspace method (RSM) to address such over learning problems. Initially, two-dimensional Principle Component Analysis (2DPCA) is adopted to obtain the full hypothesis space (i.e., eigen space). Multiple inductive biases (i.e., subspaces) are constructed, each with the corresponding basis vectors randomly chosen from the initial eigen space. This procedure can not only largely avoid over adaptation but also facilitate dimension reduction. The final classification is achieved by the decision committee which follows a majority voting criterion from the labeling results of all the subspaces. Experimental results on the benchmark USF Human ID gait database show that the proposed method is a feasible framework for gait recognition under unknown walking conditions.

Ensembles of Deep LSTM Learners for Activity Recognition using Wearables

Recently, deep learning (DL) methods have been introduced very successfully into human activity recognition (HAR) scenarios in ubiquitous and wearable computing. Especially the prospect of overcoming the need for manual feature design combined with superior classification capabilities render deep neural networks very attractive for real-life HAR applications. Even though DL-based approaches now outperform the state-of-the-art in a number of recognition tasks, still substantial challenges remain. Most prominently, issues with real-life datasets, typically including imbalanced datasets and problematic data quality, still limit the effectiveness of activity recognition using wearables. In this paper we tackle such challenges through Ensembles of deep Long Short Term Memory (LSTM) networks. LSTM networks currently represent the state-of-the-art with superior classification performance on relevant HAR benchmark datasets. We have developed modified training procedures for LSTM networks and combine sets of diverse LSTM learners into classifier collectives. We demonstrate that Ensembles of deep LSTM learners outperform individual LSTM networks and thus push the state-of-the-art in human activity recognition using wearables. Through an extensive experimental evaluation on three standard benchmarks (Opportunity, PAMAP2, Skoda) we demonstrate the excellent recognition capabilities of our approach and its potential for real-life applications of human activity recognition.

Robust gait recognition from extremely low frame-rate videos

In this paper, we propose a gait recognition method for extremely low frame-rate videos. Different from the popular temporal reconstruction-based methods, the proposed method uses the average gait over the whole sequence as input feature template. Assuming the effect caused by extremely low frame-rate or large gait fluctuations are intra-class variations that the gallery data fails to capture, we build a general model based on random subspace method. More specifically, a number of weak classifiers are combined to reduce the generalization errors. We evaluate our method on the OU-ISIR-D dataset with large/small gait fluctuations, and very competitive results are achieved when both the probe and gallery are extremely low frame-rate gait sequences (e.g., 1 fps).

Combining gait and face for tackling the elapsed time challenges

Random Subspace Method (RSM) has been demonstrated as an effective framework for gait recognition. Through combining a large number of weak classifiers, the generalization errors can be greatly reduced. Although RSM-based gait recognition system is robust to a large number of covariate factors, it is, in essence an unimodal biometric system and has the limitations when facing extremely large intra-class variations. One of the major challenges is the elapsed time covariate, which may affect the human walking style in an unpredictable manner. To tackle this challenge, in this paper we propose a multimodal-RSM framework, and side face is used to strengthen the weak classifiers without compromising the generalization power of the whole system. We evaluate our method on the TUM-GAID dataset, and it significantly outperforms other multimodal methods. Specifically, our method achieves very competitive results for tackling the most challenging elapsed time covariate, which potentially also includes the changes in shoe, carrying status, clothing, lighting condition, etc.

PCA-based denoising of Sensor Pattern Noise for source camera identification

Sensor Pattern Noise (SPN) has been proved to be an inherent fingerprint of the imaging device for source identification. However, SPN extracted from digital images can be severely contaminated by scene details. Moreover, SPN with high dimensionality may cause excessive time cost on calculating correlation between SPNs, which will limit its applicability to the source camera identification or image classification with a large dataset. In this work, an effective scheme based on principal component analysis (PCA) is proposed to address these two problems. By transforming SPN into eigenspace spanned by the principal components, the scene details and trivial information can be significantly suppressed. In addition, due to the dimensionality reduction property of PCA, the size of SPN is greatly reduced, consequently reducing the time cost of calculating similarity between SPNs. Our experiments are conducted on the Dresden database, and results demonstrate that the proposed method outperforms could achieve the state-of-art performance in terms of the Receiver Operating Characteristic (ROC) curves while reducing the dimensionality of SPN.

Enhanced SVD for Collaborative Filtering

Matrix factorization is one of the most popular techniques for prediction problems in the fields of intelligent systems and data mining. It has shown its effectiveness in many real-world applications such as recommender systems. As a collaborative filtering method, it gives users recommendations based on their previous preferences or ratings. Due to the extreme sparseness of the ratings matrix, active learning is used for eliciting ratings for a user to get better recommendations. In this paper, we propose a new matrix factorization model called Enhanced SVD ESVD which combines the classic matrix factorization method with a specific rating elicitation strategy. We evaluate the proposed ESVD method on the Movielens data set, and the experimental results suggest its effectiveness in terms of both accuracy and efficiency, when compared with traditional matrix factorization methods and active learning methods.

A compact representation of sensor fingerprint for camera identification and fingerprint matching

Sensor Pattern Noise (SPN) has been proved as an effective fingerprint of imaging devices to link pictures to the cameras that acquired them. In practice, forensic investigators usually extract this camera fingerprint from large image block to improve the matching accuracy because large image blocks tend to contain more SPN information. As a result, camera fingerprints usually have a very high dimensionality. However, the high dimensionality of fingerprint will incur a costly computation in the matching phase, thus hindering many interesting applications which require an efficient real-time camera matching. To solve this problem, an effective feature extraction method based on PCA and LDA is proposed in this work to compress the dimensionality of camera fingerprint. Our experimental results show that the proposed feature extraction algorithm could greatly reduce the size of fingerprint and enhance the performance in term of Receiver Operating Characteristic (ROC) curve of several existing methods.