Sherif Azary

Grassmannian Sparse Representations and Motion Depth Surfaces for 3D Action Recognition

Manifold learning has been effectively used in computer vision applications for dimensionality reduction that improves classification performance and reduces computational load. Grassmann manifolds are well suited for computer vision problems because they promote smooth surfaces where points are represented as subspaces. In this paper we propose Grassmannian Sparse Representations (GSR), a novel subspace learning algorithm that combines the benefits of Grassmann manifolds with sparse representations using least squares loss L1-norm minimization for optimal classification. We further introduce a new descriptor that we term Motion Depth Surface (MDS) and compare its classification performance against the traditional Motion History Image (MHI) descriptor. We demonstrate the effectiveness of GSR on computationally intensive 3D action sequences from the Microsoft Research 3D-Action and 3D-Gesture datasets.

3D Action Classification Using Sparse Spatio-temporal Feature Representations

Automatic action classification is a challenging task for a wide variety of reasons including unconstrained human motion, background clutter, and view dependencies. The introduction of affordable depth sensors allows opportunities to investigate new approaches for action classification that take advantage of depth information. In this paper, we perform action classification using sparse representations on 3D video sequences of spatio-temporal kinematic joint descriptors and compare the classification accuracy against spatio-temporal raw depth data descriptors. These descriptors are used to create over-complete dictionaries which are used to classify test actions using least squares loss L1-norm minimization with a regularization parameter. We find that the representations of raw depth features are naturally more sparse than kinematic joint features and that our approach is highly effective and efficient at classifying a wide variety of actions from the Microsoft Research 3D Dataset (MSR3D).

A spatiotemporal descriptor based on radial distances and 3D joint tracking for action classification

Action recognition is an important research area that is particularly challenging when dealing with view independent and unconstrained human motion. While progress has been made in developing pose-dependent action classification systems, the introduction of affordable 3D sensors has opened up opportunities for action classification with depth data. In this paper, we propose an efficient 3D descriptor combining radial distance measures on 2D video sequences with 3D joint tracking on depth data for action classification through Manifold Learning using supervised Locality Preserving Projections (sLPP). We find that the application of radial distances on depth data is effective at classifying actions and when combined with 3D joint tracking the action classification performance improves. We applied our method on the Microsoft Research 3D Dataset (MSR3D) and obtained good classification accuracy on all 20 unique 3D actions. Activity recognition rates were as high as 98.95% on subsets of 3D actions.

View invariant activity recognition with manifold learning

Activity recognition in complex scenes can be very challenging because human actions are unconstrained and may be observed from multiple views. While progress has been made in recognizing activities from fixed views, more research is needed in developing view invariant recognition methods. Furthermore, the recognition and classification of activities involves processing data in the space and time domains, which involves large amounts of data and can be computationally expensive to process. To accommodate for view invariance and high dimensional data we propose the use of Manifold Learning using Locality Preserving Projections (LPP). We develop an efficient set of features based on radial distance and present a Manifold Learning framework for learning low dimensional representations of action primitives that can be used to recognize activities at multiple views. Using our approach we present high recognition rates on the Inria IXMAS dataset.

Grassmannian sparse representations

Abstract. We present Grassmannian sparse representations (GSR), a sparse representation Grassmann learning framework for efficient classification. Sparse representation classification offers a powerful approach for recognition in a variety of contexts. However, a major drawback of sparse representation methods is their computational performance and memory utilization for high-dimensional data. A Grassmann manifold is a space that promotes smooth surfaces where points represent subspaces and the relationship between points is defined by the mapping of an orthogonal matrix. Grassmann manifolds are well suited for computer vision problems because they promote high between-class discrimination and within-class clustering, while offering computational advantages by mapping each subspace onto a single point. The GSR framework combines Grassmannian kernels and sparse representations, including regularized least squares and least angle regression, to improve high accuracy recognition while overcoming the drawbacks of performance and dependencies on high dimensional data distributions. The effectiveness of GSR is demonstrated on computationally intensive multiview action sequences, three-dimensional action sequences, and face recognition datasets.

Grassmannian Spectral Regression for Action Recognition

Action recognition from multiple views and computational performance associated with high-dimensional data are common challenges for real-world action classification systems. Subspace learning has received considerable attention as a means of finding an efficient low-dimensional representation that leads to better classification and efficient processing. In this paper we propose Grassmannian Spectral Regression (GRASP), a novel subspace learning algorithm which combines the benefits of Grassmann manifolds and spectral regression for fast and accurate classification. A Grassmann manifold is a space that promotes smooth surfaces where points represent subspaces and the relationship between points is defined by a mapping of an orthogonal matrix. Spectral regression is a regularized subspace learning approach that overcomes the disadvantages of eigen-based approaches. We demonstrate the effectiveness of GRASP on computationally intensive, multi-view action classification using the INRIA IXMAS dataset and the i3DPost Multi-View dataset.

Biologically inspired motion detection neural network models evolved using genetic algorithms

In this paper we describe a method to evolve biologically inspired motion detection systems utilizing artificial neural networks (ANNs). Previously, the evolution of neural networks has focused on feed-forward neural networks or networks with predefined architectures. The purpose of this paper is to present a novel method for evolving neural networks with no predefined architectures to solve various problems including motion detection models. The neural network models are evolved with genetic algorithms using an encoding that defines a functional network with no restriction on recurrence, activation function types, or the number of nodes that compose the final ANN. The genetic algorithm operates on a population of potential solutions where each potential network is represented in a chromosome. The structure of each chromosome in the population is defined with a weight matrix which allows for efficient simulation of outputs. Each chromosome is evaluated by a fitness function that scores how well the actual output of an ANN compares to the expected output. Crossovers and mutations are made with specified probabilities between population members to evolve new members of the population. After a number of iterations a near optimal network is evolved that solves the problem at hand. The approach has proven to be sufficient to create biologically realistic motion detection neural network models with results that are comparable to results obtained from the standard Reichardt model.

Continuous recognition with incremental learning on Grassmann manifolds

Incremental learning allows incorporating new data in a classifier model without full retraining for computational efficiency. In this paper, we present two ways of performing incremental learning on Grassmann manifolds. In a Grassmann kernel learning framework, data are embedded on subspaces and kernels are constructed to map data subspaces to a projection space for classification. As new data samples become available, retraining degrades computational performance since Grassmann kernels need to be recomputed on larger matrices. We propose two computationally efficient techniques for incremental Grassmann kernel learning that achieve linear time complexity. We utilize the GROUSE framework to embed new data onto a pre-existing Grassmann manifold using Incremental Singular Value Decomposition (iSVD). Then we map the embeddings from a Grassmann space onto a projection space by exploiting the positive definite structure of Grassmann kernels and solving for principal angles of modified subspace pairs (iKernel). We show that our incremental learning approach is very effective in large systems and show examples for face recognition on standard datasets.

Grassmannian Spectral Regression for Learning and Classification

Computational performance associated with high dimensional data is a common challenge for real-world action classification systems. Subspace learning, and manifold learning in particular, have received considerable attention as means of finding efficient low-dimensional representations that lead to better classification and efficient processing. A Grassmann manifold is a space that promotes smooth surfaces where points represent subspaces. In this paper, Grassmannian Spectral Regression (GRASP) is presented as a Grassmann inspired subspace learning algorithm that combines the benefits of Grassmann manifolds and spectral regression for fast and accurate classification. GRASP involves embedding high dimensional action subspaces as individual points onto a Grassmann manifold, kernelizing the embeddings onto a projection space, and then applying Spectral Regression for fast and accurate action classification. Furthermore, spatiotemporal action descriptions called Motion History Surfaces and Motion Depth Surfaces are utilized. The effectiveness of GRASP is illustrated for computationally intensive, multi-view and 3D action classification datasets.

Keypoint matching and image registration using sparse representations

The field of sparse representations has found applications in a variety of computer vision and scientific fields. Although sparse representations were initially considered for reconstruction, they have been successfully adapted for classification. In this paper, we demonstrate the application of sparse representations in matching salient keypoint descriptors for image alignment, registration, or stitching. Our method initially builds a dictionary from keypoints in a reference image. Then keypoints associated with one or more secondary images are sparsely represented using the reference dictionary. The sparse coefficient signatures are analyzed to determine if there is a matched pair and identify the reference keypoint of the match. Top keypoint matches are used to construct the homography transformation for image registration. The usefulness of our methodology is demonstrated across several types of imagery showing robust performance while delivering state-of-the-art image alignment and registration.