Nazim Ashraf

View invariant action recognition using projective depth

Abstract In this paper, we investigate the concept of projective depth , demonstrate its application and significance in view-invariant action recognition. We show that projective depths are invariant to camera internal parameters and orientation, and hence can be used to identify similar motion of body-points from varying viewpoints. By representing the human body as a set of points, we decompose a body posture into a set of projective depths . The similarity between two actions is, therefore, measured by the motion of projective depths . We exhaustively investigate the different ways of extracting planes, which can be used to estimate the projective depths for use in action recognition including (i) ground plane, (ii) body-point triplets, (iii) planes in time, and (iv) planes extracted from mirror symmetry. We analyze these different techniques and analyze their efficacy in view-invariant action recognition. Experiments are performed on three categories of data including the CMU MoCap dataset, Kinect dataset, and IXMAS dataset. Results evaluated over semi-synthetic video data and real data confirm that our method can recognize actions, even when they have dynamic timeline maps, and the viewpoints and camera parameters are unknown and totally different.

Action recognition based on homography constraints

In this paper, we present a new approach for view-invariant action recognition using constraints derived from the eigenvalues of planar homographies associated with triplets of body points. Unlike existing methods that study an action as a whole, or break it down into individual poses, we represent an action as a sequence of pose transitions. Using the fact that the homography induced by the motion of a triplet of body points in two identical pose transitions reduces to the special case of a homology, we exploit the equality of two of its eigenvalues to impose constraints on the similarity of the pose transitions between two subjects, observed by different perspective cameras and from different viewpoints. Experimental results show that our method can accurately identify human pose transitions and actions even when they include dynamic timeline maps, and are obtained from totally different viewpoints with different camera parameters.

Human action recognition in video data using invariant characteristic vectors

We introduce the concept of the “characteristic vector” as an invariant vector associated with a set of freely moving points relative to a plane. We show that if the motion of two sets of points differ only up to a similarity transformation, then the elements of the characteristic vector differ up to scale regardless of viewing directions and cameras. Furthermore, this invariant vector is given by any arbitrary homography that is consistent with epipolar geometry. The characteristic vector of moving points can thus be used to recognize the transitions of a set of points in an articulated body during the course of an action regardless of the camera orientation and parameters. Our extensive experimental results on both motion capture data and real data indicates very good performance.

View-Invariant Action Recognition Using Rank Constraint

We propose a new method for view-invariant action recognition based on the rank constraint on the family of planar homographies associated with triplets of body points. We represent action as a sequence of poses and we use the fact that the family of homographies associated with two identical poses would have rank 4 to gauge similarity of the pose between two subjects, observed by different perspective cameras and from different viewpoints. Extensive experimental results show that our method can accurately identify action from video sequences when they are observed from totally different viewpoints with different camera parameters.

Motion retrieval using consistency of epipolar geometry

In this paper, we present an efficient method for motion retrieval method based on the consistency of the homographies with the epipolar geometry. We treat the body pose as body point triplets and use the fact that each homography obtained from corresponding body point triplets should be consistent with epipolar geometry to estimate the similarity of two poses. We show that our method is invariant to camera internal parameters and viewpoint. Experiments are performed on the CMU MoCap dataset, and IXMAS dataset testing testing view-invariance, and action recognition. The results demonstrate that our method can accurately identify human action from video sequences when they are observed from totally different viewpoints with different camera parameters.

Motion retrival using low-rank decomposition of Fundamental Ratios

This paper proposes a novel framework for efficient retrieval of motion capture data. The method uses Fundamental Ratios to convert action sequences into compact representations of the action, greatly reducing the spatiotemporal dimensionality of the sequences. We propose a low-rank decomposition scheme that allows for converting the motion sequence volumes into compact lower dimensional representations, without losing the nonlinear dynamics of the motion manifold, and the proposed method performs well even when interclass differences are small or intra-class differences are large. We evaluate the performance of our retrieval framework on the CMU mocap dataset and Microsoft Kinect dataset, which demonstrate satisfying retrieval rates.

View invariant action recognition using weighted fundamental ratios

Highlights? We generalize the concept of fundamental ratios. ? We propose a generic method for weighting body point line segments. ? We study how this weighting strategy is useful when there is partial occlusion. ? We also investigate how soon our method is able to recognize the action. ? We provide extensive experiments to test our method on three different datasets. In this paper, we fully investigate the concept of fundamental ratios, demonstrate their application and significance in view-invariant action recognition, and explore the importance of different body parts in action recognition. A moving plane observed by a fixed camera induces a fundamental matrix F between two frames, where the ratios among the elements in the upper left 2i?2 submatrix are herein referred to as the fundamental ratios. We show that fundamental ratios are invariant to camera internal parameters and orientation, and hence can be used to identify similar motions of line segments from varying viewpoints. By representing the human body as a set of points, we decompose a body posture into a set of line segments. The similarity between two actions is therefore measured by the motion of line segments and hence by their associated fundamental ratios. We further investigate to what extent a body part plays a role in recognition of different actions and propose a generic method of assigning weights to different body points. Experiments are performed on three categories of data: the controlled CMU MoCap dataset, the partially controlled IXMAS data, and the more challenging uncontrolled UCF-CIL dataset collected on the internet. Extensive experiments are reported on testing (i) view-invariance, (ii) robustness to noisy localization of body points, (iii) effect of assigning different weights to different body points, (iv) effect of partial occlusion on recognition accuracy, and (v) determining how soon our method recognizes an action correctly from the starting point of the query video.

Robust auto-calibration of a PTZ camera with non-overlapping FOV

We consider the problem of auto-calibration of cameras, which are fixed in location but are free to rotate while changing their internal parameters by zooming. Our method is based on line correspondences between two views, which may have non-overlapping field of view. Camera calibration from images having non-overlapping field of view is the basic motivation behind this research. The key observation is that the planes formed by the optic center and the line correspondences are really the same plane. We use this fact together with the orthonormality constraint of the rotation matrix to estimate the unknown camera parameters. We show experimental results on synthetic and real data, and analyze the accuracy and stability of our method.

Robust Auto-Calibration using Fundamental Matrices Induced by Pedestrians

The knowledge of camera intrinsic and extrinsic parameters is useful, as it allows us to make world measurements. Unfortunately, calibration information is rarely available in video surveillance systems and is difficult to obtain once the system is installed. Auto-calibrating cameras using moving objects (humans) has recently attracted a lot of interest. Two methods were proposed by Lv-Nevatia (2002) and Krahnstoever-Mendonca (2005). The inherent difficulty of the problem lies in the noise that is generally present in the data. We propose a robust and a general linear solution to the problem by adopting a formulation different from the existing methods. The uniqueness of our formulation lies in recognizing two fundamental matrices present in the geometry obtained by observing pedestrians, and then using their properties to impose linear constraints on the unknown camera parameters. Experiments with synthetic as well as real data are presented -indicating the practicality of the proposed system.