Preben Fihl

View-invariant gesture recognition using 3D optical flow and harmonic motion context

This paper presents an approach for view-invariant gesture recognition. The approach is based on 3D data captured by a SwissRanger SR4000 camera. This camera produces both a depth map as well as an intensity image of a scene. Since the two information types are aligned, we can use the intensity image to define a region of interest for the relevant 3D data. This data fusion improves the quality of the motion detection and hence results in better recognition. The gesture recognition is based on finding motion primitives (temporal instances) in the 3D data. Motion is detected by a 3D version of optical flow and results in velocity annotated point clouds. The 3D motion primitives are represented efficiently by introducing motion context. The motion context is transformed into a view-invariant representation using spherical harmonic basis functions, yielding a harmonic motion context representation. A probabilistic Edit Distance classifier is applied to identify which gesture best describes a string of primitives. The approach is trained on data from one viewpoint and tested on data from a very different viewpoint. The recognition rate is 94.4% which is similar to the recognition rate when training and testing on gestures from the same viewpoint, hence the approach is indeed view-invariant.

Fusion of range and intensity information for view invariant gesture recognition

This paper presents a system for view invariant gesture recognition. The approach is based on 3D data from a CSEM SwissRanger SR-2 camera. This camera produces both a depth map as well as an intensity image of a scene. Since the two information types are aligned, we can use the intensity image to define a region of interest for the relevant 3D data. This data fusion improves the quality of the range data and hence results in better recognition. The gesture recognition is based on finding motion primitives in the 3D data. The primitives are represented compactly and view invariant using harmonic shape context. A probabilistic Edit Distance classifier is applied to identify which gesture best describes a string of primitives. The approach is trained on data from one viewpoint and tested on data from a different viewpoint. The recognition rate is 92.9% which is similar to the recognition rate when training and testing on gestures from the same viewpoint, hence the approach is indeed view invariant.

View invariant gesture recognition using the CSEM SwissRanger SR-2 camera

This paper introduces the use of range information acquired by a CSEM SwissRanger SR-2 camera for view invariant recognition of one and two arms gestures. The range data enables motion detection and 3D representation of gestures. Motion is detected by double difference range images and filtered by a hysteresis bandpass filter. Gestures are represented by concatenating harmonic shape contexts over time. This representation allows for a view invariant matching of the gestures. The system is trained on gestures from one viewpoint and evaluated on gestures from other viewpoints. The results show a recognition rate of 93.75%.

Tracking of individuals in very long video sequences

In this paper we present an approach for automatically detecting and tracking humans in very long video sequences. The detection is based on background subtraction using a multi-mode Codeword method. We enhance this method both in terms of representation and in terms of automatically updating the background allowing for handling gradual and rapid changes. Tracking is conducted by building appearance-based models and matching these over time. Tests show promising detection and tracking results in a ten hour video sequence.

Tamper detection for active surveillance systems

If surveillance data are corrupted they are of no use to neither manually post-investigation nor automatic video analysis. It is therefore critical to automatically be able to detect tampering events such as defocusing, occlusion and displacement. In this work we for the first time address this important problem for an active camera. We detect these events by first comparing the incoming frames to a background model using features relevant for the three different tampering types. Individual detectors are then developed capable of monitoring long video sequences and indicating the occurrence of different tampering events. In order to assess the developed methods we have collected a large data set, which contains sequences from different active cameras at different scenarios. We evaluate our system on these data and the results are encouraging with a very high detecting rate and relatively few false positives.

Invariant gait continuum based on the duty-factor

In this paper, we present a method to describe the continuum of human gait in an invariant manner. The gait description is based on the duty-factor which is adopted from the biomechanics literature. We generate a database of artificial silhouettes representing the three main types of gait, i.e. walking, jogging, and running. By generating silhouettes from different camera angles we make the method invariant to camera viewpoint and to changing directions of movement. Silhouettes are extracted using the Codebook method and represented in a scale- and translation-invariant manner by using shape contexts and tangent orientations. Input silhouettes are matched to the database using the Hungarian method. We define a classifier based on the dissimilarity between the input silhouettes and the gait actions of the database. This classification achieves an overall recognition rate of 87.1% on a diverse test set, which is better than that achieved by other approaches applied to similar data. We extend this classification and results show that our representation of the gait continuum preserves the main features of the duty-factor.

Pose Estimation of Interacting People using Pictorial Structures

Pose estimation of people have had great progress in recentyears but so far research has dealt with single persons.In this paper we address some of the challenges that arisewhen doing pose estimation of interacting people. We buildon the pictorial structures framework and make importantcontributions by combining color-based appearance andedge information using a measure of the local quality ofthe appearance feature. In this way we not only combinethe two types of features but dynamically find the optimalweighting of them. We further enable the method to handleocclusions by searching a foreground mask for possibleoccluded body parts and then applying extra strong kinematicconstraints to find the true occluded body parts. Theeffect of applying our two contributions are show throughboth qualitative and quantitative tests and show a clear improvementon the ability to correctly localize body parts.

Classification of gait types based on the duty-factor

This paper deals with classification of human gait types based on the notion that different gait types are in fact different types of locomotion, i.e., running is not simply walking done faster. We present the duty-factor, which is a descriptor based on this notion. The duty-factor is independent on the speed of the human, the cameras setup etc. and hence a robust descriptor for gait classification. The duty-factor is basically a matter of measuring the ground support of the feet with respect to the stride. We estimate this by comparing the incoming silhouettes to a database of silhouettes with known ground support. Silhouettes are extracted using the codebook method and represented using shape contexts. The matching with database silhouettes is done using the Hungarian method. While manually estimated duty-factors show a clear classification the presented system contains misclassifications due to silhouette noise and ambiguities in the database silhouettes.

Motion Primitives and Probabilistic Edit Distance for Action Recognition

The number of potential applications has made automatic recognition of human actions a very active research area. Different approaches have been followed based on trajectories through some state space. In this paper we also model an action as a trajectory through a state space, but we represent the actions as a sequence of temporal isolated instances, denoted primitives. These primitives are each defined by four features extracted from motion images. The primitives are recognized in each frame based on a trained classifier resulting in a sequence of primitives. From this sequence we recognize different temporal actions using a probabilistic Edit Distance method. The method is tested on different actions with and without noise and the results show recognition rates of 88.7% and 85.5%, respectively.

Invariant Classification of Gait Types

This paper presents a method of classifying human gait in an invariant manner based on silhouette comparison. A database of artificially generated silhouettes is created representing the three main types of gait, i.e. walking, jogging, and running. Silhouettes generated from different camera angles are included in the database to make the method invariant to camera viewpoint and to changing directions of movement. The extraction of silhouettes are done using the Codebook method and silhouettes are represented in a scale- and translation-invariant manner by using shape contexts and tangent orientations. Input silhouettes are matched to the database using the Hungarian method. A classifier is defined based on the dissimilarity between the input silhouettes and the gait actions of the database. The overall recognition rate is 88.2% on a large and diverse test set. The recognition rate is better than that achieved by other approaches applied to similar data.