Thales Vieira

Real-Time Gesture Recognition from Depth Data through Key Poses Learning and Decision Forests

Human gesture recognition is a challenging task with many applications. The popularization of real time depth sensors even diversifies potential applications to end-user natural user interface (NUI). The quality of such NUI highly depends on the robustness and execution speed of the gesture recognition. This work introduces a method for real-time gesture recognition from a noisy skeleton stream, such as the ones extracted from Kinect depth sensors. Each pose is described using a tailored angular representation of the skeleton joints. Those descriptors serve to identify key poses through a multi-class classifier derived from Support Vector learning machines. The gesture is labeled on-the-fly from the key pose sequence through a decision forest, that naturally performs the gesture time warping and avoids the requirement for an initial or neutral pose. The proposed method runs in real time and shows robustness in several experiments.

Online gesture recognition from pose kernel learning and decision forests

The recent popularization of real time depth sensors has diversified the potential applications of online gesture recognition to end-user natural user interface (NUI). This requires significant robustness of the gesture recognition to cope with the noisy data from the popular depth sensor, while the quality of the final NUI heavily depends on the recognition execution speed. This work introduces a method for real-time gesture recognition from a noisy skeleton stream, such as those extracted from Kinect depth sensors. Each pose is described using an angular representation of the skeleton joints. Those descriptors serve to identify key poses through a Support Vector Machine multi-class classifier, with a tailored pose kernel. The gesture is labeled on-the-fly from the key pose sequence with a decision forest, which naturally performs the gesture time control/warping and avoids the requirement for an initial or neutral pose. The proposed method runs in real time and its robustness is evaluated in several experiments.

Learning good views through intelligent galleries

The definition of a good view of a 3D scene is highly subjective and strongly depends on both the scene content and the 3D application. Usually, camera placement is performed directly by the user, and that task may be laborious. Existing automatic virtual cameras guide the user by optimizing a single rule, e.g. maximizing the visible silhouette or the projected area. However, the use of a static pre‐defined rule may fail in respecting the users subjective understanding of the scene. This work introduces intelligent design galleries, a learning approach for subjective problems such as the camera placement. The interaction of the user with a design gallery teaches a statistical learning machine. The trained machine can then imitate the user, either by pre‐selecting good views or by automatically placing the camera. The learning process relies on a Support Vector Machines for classifying views from a collection of descriptors, ranging from 2D image quality to 3D features visibility. Experiments of the automatic camera placement demonstrate that the proposed technique is efficient and handles scenes with occlusion and high depth complexities. This work also includes user validations of the intelligent gallery interface.

SMI 2011: Full Paper: Interactive 3D caricature from harmonic exaggeration

A common variant of caricature relies on exaggerating characteristics of a shape that differs from a reference template, usually the distinctive traits of a human portrait. This work introduces a caricature tool that interactively emphasizes the differences between two three-dimensional meshes. They are represented in the manifold harmonic basis of the shape to be caricatured, providing intrinsic controls on the deformation and its scales. It further provides a smooth localization scheme for the deformation. This lets the user edit the caricature part by part, combining different settings and models of exaggeration, all expressed in terms of harmonic filter. This formulation also allows for interactivity, rendering the resulting 3d shape in real time.

Y-STR haplotype diversity and population data for Central Brazil: implications for environmental forensics and paternity testing.

The central region of Brazil was colonized by internal migration of individuals of different origins, who contributed to the genetic diversity existing in this population. This study determined the allele frequencies and haplotype diversity of Y-STRs in Goiás State, Central Brazil, and compared the data obtained with a sample of the Brazilian population, consisting of individuals from the five geographical regions of Brazil. A total of 353 males were typed for 12 Y-chromosome short tandem repeat (Y-STR) markers. We selected males who had no degree of relatedness, from the five mesoregions of Goiás State. DNA was extracted from blood samples followed by the amplification of the 12 Y-chromosome loci. The products were analyzed to obtain the allele profiles on an ABI3500 automated sequencer using the Gene Mapper software. Allele frequencies and haplotype diversity were estimated by direct counting, and gene diversity for each locus was computed using the Arlequin software. The results are consistent with the history of miscegenation of the population of Central Brazil, in which we observed 321 different haplotypes. The average gene diversity at the 12 loci was 0.645. DYS385b and DYS389I showed the highest (0.704) and lowest (0.520) genetic diversity values, respectively. The FST value between the Brazilian and Goiás populations was 0.00951, showing no statistical significance. The results of this study allowed the establishment of haplotypes found in the forensic samples of Goiás State serving as a reference in the elucidation of criminal cases and paternity tests, as well as population and evolutionary inferences.

Estimating affine-invariant structures on triangle meshes

Abstract Affine invariant measures are powerful tools to develop robust shape descriptors that can be applied, for example, to shape matching, shape retrieval, or symmetry detection problems. In this work we introduce estimators for the affine structure of surfaces represented by triangle meshes, i.e. affine co-normal and normal vectors, affine curvature tensors, affine mean and Gaussian curvatures, and affine principal directions and curvatures. The proposed method estimates the affine normal using a finite differences scheme together with a least-squares approximation, followed by a weighted average strategy to approach discrete affine curvature tensors. When compared to the exact geometric measures of analytic models, experiments on regular meshes obtain small error, which decreases for finer meshes, and outperforms the state-of-the-art method in some cases. Experiments to evaluate affine invariance show that the difference between measures before and after equi-affine transformations remains small even after large deformations.

Tuning Manifold Harmonics Filters

There are several techniques for automatic music visualization, which are included with virtually any media player. The basic ingredient of those techniques is spectral analysis of the sound, used to automatically generate parameters for procedural image generation. However, only a few music visualizations rely on 3d models. This paper proposes to use spectral mesh processing techniques, namely manifold harmonics, to produce 3d music visualization. The images are generated from 3d models by deforming an initial shape, mapping the sound frequencies to the mesh harmonics. A concise representation of such frequency mapping is proposed to permit for an animated gallery interface with genetic reproduction. Such galleries allow the user to quickly navigate between visual effects. Rendering such animated galleries in real-time is a challenging task, since it requires computing and rendering the deformed shapes at a very high rate. This paper introduces a direct GPU implementation of manifold harmonics filters, which allows to display animated gallery.

Online human moves recognition through discriminative key poses and speed-aware action graphs

Recognizing user-defined moves serves a large number of applications including sport monitoring, virtual reality or natural user interfaces (NUI). However, many of the efficient human move recognition methods are still limited to specific situations, such as straightforward NUI gestures or everyday human actions. In particular, most methods depend on a prior segmentation of recordings to both train and recognize moves. This segmentation step is generally performed manually or based on heuristics such as neutral poses or short pauses, limiting the range of applications. Besides, speed is generally not considered as a criterion to distinguish moves. We present an approach composed of a simplified move training phase that requires minimal user intervention, together with a novel online method to robustly recognize moves online from unsegmented data without requiring any transitional pauses or neutral poses, and additionally considering human move speed. Trained gestures are automatically segmented in real time by a curvature-based method that detects small pauses during a training session. A set of most discriminant key poses between different moves is also extracted in real time, optimizing the number of key poses. All together, this semi-supervised learning approach only requires continuous move performances from the user with small pauses. Key pose transitions and moves execution speeds are used as input to a novel human move recognition algorithm that recognizes unsegmented moves online, achieving high robustness and very low latency in our experiments, while also effective in distinguishing moves that differ only in speed.

High performance moves recognition and sequence segmentation based on key poses filtering

We present a discriminative key pose-based approach for moves recognition and segmentation of training sequences for high performance sports. Compared to daily human gestures, moves in high performance sports are faster and have low inter-class variability, which produce noisy features and ambiguity. Our approach combines a robust filtering strategy to select frames composed of discriminative poses (key poses) and the discriminative Latent-Dynamic Conditional Random Fields (LDCRF) model to predict a label for each frame from the training sequence. We evaluate our approach on unsegmented sequences of Taekwondo training. Experimental results indicate that our methodology outperforms the Decision Forests method in terms of efficiency and accuracy. Our average recognition rate was equal to 74.72% while Decision Forests achieves 58.29%. The experiments also show that our approach was able to recognize and segment high speed moves like roundhouse kicks, which can reach peak linear speeds up to 26 m/s.

Simplified Training for Gesture Recognition

Since gesture is a fundamental form of human communication, its recognition by a computer generates a strong interest for many applications such as natural user interface and gaming. The popularization of real-time depth sensors brings such applications to the public at large. However, familiar gestures are culture-specific, and their automatic recognition must therefore result from a machine learning process. In particular this requires either teaching the user how to communicate with the machine, such as for popular mobile devices or gaming consoles, or tailoring the application to a specific public. The latter option serves a large number of applications such as sport monitoring, virtual reality or surveillance -- although it requires a usually tedious training. This work intends to simplify the training required by gesture recognition methods. While the traditional procedure uses a set of key poses, which must be defined and trained, prior to a set of gestures that must also be defined and trained, we propose to automatically deduce the set of key poses from the gesture training. We represent a record of gestures as a curve in high dimension and robustly segment it according to the curvature of that curve. Then we use an asymmetric Hausdorff distance between gestures to define a discriminant key pose as the most distant pose between gestures. This further allows to dynamically group gestures by similarity. The training only requires the user to perform the gestures and eventually refine the gesture labeling. The generated set of key poses and gestures then fits in previous human action recognition algorithms. Furthermore, this semi-supervised learning allows re-using a previous training to extend the set of gestures the computer should be able to recognize. Experimental results show that the automatically generated discriminant key poses lead to similar recognition accuracy as previous work.