Sami Gazzah

Unsupervised facial expressions recognition and avatar reconstruction from kinect

This paper presents a solution capable of recognizing the facial expressions performed by a persons face and mapping them to a 3D face virtual model using the depth and RGB data captured from Microsofts Kinect sensor. This solution starts by detecting the face and segmenting its regions, then, it identifies the actual expression using EigenFaces metrics on the RGB images and reconstructs the face from the filtered Depth data. A new dataset relative to 20 human subjects is introduced for learning purposes. It contains the images and point clouds for the different facial expressions performed. The algorithm seeks and displays automatically the seven state of the art expressions including surprise, fear, disgust, anger, joy, sadness and the neutral appearance. As result our system shows a morphing sequence between the sets of 3D face avatar models.

New Oversampling Approaches Based on Polynomial Fitting for Imbalanced Data Sets

In classification tasks, class-modular strategy has been widely used. It has outperformed classical strategy for pattern classification task in many applications. However, in some modular architecture, such as one against all in support vector machines classifier, the training dataset for one class risks to heavily outnumber the other classes. In this challenging situation, the trained classifier will accurately classify the majority class; nevertheless, it marginalizes the minority class. As a result, True Negatives rate (TNr) will be very high while the True Positives rate (TPr) will be low. The main goal of this work is to improve TPr without much sacrifice in TNr. In this paper, we propose oversampling the minority class using polynomial fitting functions. Four new approaches were proposed: star topology, bus topology, polynomial curve topology and mesh topology. Star and mesh topologies approach had led to the best performances.

Transductive Transfer Learning to Specialize a Generic Classifier Towards a Specific Scene

In this paper, we tackle the problem of domain adaptation to perform object-classification and detection tasks in video surveillance starting by a generic trained detector. Precisely, we put forward a new transductive transfer learning framework based on a sequential Monte Carlo filter to specialize a generic classifier towards a specific scene. The proposed algorithm approximates iteratively the target distribution as a set of samples (selected from both source and target domains) which feed the learning step of a specialized classifier. The output classifier is applied to pedestrian detection into a traffic scene. We have demonstrated by many experiments, on the CUHK Square Dataset and the MIT Traffic Dataset, that the performance of the specialized classifier outperforms the generic classifier and that the suggested algorithm presents encouraging results.

Faster R-CNN Scene Specialization with a Sequential Monte-Carlo Framework

The performance of the learning-based detector depends much on its training dataset and decreases rapidly when it is tested on a new scene. The reason is that in the large variations between the source training dataset and target scene. To solve this problem, we propose a novel approach to automatically specialize a generic detector to specific scene by utilizing the sequential Monte Carlo filter and the Faster R-CNN deep model. The main idea is to consider the Faster R-CNN as a function that generates realizations from the probability distribution of the object to be detected in the target sequence. Our contribution is to approximate this target probability distribution with a set of samples and an associated specialized Faster R-CNN estimated in a sequential Bayesian filter framework. The resulting algorithm is compared to the state of the art scene specialization methods on several challenging datasets, the results are very promising.

Augmented skeletal joints for temporal segmentation of sign language actions

We present in this paper a novel solution for temporal segmentation of human gestures that takes advantage of the skeletal-joints streams offered by the Kinect sensor. Our contribution consist in introducing an improved skeletal representation and its usage in a multilayer motion delimitation that distinguishes the non-vocabulary actions. The evaluation of the solution is presented on a subset of the Chalearn Gesture Challenge (CGC) 2014 dataset. The obtained temporal segmentation is better than the CGC baseline methods and has proved to be important for the task of human-action recognition.

Data embedding of 3D triangular mesh models using ordered ring facets

In this paper we propose a method for embedding binary data in triangular mesh models. Contrary to digital images and audio which benefit from the intrinsically ordered structure of the matrix and the array. 3D triangular mesh model lacks this capital property even though it can be encoded in an array date structure. Such a lack often complicates the different aspects of data embedding, like model traversal, data insertion, and synchronization. We address this problem with a mesh data representation which encodes the mesh data into a novel ordered structure, dubbed, the Ordered Rings Facets (ORF). This structure is composed of concentric rings in which the triangles are arranged in a circular fashion. This representation exhibits several interesting features that include a systematic traversing of the whole mesh model, simple mechanisms for avoiding the causality problem, and an efficient computation of the embedding distortion. Our method can be also adapted to different scenarios of data embedding, which includes stenography and fragile watermarking.

Vehicle detection on a video traffic scene: Review and new perspectives

Vehicle detection applications play an important role in the reduction of the number of road accidents. In the same vein, this paper tends to summarize the recent advances in vehicle detection approaches. Both the approaches based on motion and those based on appearance are dealt with. Also, the challenges and limitations of using handcraft features are discussed. Moreover, we compare different approaches cited as new perspectives in object detection. The experiments performed using two videos illustrate the robustness of the approach based on deep learning with specialization of the generic detector to a specific scene.

A Faster R-CNN Multi-Object Detector on a Nvidia Jetson TX1 Embedded System: Demo

This paper details the implementation of the deep Faster R-CNN algorithm (Faster R-CNN model) for multi-object detection on Nvidia Jetson TX1 Embedded System. The Jetson TX1 device is a development board proposed by NVIDIA for applications requiring high computational performance in a low-power envelope.

Modalities Combination for Italian Sign Language Extraction and Recognition

We propose in this work an approach for the automatic extraction and recognition of the Italian sign language using the RGB, depth and skeletal-joint modalities offered by Microsoft’s Kinect sensor. We investigate the best modality combination that improves the human-action spotting and recognition in a continuous stream scenario. For this purpose, we define per modality a complementary feature representation and fuse the decisions of multiple SVM classifiers with probability outputs. We contribute by proposing a multi-scale analysis approach that combines a global Fisher vector representation with a local frame-wise one. In addition we define a temporal segmentation strategy that allows the generation of multiple specialized classifiers. The final decision is obtained using the combination of their results. Our tests have been carried out on the Chalearn gesture challenge dataset, and promising results have been obtained on primary experiments.

Hands, face and joints for multi-modal human-action temporal segmentation and recognition

We present in this paper a new approach for human-action extraction and recognition in a multi-modal context. Our solution contains two modules. The first one applies temporal action segmentation by combining a heuristic analysis with augmented-joint description and SVM classification. The second one aims for a frame-wise action recognition using skeletal, RGB and depth modalities coupled with a label-grouping strategy in the decision level. Our contribution consists of (1) a selective concatenation of features extracted from the different modalities, (2) the introduction of features relative to the face region in addition to the hands, and (3) the applied multilevel frames-grouping strategy. Our experiments carried on the Chalearn gesture challenge 2014 dataset have proved the effectiveness of our approach within the literature.