Patrice Dalle

Segmented AAMs Improve Person-Indepedent Face Fitting.

An Active Appearance Model (AAM) is a variable shape and appearance model built from annotated training images. It has been largely used to synthesize or fit face images. Person-independent face AAM fitti ng is a challenging open issue. For standard AAMs, fitting a face image fo r an individual which is not in the training set is often limited in accuracy, thereby restricting the range of application. As a first contribution, we show that the limitation mainly co mes from the inability of the AAM appearance counterpart to generalize, i.e. to accurately generate previously unseen visual data. As a second contribution, we propose an efficient person-independent face fitting framework based on what we call multi-level segmented AAMs. Each segment encodes a physically meaningful part of the face, such as an eye. A coarse-to-fine fi tting strategy with a gradually increasing number of segments is used in order to ensure a large convergence basin. Fitting accuracy is assessed by comparison with manual labelling statistics constructed from multiple data annotations. Experimental results support the claim that standard AAMs are well-adapted to person-specific fitting while segmented AAMs outperform the classical AAMs in a personindependent context in terms of accuracy, and ability to generate new faces.

Using signing space as a representation for sign language processing

Sign language processing is often performed by processing each individual sign. Such an approach relies on an exhaustive description of the signs and does not take in account the spatial structure of the sentence. In this paper, we will present a general model of sign language sentences that uses the construction of the signing space as a representation of both the meaning and the realisation of the sentence. We will propose a computational model of this construction and explain how it can be attached to a sign language grammar model to help both analysis and generation of sign language utterances.

Sign language applications: preliminary modeling

For deaf persons to have ready access to information and communication technologies (ICTs), the latter must be usable in sign language (SL), i.e., include interlanguage interfaces. Such applications will be accepted by deaf users if they are reliable and respectful of SL specificities—use of space and iconicity as the structuring principles of the language. Before developing ICT applications, it is necessary to model these features, both to enable analysis of SL videos and to generate SL messages by means of signing avatars. This paper presents a signing space model, implemented within a context of automatic analysis and automatic generation, which are currently under development.

Toward an efficient and accurate AAM fitting on appearance varying faces

Automatic extraction of facial feature deformations (either due to identity change or expression) is a challenging task and could be the base of a facial expression interpretation system. We use active appearance models and the simultaneous inverse compositional algorithm to extract facial deformations as a starting point and propose a modified version addressing the problem of facial appearance variation in an efficient manner. To consider important variation of facial appearance is a first step toward a realistic facial feature deformation extraction system able to adapt to a new face or to track a face with changing video conditions. Moreover, in order to test fittings, we design an experiment protocol that takes human inaccuracies into account when building a ground truth

Body posture estimation in sign language videos

This article deals with the posture reconstruction from a mono view video of a signed utterance. Our method makes no use of additional sensors or visual markers. The head and the two hands are tracked by means of a particle filter. The elbows are detected as convolution local maxima. A non linear filter is first used to remove the outliers, then some criteria using French Sign Language phonology are used to process the hand disambiguation. The posture reconstruction is achieved by using inverse kinematics, using a Kalman smoothing and the correlation between strong and week hand depth that can be noticed in the signed utterances. The article ends with a quantitative and qualitative evaluation of the reconstruction. We show how the results could be used in the framework of automatic Sign Language video processing.

Occluded facial expression tracking

The work presented here takes place in the field of computer aided analysis of facial expressions displayed in sign language videos. We use Active Appearance Models to model a face and its variations of shape and texture caused by expressions. The inverse compositional algorithm is used to accurately fit an AAM to the face seen on each video frame. In the context of sign language communication, the signers face is frequently occluded, mainly by hands. A facial expression tracker has then to be robust to occlusions. We propose to rely on a robust variant of the AAM fitting algorithm to explicitly model the noise introduced by occlusions. Our main contribution is the automatic detection of hand occlusions. The idea is to model the behavior of the fitting algorithm on unoccluded faces, by means of residual image statistics, and to detect occlusions as being what is not explained by this model. We use residual parameters with respect to the fitting iteration i.e., the AAM distance to the solution, which greatly improves occlusion detection compared to the use of fixed parameters. We also propose a robust tracking strategy used when occlusions are too important on a video frame, to ensure a good initialization for the next frame.

Langues des signes, enseignement et nouvelles technologies

La Langue des signes francaise (LSF) possede differents statuts suivant l’usage qu’on en fait, puisque c’est a la fois un objet d’etude pour differentes disciplines (linguistique, sociolinguistique, informatique, …), une langue enseignee dans differents cursus scolaires et universitaires et une langue utilisee dans l’enseignement et dans la vie quotidienne des personnes sourdes. Ces differents usages, amplifies par la loi de 2005, ont fait emerger de nouveaux besoins d’outils faisant appel aux TIC, puisque le support naturel de la LSF est la video. C’est ainsi qu’on doit pouvoir illustrer le fonctionnement de la LSF, fabriquer et exploiter des documents en LSF, evaluer et corriger des productions en LSF et, enfin, permettre son apprentissage. La conception de ces outils doit respecter des contraintes liees aux situations d’utilisation, notamment celles des enseignants, et aux particularites de la LSF elle-meme. Enfin, l’impact de ces developpements technologiques sur la LSF elle-meme doit etre pris en compte pour eviter d’en diffuser une version appauvrie. Nous presentons plusieurs de ces outils et montrons dans quelle mesure ils permettent a la video de compenser l’absence d’une forme ecrite de la LSF. Nous montrons enfin les potentialites d’Internet pour diffuser des contenus en LSF ou apprendre cette langue. Si des travaux doivent encore etre menes pour que ces technologies repondent a l’ensemble des besoins tout en etant simples et rapides a utiliser et peu couteuses, les outils existants et les savoir-faire permettent deja d’utiliser la LSF dans l’enseignement et de l’enseigner. Cela suppose une volonte politique de developper ces usages dans l’enseignement et de permettre aux locuteurs sourds d’acceder aux fonctions d’enseignants.

Analyse des langues des signes: Démarche de conception pluridisciplinaire d'outils d'analyse de discours en langues des signes

By now, professionals of Sign Language (SL) do not have yet adapted tools to process and analyze videos of SL utterances. The creation of such softwares requires a real collaboration between SL teachers, interpreter trainers, linguists, sociologists and computer scientists. This article illustrates through two examples, the approach of such multidisciplinary projects. It also shows how taking into account the specific needs of each profession can lead to tools providing adapted solutions that remain generic enough to fulfill other uses of SL in the media. Using those softwares, it is possible to annotate an SL production in SL. It is also possible to represent its grammatical structure by means of augmented reality.