Jérôme Martin

Active hand tracking

This paper describes a system which uses multiple visual processes to detect and track hands for gesture recognition and human-computer interaction. This system is based on an architecture in which a supervisor selects and activates visual processes. Each process provides a confidence factor which makes it possible for the system to dynamically reconfigure itself in response to events in the scene. Visual processes for hand tracking are described using image differencing and normalized histogram matching. The result of hand detection is used by a recursive estimator (Kalman filter) to provide an estimate of the position and size of the hand. The resulting system provides robust and precise tracking which operates continuously at approximately 5 images per second on a 150 megahertz Silicon Graphics Indy.

An Appearance-Based Approach to Gesture-Recognition

This paper describes techniques for the design of a system able to interact with the user by visual recognition of hand gestures. The system is composed of three modules including tracking, posture classification and gesture recognition. A description of each module is given. In order to increase the robustness and the precision of the tracking, several complementary tracking processes are coupled. A classification process is presented for recognizing hand posture using distance in an eigenspace. The classification of hand posture leads to the gesture recognition by a set of finite state machines.

MagicBoard: A contribution to an intelligent office environment

Abstract In this paper, we describe an augmented reality tool for collaborative work called the MagicBoard. The MagicBoard is based on an ordinary white board which has been enhanced by a video-projector and a steerable camera. A supervisor coordinates the cooperation of several modules including gesture recognition, finger tracking and white board scanning for digitalisation of the content. The gesture recognition module uses an approach based on local spatio-temporal appearance of activities. The tracking module is designed for use with cluttered backgrounds and variable lighting conditions. The white board scanner eliminates global luminosity differences by adaptive thresholding and the result can serve to digitise the content of the board. The supervisor is based on a rule-based architecture and is easily extendable. The selected modules fit together to a compact system, that largely increases the functionality of a white board and makes it a useful tool in the future office environments.

A Probabilistic Sensor for the Perception and Recognition of Activities

This paper presents a new technique for the perception and recognition of activities using statistical descriptions of their spatio-temporal properties. A set of motion energy receptive fields is designed in order to sample the power spectrum of a moving texture. Their structure relates to the spatio-temporal energy models of Adelson and Bergen where measures of local visual motion information are extracted by comparing the outputs of a triad of Gabor energy filters. Then the probability density function required for Bayes rule is estimated for each class of activity by computing multi-dimensional histograms from the outputs from the set of receptive fields. The perception of activities is achieved according to Bayes rule. The result at each instant of time is the map of the conditional probabilities that each pixel belongs to each one of the activities of the training set. Since activities are perceived over a short integration time, a temporal analysis of outputs is done using Hidden Markov Models. The approach is validated with experiments in the perception and recognition of activities of people walking in visual surveillance scenari. The presented work is in progress and preliminary results are encouraging, since recognition is robust to variations in illumination conditions, to partial occlusions and to changes in texture. It is shown that it constitute a powerful early vision tool for human behaviors analysis for smart-environnements.

Automatic handwriting gestures recognition using hidden Markov models

Hidden Markov models have been successfully employed in speech recognition and, more recently, in sign language interpretation. They seem adequate for visual recognition of gestures. In this paper, two problems often eluded are considered. We propose to use the Bayesian information criterion in order to determine the optimal number of model states. We describe the contribution of continuous models in opposition to symbolic ones. Experiments on handwriting gestures show recognition rate between 88% and 100%.

Statistical Gesture Recognition Through Modelling of Parameter Trajectories

The recognition of human gestures is a challenging problem that can contribute to a natural man-machine interface. In this paper, we present a new technique for gesture recognition. Gestures are modelled as temporal trajectories of parameters. Local sub-sequences of these trajectories are extracted and used to define an orthogonal space using principal component analysis. In this space the probabilistic density function of the training trajectories is represented by a multidimensional histogram, which builds the basis for the recognition. Experiments on three different recognition problems show the general utility of the approach.

Smart Office: An Intelligent and Interactive Environment

This paper presents our Intelligent Environment called Smart Office. In the Smart Office the user can work as in a normal office. The office’s intelligence observes the user in order to anticipate his intentions and augments his environment to communicate useful information. Computers are involved in user activities in order to help in everyday tasks. The system interacts with users using voice, gesture or movement.