Antonio Colmenarez

Face detection with information-based maximum discrimination

In this paper we present a visual learning technique that maximizes the discrimination between positive and negative examples in a training set. We demonstrate our technique in the context of face detection with complex background without color or motion information, which has proven to be a challenging problem. We use a family of discrete Markov processes to model the face and background patterns and estimate the probability models using the data statistics. Then, we convert the learning process into an optimization, selecting the Markov process that optimizes the information-based discrimination between the two classes. The detection process is carried out by computing the likelihood ratio using the probability model obtained from the learning procedure. We show that because of the discrete nature of these models, the detection process is at least two orders of magnitude less computationally expensive than neural network approaches. However, no improvement in terms of correct-answer/false-alarm tradeoff is achieved.

A probabilistic framework for embedded face and facial expression recognition

We present a Bayesian recognition framework in which a model of the whole face is enhanced by models of facial feature position and appearances. Face recognition and facial expression recognition are carried out using maximum likelihood decisions. The algorithm finds the model and facial expression that maximizes the likelihood of a test image. In this framework, facial appearance matching is improved by facial expression matching. Also, changes in facial features due to expressions are used together with facial deformation. Patterns to jointly perform expression recognition. In our current implementation, the face is divided into 9 facial features grouped in 4 regions which are detected and tracked automatically in video segments. The feature images are modeled using Gaussian distributions on a principal component sub-space. The training procedure is supervised; we use video segments of people in which the facial expressions have been segmented and labeled by hand. We report results on face and facial expression recognition using a video database of 18 people and 6 expressions.

Detection and tracking of faces and facial features

We describe a real-time system for face and facial feature detection and tracking in continuous video. The core of this system consists of a set of novel facial feature detectors based on our previously proposed information-based maximum discrimination learning technique. These classifiers are very fast and allow us to implement a fully automatic, real-time system for detection and tracking multiple faces. In addition to locking onto up to four target faces, this system locates and tracks nine facial features as they move under facial expression changes.

Maximum likelihood face detection

We present a visual learning approach that uses non-parametric probability estimators. We use entropy analysis over the training set in order to select the features that best represent the pattern class of faces, and set up discrete probability models. These models are tested in the context of maximum likelihood detection of faces. Excellent results are reported in terms of the correct-answer-false-alarm tradeoff as well as in terms of the computational requirements of the systems.

Speech activity detection using acoustic and facial characteristics in an automatic speech recognition system

An automatic speech recognizer only responsive to acoustic speech utterances is activated only in response to acoustic energy having a spectrum associated with the speech utterances and at least one facial characteristic associated with the speech utterances. In one embodiment, a speaker must be looking directly into a video camera and the voices and facial characteristics of plural speakers must be matched to enable activation of the automatic speech recognizer.

A computer vision system for on-screen item selection by finger pointing

Pointing at planar surfaces such as TV and computer monitors or projection screens can be a useful mode of interaction between humans and machines. To a large extent what seems to hinder the use of vision in such practical applications is the difficulty of the computational task, which is typically defined as 3-D reconstruction from uncalibrated 2-D images of a non-static scene. We describe below two designs where, using one or two cameras, the target of pointing on a flat monitor or screen is identified without 3-D inference, using only image morphing and line intersection. This is accomplished by registering the images with the target plane. When used to identify a pointing target on a surface hidden from the camera (e.g., a computer monitor which supports the camera itself as in most PC configurations), we add aperture(s) coplanar with the target surface in front of the camera(s). We describe experimental results showing a fully automated procedure for pointing target detection with high accuracy. The simplicity of our method and its robustness, as well as the relative accuracy of our results, can make pointing a practical means of human-machine interaction.

Embedded face and facial expression recognition

A framework for embedded recognition of faces and facial expressions is described. Faces are modeled based on the appearances and positions of facial features. Hidden states are used to represent discrete facial expressions. A face model is constructed for each person in the database using video segments showing different facial expressions. Face recognition and facial expression recognition are carried out using Bayesian classification. In our current implementation, the face is divided into nine facial features grouped in four regions which are detected and tracked automatically in video segments. We report results on face and facial expression recognition using a video database of 18 people and six expressions.

Pattern detection with information-based maximum discrimination and error bootstrapping

We have previously (1996, 1997) introduced a visual learning technique based on information-theoretic entropy. In that approach, positive and negative examples of a class of visual patterns were analyzed to obtain the probability model that best discriminate the class among others. Such models were tested in the context of maximum likelihood detection of faces and facial features. In this paper we further improve on that technique by using other family of probability model and by extending the optimization criteria to allow for error bootstrapping. The results include a detail analysis of the improvements obtained and a comparison of these pattern recognition algorithms.

Face Detection and Recognition

Two of the most important aspects in the general research framework of face recognition by computer are addressed here: face and facial feature detection, and face recognition — or rather face comparison. The best reported results of the mug-shot face recognition problem are obtained with elastic matching using jets. In this approach, the overall face detection, facial feature localization, and face comparison is carried out in a single step. This paper describes our research progress towards a different approach for face recognition. On the one hand, we describe a visual learning technique and its application to face detection in complex background, and accurate facial feature detection/tracking. On the other hand, a fast algorithm for 2D-template matching is presented as well as its application to face recognition. Finally, we report an automatic, real-time face recognition system.

3D model-based head tracking

This paper introduces a new approach to feature-based head tracking and pose estimation. Head tracking and pose estimation find their most important applications in motion analysis for model-based video coding. The proposed algorithm employs an underlying 3D head model, feature-based pose estimation, and texture mapping to produce accurate templates for the feature tracking. In this way, the set of templates used for the matching is constantly updated with the pose changes, allowing the algorithm to track the features over a large range of head motion without loss of precision and error accumulation. Given a rough estimate of the head scale, the initial feature identification is performed automatically and the tracking is successful over a large number of video frames. Computational complexity is also considered with the aim towards creating a real-time end-to-end model-based video coding system.