Filareti Tsalakanidou

Use of depth and colour eigenfaces for face recognition

In the present paper a face recognition technique is developed based on depth and colour information. The main objective of the paper is to evaluate three different approaches (colour, depth, combination of colour and depth) for face recognition and quantify the contribution of depth. The proposed face recognition technique is based on the implementation of the principal component analysis algorithm and the extraction of depth and colour eigenfaces. Experimental results show significant gains attained with the addition of depth information.

Real-time 2D+3D facial action and expression recognition

This paper presents a completely automated facial action and facial expression recognition system using 2D+3D images recorded in real-time by a structured light sensor. It is based on local feature tracking and rule-based classification of geometric, appearance and surface curvature measurements. Several experiments conducted under relatively non-controlled conditions demonstrate the accuracy and robustness of the approach.

Face localization and authentication using color and depth images

This work presents a complete face authentication system integrating both two-dimensional (color or intensity) and three-dimensional (3-D) range data, based on a low-cost 3-D sensor, capable of real-time acquisition of 3-D and color images. Novel algorithms are proposed that exploit depth information to achieve robust face detection and localization under conditions of background clutter, occlusion, face pose alteration, and harsh illumination. The well-known embedded hidden Markov model technique for face authentication is applied to depth maps and color images. To cope with pose and illumination variations, the enrichment of face databases with synthetically generated views is proposed. The performance of the proposed authentication scheme is tested thoroughly on two distinct face databases of significant size. Experimental results demonstrate significant gains resulting from the combined use of depth and color or intensity information.

A 3D face and hand biometric system for robust user-friendly authentication

A complete authentication system based on fusion of 3D face and hand biometrics is presented and evaluated in this paper. The system relies on a low cost real-time sensor, which can simultaneously acquire a pair of depth and color images of the scene. By combining 2D and 3D facial and hand geometry features, we are able to provide highly reliable user authentication robust to appearance and environmental variations. The design of the proposed system addresses two basic requirements of biometric technologies: dependable performance under real-world conditions along with user convenience. Experimental evaluation on an extensive database recorded in a real working environment demonstrates the superiority of the proposed multimodal scheme against unimodal classifiers in the presence of numerous appearance and environmental variations, thus making the proposed system an ideal solution for a wide range of real-world applications, from high-security to personalization of services and attendance control.

Real-time acquisition of depth and color images using structured light and its application to 3D face recognition

In this paper, a novel real-time 3D and color sensor for the mid-distance range (0.1-3m) based on color-encoded structured light is presented. The sensor is integrated using low-cost of-the-shelf components and allows the combination of 2D and 3D image processing algorithms, since it provides a 2D color image of the scene in addition to the range data. Its design is focused on enabling the system to operate reliably in real-world scenarios, i.e. in uncontrolled environments and with arbitrary scenes. To that end, novel approaches for encoding and recognizing the projected light are used, which make the system practically independent of intrinsic object colors and minimize the influence of the ambient light conditions. The system was designed to assist and complement a face authentication system integrating both 2D and 3D images. Depth information is used for robust face detection, localization and 3D pose estimation. To cope with illumination and pose variations, 3D information is used for the normalization of the input images. The performance and robustness of the proposed system is tested on a face database recorded in conditions similar to those encountered in real-world applications.

Integration of 2D and 3D images for enhanced face authentication

This work presents a complete face authentication system integrating 2D intensity and 3D range data, based on a low-cost, real-time structured light sensor. Novel algorithms are proposed that exploit depth data to achieve robust face detection, localization and authentication under conditions of background clutter, occlusion, face pose alteration and harsh illumination. The well known embedded hidden Markov model technique for face authentication is applied to depth maps. A method for the enrichment of face databases with synthetically generated views depicting various head poses and illumination conditions is proposed. The performance of the proposed system is tested on an extensive face database of 3,000 images. Experimental results demonstrate significant gains resulting from the combined use of depth and intensity.

A face and gesture recognition system based on an active stereo sensor

The paper presents several novel 3D image analysis algorithms, applied towards the segmentation and modeling of faces and hands. These are subsequently used to build a face-based authentication system and a system for human-computer interaction based on static and dynamic gestures. The system relies on an active stereo sensor that uses a structured light approach to obtain 3D information. In this paper we demonstrate how the use of 3D information may significantly improve the efficiency of traditional face and gesture recognition techniques that use 2D images only.

Robust facial action recognition from real-time 3D streams

This paper presents a completely automated facial action and facial expression recognition system using 2D + 3D images recorded in real-time by a structured light sensor. It is based on local feature tracking and rule-based classification of geometric, appearance and surface curvature measurements. Good performance is achieved under relatively non-controlled conditions.

Flame Detection for Video-based Early Fire Warning Systems and 3D Visualization of Fire Propagation

Early and accurate detection and localization of flame is an essential requirement of modern early fire warning systems. Video-based systems can be used for this purpose; however, flame detection remains a challenging issue due to the fact that many natural objects have similar characteristics with fire. In this paper, we present a new algorithm for video based flame detection, which employs various spatio-temporal features such as colour probability, contour irregularity, spatial energy, flickering and spatio-temporal energy. Various background subtraction algorithms are tested and comparative results in terms of computational efficiency and accuracy are presented. Experimental results with two classification methods show that the proposed methodology provides high fire detection rates with a reasonable false alarm ratio. Finally, a 3D visualization tool for the estimation of the fire propagation is outlined and simulation results are presented and discussed.

A Secure and Privacy Friendly 2D+3D Face Authentication System Robust Under Pose and Illumation Variation

An end-to-end face authentication system integrating 2D color images and depth data is presented in this paper, based on a low-cost sensor capable of real-time acquisition of 3D images and associated color images. Depth data is used for robust face detection, localization and 3D pose estimation, as well as for compensating pose and illumination variations of facial images prior to classification. The performance of the proposed system is tested on an extended face database recorded in real-world conditions, while a complete security and privacy analysis is conducted in order to ensure that all necessary countermeasures are built into the system.