Nuri Murat Arar

Towards Convenient Calibration for Cross-Ratio Based Gaze Estimation

Eye gaze movements are considered as a salient modality for human computer interaction applications. Recently, cross-ratio (CR) based eye tracking methods have attracted increasing interest because they provide remote gaze estimation using a single uncalibrated camera. However, due to the simplification assumptions in CR-based methods, their performance is lower than the model-based approaches [8]. Several efforts have been made to improve the accuracy by compensating for the assumptions with subject specific calibration. This paper presents a CR-based automatic gaze estimation system that accurately works under natural head movements. A subject-specific calibration method based on regularized least-squares regression (LSR) is introduced for achieving higher accuracy compared to other state-of-the-art calibration methods. Experimental results also show that the proposed calibration method generalizes better when fewer calibration points are used. This enables user friendly applications with minimum calibration effort without sacrificing too much accuracy. In addition, we adaptively fuse the estimation of the point of regard (PoR) from both eyes based on the visibility of eye features. The adaptive fusion scheme reduces accuracy error by around 20% and also increases the estimation coverage under natural head movements.

Cross-pose facial expression recognition

In real world facial expression recognition (FER) applications, it is not practical for a user to enroll his/her facial expressions under different pose angles. Therefore, a desirable property of a FER system would be to allow the user to enroll his/her facial expressions under a single pose, for example frontal, and be able to recognize them under different pose angles. In this paper, we address this problem and present a method to recognize six prototypic facial expressions of an individual across different pose angles. We use Partial Least Squares to map the expressions from different poses into a common subspace, in which covariance between them is maximized. We show that PLS can be effectively used for facial expression recognition across poses by training on coupled expressions of the same identity from two different poses. This way of training lets the learned bases model the differences between expressions of different poses by excluding the effect of the identity. We have evaluated the proposed approach on the BU3DFE database and shown that it is possible to successfully recognize expressions of an individual from arbitrary viewpoints by only having his/her expressions from a single pose, for example frontal pose as the most practical case. Overall, we achieved an average recognition rate of 87.6% when using frontal images as gallery and 86.6% when considering all pose pairs.

Multiple Local Curvature Gabor Binary Patterns for Facial Action Recognition

Curvature Gabor features have recently been shown to be powerful facial texture descriptors with applications on face recognition. In this paper we introduce their use in facial action unit (AU) detection within a novel framework that combines multiple Local Curvature Gabor Binary Patterns (LCGBP) on different filter sizes and curvature degrees. The proposed system uses the distances of LCGBP histograms between neutral faces and AU containing faces combined with an AU-specific feature selection and classification process. We achieve 98.6% overall accuracy in our tests with the extended Cohn-Kanade database, which is higher than achieved previously by any state-of-the-art method.

Robust gaze estimation based on adaptive fusion of multiple cameras

Gaze movements play a crucial role in human-computer interaction (HCI) applications. Recently, gaze tracking systems with a wide variety of applications have attracted much interest by the industry as well as the scientific community. The state-of-the-art gaze trackers are mostly non-intrusive and report high estimation accuracies. However, they require complex setups such as camera and geometric calibration in addition to subject-specific calibration. In this paper, we introduce a multi-camera gaze estimation system which requires less effort for the users in terms of the system setup and calibration. The system is based on an adaptive fusion of multiple independent camera systems in which the gaze estimation relies on simple cross-ratio (CR) geometry. Experimental results conducted on real data show that the proposed system achieves a significant accuracy improvement, by around 25%, over the traditional CR-based single camera systems through the novel adaptive multi-camera fusion scheme. The real-time system achieves <;0.9° accuracy error with very few calibration data (5 points) under natural head movements, which is competitive with more complex systems. Hence, the proposed system enables fast and user-friendly gaze tracking with minimum user effort without sacrificing too much accuracy.

Selection and combination of local Gabor classifiers for robust face verification

Gabor features have been extensively used for facial image analysis due to their powerful representation capabilities. This paper focuses on selecting and combining multiple Gabor classifiers that are trained on, for example, different scales and local regions. The system exploits curvature Gabor features in addition to conventional Gabor features. Final classifier is obtained by combining selected classifiers using Sequential Forward Floating Search-based selection mechanism. In addition, we combine classifiers trained on different local representations at score-level by learning the weights with partial least square regression. The system is evaluated on Face Recognition Grand Challenge (FRGC) version 2.0 Experiment 4. The proposed system achieves 94.16% verification rate @ 0.1% FAR, which is the highest accuracy reported on this experiment so far in the literature.

A Regression-Based User Calibration Framework for Real-Time Gaze Estimation

Eye movements play a very significant role in human–computer interaction (HCI) as they are natural and fast, and contain important cues for human cognitive state and visual attention. Over the last two decades, many techniques have been proposed to accurately estimate the gaze. Among these, video-based remote eye trackers have attracted much interest, since they enable nonintrusive gaze estimation. To achieve high estimation accuracies for remote systems, user calibration is inevitable in order to compensate for the estimation bias caused by person-specific eye parameters. Although several explicit and implicit user calibration methods have been proposed to ease the calibration burden, the procedure is still cumbersome and needs further improvement. In this paper, we present a comprehensive analysis of regression-based user calibration techniques. We propose a novel weighted least squares regression-based user calibration method together with a real-time cross-ratio based gaze estimation framework. The proposed system enables to obtain high estimation accuracy with minimum user effort, which leads to user-friendly HCI applications. Experimental results conducted on both simulations and user experiments show that our framework achieves a significant performance improvement over the state-of-the-art user calibration methods when only a few points are available for the calibration.

Estimating fusion weights of a multi-camera eye tracking system by leveraging user calibration data

Cross-ratio (CR)-based eye tracking has been attracting much interest due to its simple setup, yet its accuracy is lower than that of the model-based approaches. In order to improve the estimation accuracy, a multi-camera setup can be exploited rather than the traditional single camera systems. The overall gaze point can be computed by fusion of available gaze information from all cameras. This paper presents a real-time multi-camera eye tracking system in which the estimation of gaze relies on simple CR geometry. A novel weighted fusion method is proposed, which leverages the user calibration data to learn the fusion weights. Experimental results conducted on real data show that the proposed method achieves a significant accuracy improvement over single camera systems. The real-time system achieves 0.82° of visual angle accuracy error with very few calibration data (5 points) under natural head movements, which is competitive with more complex model-based systems.

Open-set face recognition system

This work introduces a real-time video-based open-set face recognition system. The system has been developed for the identification of people who stand in front of an interactive screen to communicate with a virtual application. The system uses Discrete Cosine Transform (DCT) features obtained from non-overlapping 20 blocks, and Support Vector Machines (SVM) based verifiers are employed for the classification. To evaluate the systems performance in this application scenario, a face database has been collected in front of the interactive screen. The results on the collected database show that the developed system can operate reliably under real-world conditions.

Face recognition using curvature Gabor features

This paper introduces a homogeneous Gabor feature based face recognition approach under uncontrolled conditions such as unexpected illumination changes, pose changes, blurring and facial expression changes. The system uses curvature Gabor features instead of conventional Gabor features, and the classifiers are obtained by applying PCLDA to the selected features. By combining some of the obtained classifiers using different fusion methods, good verification accuracies are achieved with low computational complexity. The system is tested on FRGC version 2.0 database, and it achieves 93.11% verification rate.

REAL-TIME FACE SWAPPING IN VIDEO SEQUENCES - Magic Mirror

Magic Mirror is a face swapping tool that replaces the users face with a selected famous persons face in the database. System interacts with the user via a user interface which enables the selection of the replacement face and directly reflects the changed appearance. First, we apply a face detection mechanism to locate the face in the frame coming from the capturing device. Then, we feed the detection result to an active appearance model to get the exact shape of the face. By using extracted information, we replace the users face with selected target face.We display output after some post processing for color and lighting adjustments.