Neslihan Kose

KinectFaceDB: A Kinect Database for Face Recognition

The recent success of emerging RGB-D cameras such as the Kinect sensor depicts a broad prospect of 3-D data-based computer applications. However, due to the lack of a standard testing database, it is difficult to evaluate how the face recognition technology can benefit from this up-to-date imaging sensor. In order to establish the connection between the Kinect and face recognition research, in this paper, we present the first publicly available face database (i.e., KinectFaceDB1) based on the Kinect sensor. The database consists of different data modalities (well-aligned and processed 2-D, 2.5-D, 3-D, and video-based face data) and multiple facial variations. We conducted benchmark evaluations on the proposed database using standard face recognition methods, and demonstrated the gain in performance when integrating the depth data with the RGB data via score-level fusion. We also compared the 3-D images of Kinect (from the KinectFaceDB) with the traditional high-quality 3-D scans (from the FRGC database) in the context of face biometrics, which reveals the imperative needs of the proposed database for face recognition research.

Countermeasure for the protection of face recognition systems against mask attacks

There are several types of spoofing attacks to face recognition systems such as photograph, video or mask attacks. Recent studies show that face recognition systems are vulnerable to these attacks. In this paper, a countermeasure technique is proposed to protect face recognition systems against mask attacks. To the best of our knowledge, this is the first time a countermeasure is proposed to detect mask attacks. The reason for this delay is mainly due to the unavailability of public mask attacks databases. In this study, a 2D+3D face mask attacks database is used which is prepared for a research project in which the authors are all involved. The performance of the countermeasure is evaluated on both the texture images and the depth maps, separately. The results show that the proposed countermeasure gives satisfactory results using both the texture images and the depth maps. The performance of the countermeasure is observed to be slight better when the technique is applied on texture images instead of depth maps, which proves that face texture provides more information than 3D face shape characteristics using the proposed approach.

Classification of captured and recaptured images to detect photograph spoofing

In this paper, a new face anti-spoofing approach, which is based on analysis of contrast and texture characteristics of captured and recaptured images, is proposed to detect photograph spoofing. Since photo image is a recaptured image, it may show quite different contrast and texture characteristics when compared to a real face image. In a spoofing attempt, image rotation is quite possible. Therefore, in this paper, a rotation invariant local binary pattern variance (LBPV) based method is selected to be used. The approach is tested on the publicly available NUAA photo-impostor database, which is constructed under illumination and place change. The results show that the approach is competitive with other existing methods tested on the same database. It is especially useful for conditions when photos are held by hand to spoof the system. Since an LBPV based method is used, it is robust to illumination changes. It is non-intrusive and simple.

Reflectance analysis based countermeasure technique to detect face mask attacks

Face photographs, videos or masks can be used to spoof face recognition systems. Recent studies show that face recognition systems are vulnerable to these attacks. In this paper, a countermeasure technique, which analyzes the reflectance characteristics of masks and real faces, is proposed to detect mask attacks. There are limited studies on countermeasures against mask attacks. The reason for this delay is mainly due to the unavailability of public mask attack databases. In this study, a 2D+3D face mask attack database is used which is prepared for a research project in which the authors are all involved. The performance of the countermeasure is evaluated using the texture images which were captured during the acquisition of 3D scans. The results of the proposed countermeasure outperform the results of existing techniques, achieving a classification accuracy of 94.47%. In this paper, it is also proved that reflectance analysis may provide more information for the purpose of mask spoofing detection compared to texture analysis.

On the vulnerability of face recognition systems to spoofing mask attacks

There are several types of spoofing attacks to face recognition systems such as photograph, video or mask attacks. To the best of our knowledge, the impact of mask spoofing on face recognition has not been analyzed yet. The reason for this delay is mainly due to the unavailability of public mask attacks databases. In this study, we use a 2D+3D mask database which was prepared for a research project in which the authors are all involved. This paper provides new results by demonstrating the impact of mask attacks on 2D, 2.5D and 3D face recognition systems. The results show that face recognition systems are vulnerable to mask attacks, thus countermeasures have to be developed to reduce the impact of mask attacks on face recognition. The results also show that 2D texture analysis provides more information than 3D face shape analysis in order to develop a countermeasure against high-quality mask attacks.

Shape and Texture Based Countermeasure to Protect Face Recognition Systems against Mask Attacks

Photographs, videos or masks can be used to spoof face recognition systems. In this paper, a countermeasure is proposed to protect face recognition systems against 3D mask attacks. The reason for the lack of studies on countermeasures against mask attacks is mainly due to the unavailability of public databases dedicated to mask attack. In this study, a 2D+3D mask attacks database is used that is prepared for a research project in which the authors are all involved. The proposed countermeasure is based on the fusion of the information extracted from both the texture and the depth images in the mask database, and provides satisfactory results to protect recognition systems against mask attacks. Another contribution of this study is that the countermeasure is integrated to the selected baseline systems for 2D and 3D face recognition, which provides to analyze the performances of the systems with/without attacks and with/without the countermeasure.

Facial cosmetics database and impact analysis on automatic face recognition

Facial cosmetics, also called makeup, may change the appearance of a face that we could perceive. In order to contribute to studies in image processing related to facial cosmetics, a database is built which contains multiple images per person with and without applied cosmetics. Annotations provide detailed information about the amount and location of applied makeup for each picture. Furthermore, a classification approach is presented. The classification approach takes the altering effect of the applied cosmetics and the application area into account. Since automatic face recognition evolved to an important topic over the last decades and is affected by facial cosmetics, preliminary tests are done to evaluate their impact on automatic face recognition. The face as a whole as well as its most significant makeup application areas that are skin, eyes and mouth are investigated separately.

Mask spoofing in face recognition and countermeasures

Abstract In this paper, initially, the impact of mask spoofing on face recognition is analyzed. For this purpose, one baseline technique is selected for both 2D and 3D face recognition. Next, novel countermeasures, which are based on the analysis of different shape, texture and reflectance characteristics of real faces and mask faces, are proposed to detect mask spoofing. In this paper, countermeasures are developed using both 2D data (texture images) and 3D data (3D scans) available in the mask database. The results show that each of the proposed countermeasures is successful in detecting mask spoofing, and the fusion of these countermeasures further improves the results compared to using a single countermeasure. Since there is no publicly available mask database, studies on mask spoofing are limited. This paper provides significant results by proposing novel countermeasures to protect face recognition systems against mask spoofing.

Facial makeup detection technique based on texture and shape analysis

Recent studies show that the performances of face recognition systems degrade in presence of makeup on face. In this paper, a facial makeup detector is proposed to further reduce the impact of makeup in face recognition. The performance of the proposed technique is tested using three publicly available facial makeup databases. The proposed technique extracts a feature vector that captures the shape and texture characteristics of the input face. After feature extraction, two types of classifiers (i.e. SVM and Alligator) are applied for comparison purposes. In this study, we observed that both classifiers provide significant makeup detection accuracy. There are only few studies regarding facial makeup detection in the state-of-the art. The proposed technique is novel and outperforms the state-of-the art significantly.

Impact analysis of nose alterations on 2D and 3D face recognition

Numerous major challenges in face recognition, such as pose, illumination, expression and aging, have been investigated extensively. All those variations modify the texture and/or the shape of the face in a similar manner for different individuals. However, studies on alterations applied on face via plastic surgery or prosthetic make-up which can be in countless different ways and amounts, are still very limited. In this paper, we analyze how such changes on nose region affect the face recognition performances of several key techniques. For this purpose, a simulated face database is prepared using FRGC v1.0 in which nose in each sample is replaced with another randomly chosen one. Since this is a 3D database, the impact analysis is not limited to only 2D, which is one of the novelties of this study. Performance comparisons of three 2D and four 3D algorithms are provided. In addition, differently from previous works, baseline results for the original database are also reported. Hence, the impact which is purely due to the applied nose alterations can be measured. The experimental results indicate that with the introduction of alterations both modalities lose precision, especially 3D.