Pierluigi Carcagnì

Facial expression recognition and histograms of oriented gradients: a comprehensive study

Automatic facial expression recognition (FER) is a topic of growing interest mainly due to the rapid spread of assistive technology applications, as human–robot interaction, where a robust emotional awareness is a key point to best accomplish the assistive task. This paper proposes a comprehensive study on the application of histogram of oriented gradients (HOG) descriptor in the FER problem, highlighting as this powerful technique could be effectively exploited for this purpose. In particular, this paper highlights that a proper set of the HOG parameters can make this descriptor one of the most suitable to characterize facial expression peculiarities. A large experimental session, that can be divided into three different phases, was carried out exploiting a consolidated algorithmic pipeline. The first experimental phase was aimed at proving the suitability of the HOG descriptor to characterize facial expression traits and, to do this, a successful comparison with most commonly used FER frameworks was carried out. In the second experimental phase, different publicly available facial datasets were used to test the system on images acquired in different conditions (e.g. image resolution, lighting conditions, etc.). As a final phase, a test on continuous data streams was carried out on-line in order to validate the system in real-world operating conditions that simulated a real-time human–machine interaction.

Improved Performance in Facial Expression Recognition Using 32 Geometric Features

Automatic facial expression recognition is one of the most interesting problem as it impacts on important applications in human-computer interaction area. Many applications in this field require real-time performance but not all the approach are suitable to satisfy this requirement. Geometrical features are usually the most light in terms of computational load but sometimes they exploits a huge number of features and do not cover all the possible geometrical aspect. In order to face up this problem, we propose an automatic pipeline for facial expression recognition that exploits a new set of 32 geometric facial features from a single face side covering a wide set of geometrical peculiarities. As a results, the proposed approach showed a facial expression recognition accuracy of 95,46% with a six-class expression set and an accuracy of 94,24% with a seven-class expression set.

Automatic Emotion Recognition in Robot-Children Interaction for ASD Treatment

Autism Spectrum Disorders (ASD) are a group of lifelong disabilities that affect peoples communication and understanding social cues. The state of the art witnesses how technology, and in particular robotics, may offer promising tools to strengthen the research and therapy of ASD. This work represents the first attempt to use machine-learning strategies during robot-ASD children interactions, in terms of facial expression imitation, making possible an objective evaluation of childrens behaviours and then giving the possibility to introduce a metric about the effectiveness of the therapy. In particular, the work focuses on the basic emotion recognition skills. In addition to the aforementioned applicative innovations this work contributes also to introduce a facial expression recognition (FER) engine that automatically detects and tracks the childs face and then recognize emotions on the basis of a machine learning pipeline based on HOG descriptor and Support Vector Machines. Two different experimental sessions were carried out: the first one tested the FER engine on publicly available datasets demonstrating that the proposed pipeline outperforms the existing strategies in terms of recognition accuracy. The second one involved ASD children and it was a preliminary exploration of how the introduction of the FER engine in the therapeutic protocol can be effectively used to monitor childrens behaviours.

Visual Interaction Including Biometrics Information for a Socially Assistive Robotic Platform

This work introduces biometrics as a way to improve human-robot interaction. In particular, gender and age estimation algorithms are used to provide awareness of the user biometrics to a humanoid robot (Aldebaran NAO), in order to properly react with a specific gender/age behavior. The system can also manage multiple persons at the same time, recognizing the age and gender of each participant. All the estimation algorithms employed have been validated through a k-fold test and successively practically tested in a real human-robot interaction environment, allowing for a better natural interaction. Our system is able to work at a frame rate of 13 fps with 640\(\times \)480 images taken from NAO’s embedded camera. The proposed application is well-suited for all assisted environments that consider the presence of a socially assistive robot like therapy with disable people, dementia, post-stroke rehabilitation, Alzheimer disease or autism.

Real-Time Gender Based Behavior System for Human-Robot Interaction

This work introduces a real-time system able to lead humanoid robot behavior depending on the gender of the interacting person. It exploits Aldebaran NAO humanoid robot view capabilities by applying a gender prediction algorithm based on the face analysis. The system can also manage multiple persons at the same time, recognizing if the group is composed by men, women or is a mixed one and, in the latter case, to know the exact number of males and females, customizing its response in each case. The system can allow for applications of human-robot interaction requiring an high level of realism, like rehabilitation or artificial intelligence.

Features Descriptors for Demographic Estimation: A Comparative Study

Estimation of demographic information from video sequence with people is a topic of growing interest in the last years. Indeed automatic estimation of audience statistics in digital signage as well as the human interaction in social robotic environment needs of increasingly robust algorithm for gender, race and age classification. In the present paper some of the state of the art features descriptors and sub space reduction approaches for gender, race and age group classification in video/image input are analyzed. Moreover a wide discussion about the influence of dataset distribution, balancing and cardinality is shown. The aim of our work is to investigate the best solution for each classification problem both in terms of estimation approach and dataset training. Additionally the computational problem it considered and discussed in order to contextualize the topic in a practical environment.

Assessment of deep learning for gender classification on traditional datasets

Deep Learning has becoming a popular and effective way to address a large set of issues. In particular, in computer vision, it has been exploited to get satisfying recognition performance in unconstrained conditions. However, this wild race towards even better performance in extreme conditions has overshadowed an important step i.e. the assessment of the impact of this new methodology on traditional issues on which for years the researchers had worked. This is particularly true for biometrics applications where the evaluation of deep learning has been made directly on newest large and more challencing datasets. This lead to a pure data driven evaluation that makes difficult to analyze the relationships between network configurations, learning process and experienced outcomes. This paper tries to partially fill this gap by applying a DNN for gender recognition on the MORPH dataset and evaluating how a lower cardinality of examples used for learning can bias the recognition performance.

Soft Biometrics for a Socially Assistive Robotic Platform

Abstract In thiswork, a real-time system able to automatically recognize soft-biometric traits is introduced and used to improve the capability of a humanoid robot to interact with humans. In particular the proposed system is able to estimate gender and age of humans in images acquired from the embedded camera of the robot. This knowledge allows the robot to properly react with customized behaviors related to the gender/age of the interacting individuals. The system is able to handle multiple persons in the same acquired image, recognizing the age and gender of each person in the robot’s field of view. These features make the robot particularly suitable to be used in socially assistive applications.

Multi-branch CNN for Multi-scale Age Estimation

Convolutional Neural Networks (CNNs) attracted growing interest in recent years thanks to their high generalization capabilities that are highly recommended especially for applications working in the wild context. However CNNs rely on a huge number of parameters that must be set during training sessions based on very large datasets in order to avoid over-fitting issues. As a consequence the lack in training data is one of the greatest limits for the applicability of deep networks. Another problem is represented by the fixed scale of the filter in the first convolutional layer that limits the analysis performed through the subsequent layers of the network.

UA-DETRAC 2017: Report of AVSS2017 & IWT4S Challenge on Advanced Traffic Monitoring

The rapid advances of transportation infrastructure have led to a dramatic increase in the demand for smart systems capable of monitoring traffic and street safety. Fundamental to these applications are a community-based evaluation platform and benchmark for object detection and multi-object tracking. To this end, we organize the AVSS2017 Challenge on Advanced Traffic Monitoring, in conjunction with the International Workshop on Traffic and Street Surveillance for Safety and Security (IWT4S), to evaluate the state-of-the-art object detection and multi-object tracking algorithms in the relevance of traffic surveillance. Submitted algorithms are evaluated using the large-scale UA-DETRAC benchmark and evaluation protocol. The benchmark, the evaluation toolkit and the algorithm performance are publicly available from the website http://detrac-db.rit.albany.edu.