José Augusto Prado

Visuo-auditory Multimodal Emotional Structure to Improve Human-Robot-Interaction

We propose an approach to analyze and synthesize a set of human facial and vocal expressions, and then use the classified expressions to decide the robot’s response in a human-robot-interaction. During a human-to-human conversation, a person senses the interlocutor’s face and voice, perceives her/his emotional expressions, and processes this information in order to decide which response to give. Moreover, observed emotions are taken into account and the response may be aggressive, funny (henceforth meaning humorous) or just neutral according to not only the observed emotions, but also the personality of the person. The purpose of our proposed structure is to endow robots with the capability to model human emotions, and thus several subproblems need to be solved: feature extraction, classification, decision and synthesis. In the proposed approach we integrate two classifiers for emotion recognition from audio and video, and then use a new method for fusion with the social behavior profile. To keep the person engaged in the interaction, after each iterance of analysis, the robot synthesizes human voice with both lips synchronization and facial expressions. The social behavior profile conducts the personality of the robot. The structure and work flow of the synthesis and decision are addressed, and the Bayesian networks are discussed. We also studied how to analyze and synthesize the emotion from the facial expression and vocal expression. A new probabilistic structure that enables a higher level of interaction between a human and a robot is proposed.

A real time and robust facial expression recognition and imitation approach for affective human-robot interaction using Gabor filtering

Facial expressions are a rich source of communicative information about human behavior and emotion. This paper presents a real-time system for recognition and imitation of facial expressions in the context of affective Human Robot Interaction. The proposed method achieves a fast and robust facial feature extraction based on consecutively applying filters to the gradient image. An efficient Gabor filter is used, along with a set of morphological and convolutional filters to reduce the noise and the light dependence of the image acquired by the robot. Then, a set of invariant edge-based features are extracted and used as input to a Dynamic Bayesian Network classifier in order to estimate a human emotion. The output of this classifier updates a geometric robotic head model, which is used as a bridge between the human expressiveness and the robotic head. Experimental results demonstrate the accuracy and robustness of the proposed approach compared to similar systems.

Bayesian sensor fusion for land-mine detection using a dual-sensor hand-held device

This work presents a methodology and practical implementation of sensor fusion for land-mine detection using a novel multi-sensor hand-held device composed by a triple coil metal detector and a gas sensor. The proposed approach consists on merging data from both sensors in order to reduce the false alarm rate, particularly by using odor information. A Bayesian approach is proposed for the sensor fusion. Results show a false alarm rate of 1.4 to 1, a mine detection rate of 100% and a mine localization mean absolute error of 3 cm. Furthermore the resulting mine presence probability distribution maps represent an important visualization tool for mine clearance hand-held device users.

An autonomous all terrain robotic system for field demining missions

Mines deployed in post-war countries pose severe threats to civilians and hamper the reconstruction effort in war hit societies. In the scope of the EU FP7 TIRAMISU Project, a toolbox for humanitarian demining missions is being developed by the consortium members. In this article we present the FSR Husky, an affordable, lightweight and autonomous all terrain robotic system, developed to assist human demining operation teams. Intended to be easily deployable on the field, our robotic solution has the ultimate goal of keeping humans away from the threat, safeguarding their lives. A detailed description of the modular robotic system architecture is presented, and several real world experiments are carried out to validate the robots functionalities and illustrate continuous work in progress on minefield coverage, mine detection, outdoor localization, navigation, and environment perception. Proposal of a new affordable, lightweight and autonomous all-terrain robot for demining operations.Complete description of the mechanical system and design of the modular robotic architecture.Validation of the robots intelligent capabilities in outdoor experiments.Work in progress and lessons learned for minefield coverage, and mine detection.

Human Robot interaction studies on laban human movement analysis and dynamic background segmentation

Human movement analysis through vision sensing systems is an important subject regarding Human-Robot interaction. This is a growing area of research, with wide range of applications fields. The ability to recognize human actions using passive sensing modalities, is a decisive factor for machine interaction. In mobile platforms, image processing is regarded as a problem, due to constant changes. We propose an approach, based on Horopter technique, to extract Regions Of Interest (ROI) delimiting human contours. This fact will allow tracking algorithms to provide faster and accurate responses to human feature extraction. The key features are head and both hand positions, that will be tracked within image context. Posterior to feature acquisition, they will be contextualized within a technique, Laban Movement Analysis (LMA) and will be used to provide sets of classifiers. The implementation of the LMA technique will be based on Bayesian Networks. We will use these Bayesian classifiers to label/classify human emotion within the context of expressive movements. Compared to full image tracking, results improved with the implemented approach, the horopter and consequently so did classification results.

Energy Efficient Area Coverage for an Autonomous Demining Robot

This work studies different coverage approaches with a mobile robot equipped with a landmine detecting sensor attached to an actuator arm. Different coverage techniques were experiment in this work and the cost benefit was analyzed in terms of energy consumption. The problem of optimising the combined motion of a mobile platform with an arm was addressed. The feasibility and effectiveness of both algorithms are demonstrated by simulation results for coverage work of target area.

Robotic implementation of biological Bayesian models for visuo-inertial image stabilization and gaze control

Robotic implementations of gaze control and image stabilization have been previously proposed, that rely on fusing inertial and visual sensing modalities. Human and biological system also combine the two sensing modalities for the same goal. In this work we build upon these previous results and, with the contribution of psychophysical studies, attempt a more bio-inspired approach to the robotic implementation. Since Bayesian models have been successfully used to explain psychophysical experimental findings, we propose a robotic implementation using Bayesian inference.

Multi-sensor and Multi-platform Data Fusion for Buried Objects Detection and Localization

This paper describes a multi-sensor data-fusion approach to detect and localize landmines and unexploded ordnances (UXO) in a field using multiple mobile sensor carriers covering the area of interest. A procedure for high accuracy geo-referencing the field-data acquired by multiple platforms is proposed. Later, the geo-referenced sensor data is fused by a multi-step algorithm consisting on the detection of heterogeneities corresponding to objects followed by the classification of those objects through a decision-level fusion. The proposed approach was tested and validated in a military field containing more than twenty different objects buried on the ground, using two different metal detector (MD) arrays and one ground penetrating radar (GPR) array. Although the preliminary results can be considered encouraging, since it was possible to detect and classify all the buried objects, further tests are necessary in terms of classification, since the classification algorithm was trained with data collected previously from the same soil.

Comparing Bayesian Networks to Classify Facial Expressions

In this paper are presented two distinct Bayesian networks to analyse human beings’ facial expressions. Both classifiers are completely defined: structure of the networks, belief variables and respective events, likelihoods, initial priors and procedure to change dynamically priors. The performance (relatively to the convergence) of the two approaches is compared. For both networks, the classification is done associating the facial expression to the probabilities of five emotional states: anger, fear, happy, sad and neutral. A justification for the usage of this set is presented: it is based in emotional states presented by human beings during social relationships. Classifiers as these described here can be used in Human Robot Interation. We believe that this interaction shall be done in a similar way of that used by human beings to communicate between them and, after all, facial expressions is one of the main non-verbal means of communication used by human.

The 2016 Humanitarian Robotics and Automation Technology Challenge [Competitions]

Presents information on the 2016 Humanitarian Robotics and Automation Technology Challenge.