Vinicius Corrêa Alves Silva

Happiness and Sadness Recognition System—Preliminary Results with an Intel RealSense 3D Sensor

Systems and devices that can recognize human affects have been in development for a considerable time. Facial features are usually extracted by using video data or a Microsoft Kinect sensor. The present paper proposes an emotion recognition system that uses the recent Intel RealSense 3D sensor, whose reliability and validity in the field of emotion recognition has not yet been studied. This preliminary work focus on happiness and sadness. The system extracts the user’s facial Action Units and head motion data. Then, it uses a Support Vector Machine to automatically classify the emotion expressed by the user. The results point out the adequacy of Intel RealSense for facial features extraction in emotion recognition systems as well as the importance of determining head motion when recognizing sadness.

Mirroring and recognizing emotions through facial expressions for a RoboKind platform

The face embodies a large portion of the human emotionally expressive behaviour. Moreover, facial expressions are used to display emotional states and to manage interactions, being one of the most important channels of non-verbal communication. Although the process of recognition and displaying emotions is an easy task for the majority of humans, it is a very difficult task for individuals with Autism Spectrum Disorders (ASD). The present paper is a summary of a work developed under a Master Thesis on Industrial Electronics Engineering and Computers from the University of Minho, Portugal. The main goal of the work was the development and application of interactive and assistive technologies to support and promote new adaptive teaching/learning approaches for children with ASD. Therefore, it was proposed a system that uses the recent Intel RealSense 3D sensor to promote imitation and recognition of facial expressions, using a RoboKind Zeno R50 robot (ZECA) as a mediator in social activities. The system was first validated in the research laboratory and then tested in school environment with typically developing children and children with ASD.

iBoccia: monitoring elderly while playing Boccia gameplay

We would like to acknowledge the financial support obtained from North Portugal Regional Operational Programme (NORTE 2020), Portugal 2020 and the European Regional Development Fund (ERDF) from European Union through the project Symbiotic technology for societal efficiency gains: Deus ex Machina (DEM), NORTE-01-0145-FEDER-000026.

Imitate Me!—Preliminary Tests on an Upper Members Gestures Recognition System

This work describes the design, implementation, and preliminary tests of a system that uses a humanoid robot to mimic non-standard upper members gestures of a human body. The final goal is to use the robot as a mediator in motor imitation activities with children with special needs, either cognitive or motor impairments. A Kinect sensor and the humanoid robot ZECA (a Zeno R-50 robot from Hanson RoboKind) are used to identify and mimic upper members gestures. The system allows direct control of the humanoid robot by the user. The proposed system was tested in laboratory environment with adults with typical development. Furthermore, the system was tested with three children between 4 and 12 years old with motor and cognitive difficulties in a clinical-like environment. The main goal of these preliminary tests was to detect the constraints of the system.

Real-time emotions recognition system

This paper presents the experimental setup and methodology for a real-time emotions recognition system, based on the recent Intel RealSense 3D sensor, to identify six emotions: happiness, sadness, anger, surprise, fear, and neutral. The process includes the database construction, with 43 participants, based on facial features extraction and a multiclass Support Vector Machine classifier. The system was first tested offline using Linear kernel and Radial Basis Function (RBF) kernel. In the offline evaluation, the system performance was quantified in terms of confusion matrix, accuracy, sensitivity, specificity, Area Under the Curve, and Mathews Correlation Coefficient metrics. The RBF kernel achieved the best performance, with an average accuracy of 93.6%. Then, the real-time system was evaluated in a laboratorial setup, achieving an overall accuracy of 88%. The time required for the system to perform facial expression recognition efficiently is 1–3ms. The results, obtained by simulation and experimentally, point out that the present system can recognize facial expressions accurately.

Mirroring emotion system - on-line synthesizing facial expressions on a robot face

Social skills are an important issue throughout human life. Therefore, systems that can synthesize emotions, for example virtual characters (avatars) and robotic platforms, are gaining special attention in the literature. In particular, those systems may be important tools in order to promote social and emotional competences in children (or adults) that have some communication/interaction impairments. The present paper proposes a mirroring emotion system that uses the recent Intel RealSense 3D sensor along with a humanoid robot. The system extracts the users facial Action Units (AUs) and head motion data. Then, it sends the information to the robot allowing on-line imitation. The first tests were conducted in a laboratorial environment using the software FaceReader in order to verify its correct functioning. Next, a perceptual study was performed to verify the similarity between the expressions of a performer and those of the robot using a quiz distributed to 59 respondents. Finally, the system was evaluated with typically developing children with 6 to 9 years old. The robot mimicked the childrens emotional facial expressions. The results point out that the present system can on-line and accurately map facial expressions of a user onto the robot.

PlayCube: Designing a Tangible Playware Module for Human-Robot Interaction

In general, humans can express their intents effortlessly. On the contrary, individuals with Autism Spectrum Disorder (ASD) present impairments in this area. Researchers are employing different technological strategies in order to improve the emotion recognition skills of individuals with ASD. Among those technological solutions, the use of Objects based on Playware Technology (OPT) in context of serious games is getting increasing attention. Following this trend, the present work proposes the development of an OPT module to be used as an add-on to the human-robot interaction with children with ASD in emotion recognition activities. To evaluate the proposed approach, usability tests with typically developing children in a school environment were conducted. Overall, the different evaluations allow estimating how the children interacted with the OPT.

Building a Hybrid Approach for a Game Scenario Using a Tangible Interface in Human Robot Interaction

Understanding others intention can be a very difficult task for some individuals, in particular, individuals with Autism Spectrum Disorder (ASD). ASD is characterized by difficulties in social communication and restricted patterns of behaviour. In order to mitigate the emotion recognition impairments that individuals with ASD usually present, researchers are employing different technological strategies. Among those technological solutions, the use of assistive robots and Objects based on Playware Technology (OPT) in context of serious games are getting more attention. Following this trend, the present work targets a novel hybrid approach using a humanoid robot and one OPT. The proposed approach consists of a humanoid robot capable of displaying social behaviours, particularly facial expressions, and an OPT called PlayCube. The system was designed for emotion recognition activities with children with ASD. To evaluate the proposed approach, two pilot studies were performed: one with typically developing children and another with children with ASD. Overall, the different evaluations demonstrated the possible positive outcomes that this child-OPT-robot interaction can produce.

An Application to Promote Emotional Skills in Children with Autism Spectrum Disorders

This paper presents an approach regarding the use of a serious game with a playware object to improve the development of emotional skills in children with Autism Spectrum Disorder (ASD). The playware object is an interactive way for the user to play the game. It acts as the game controller, has six buttons, each displaying an emoji with a different facial expression, and communicates wirelessly with the android device through Bluetooth. For this purpose, the six facial expressions tested are happiness, sadness, fear, anger, surprise and a neutral/normal, which were implemented in three different game activities: imitation, recognition, and storytelling. The avatars used in the game to represent these facial expressions were first validated through an on-line questionnaire (with 114 answers) with a mean success rate of 96.2%. In order to assess the usability of the game and the playware object, a test was performed with six typically developing children, with 94.4% answer accuracy. At last, the recognition activity was tested with six children with ASD during three/four sessions. Due to the small group test and the short number of sessions, the goal was to test the acceptance of the game rather than the users’ improvement in the activity. It is worth referring that both the serious game and the playware object had a high level of approval from the children and they expressed their interest during the activities. With this preliminary study its intended to contribute to the development of pedagogical resources to be used by professionals and families in the support of children with ASD.

Building a Behaviour Architecture: An Approach for Promoting Human-Robot Interaction

Human distraction behaviour is a paramount subject to take into account. Several woks in the literature try to tackle this topic mostly in the automotive industry. Following this trend, the present work proposes a system to detect the patterns of distraction/attention during an interaction activity between a human and a robot. The goal is to analyse selected patterns of distraction, such as eye gaze, head pose, blinking rate, among others, and adapting the robot behaviour, consequently promoting a more fluid interaction. A behavioural state machine that takes into account the engagement and the performance of the user in the activity is proposed.