Talles Marcelo G. de A. Barbosa

Emopad: An Affective Gamepad

Affective Systems have been used in different applications, such as stress monitoring in aircraft seats and managing sensitivity in autism spectrum disorder. Although many scientific progresses have been made there are many computational challenges to be overcome in order to embedded affectivity into traditional user interfaces. For example, context-sensitive algorithms, low-complexity pattern recognition models and hardware customizations are requirements to support the simplification of user’s experience becoming more intuitive, transparent and less obstructive. In this paper an affective gamepad is presented. This acquisition system has been developed to improve user’s biofeedback when they are playing games on Microsoft Xbox or Sony PlayStation. The preliminary version of Emopad is able to capture Galvanic Skin Response (GSR), Temperature, Force, Heart Rate (HR) and its variability (HRV) by photoplethysmography (PPG) sensor while complementary algorithms are executed to filter noise, recognize interests patterns and classify events related to user’s emotional states. All processing phases are embedded into Emopad, and they explore microcontroller’s dynamic power management. Emopad has been developed to deal with energy-efficient and platform-independent requirements. Concerning to this paper’s aim, the acquisition systems development is highlighted. Also, the sliding window-based algorithm is presented and evaluated. It has been applied after sampling in GSR, Force, Temperature and HR signals to detect events related to emotional responses. The success of affective gamepads can lead to a paradigm shift, because traditional consoles can be configured to work as Point-of-Care technologies. Consequently, they can receive, process and transmit physiological data and events related to clinical conditions about their users.

Developing an affective Point-of-Care technology

Mobile intelligent clinical monitoring systems provide mobility and out of hospital monitoring. It can be used in the follow-up of high-risk patients in out of hospital situations and to monitor “healthy” persons to prevent medical events. The inherent characteristics of local diagnosis and actuation permit an improvement and advance in the diagnosis and emergency decision support. Additionally, Affective Systems have been used in different applications, such as stress monitoring in aircraft seats and managing sensitivity in autism spectrum disorder. Although many scientific progresses have been made there are many computational challenges in order to embedded affectivity into traditional user interfaces. For example, context-sensitive algorithms, low-complexity pattern recognition models and hardware customizations are requirements to support the simplification of users experience becoming more intuitive, transparent and less obstructive. In this paper a multiparametric affective monitor is presented. The Emopad acquisition system has been developed to analyze users biofeedback particularly when they are playing games. It is able to capture Galvanic Skin Response (GSR), Temperature, Force, Heart Rate (HR) and its variability (HRV) while complementary algorithms are executed to recognize events related to users emotional states. Also, in this paper a sliding window-based algorithm is presented and evaluated to detect specific events related to emotional responses. The success of multiparametric affective monitors can lead to a paradigm shift, establishing new scenarios for the Point-of-Care technologies applications.

HRVCam: A software for real-time feedback of heart rate and HRV

The heart rate (HR) and its variability, known as Heart Rate Variability (HRV), are indispensable measurements for cardiorespiratory monitoring, recognition and quantification of emotions, detection of abnormalities, and heart disease control. In general, the acquisition systems for HR and HRV require a contact area for sensors installation and positioning, creating restrictions and/or obstructions on users movements. This paper proposes a noninvasive and noncontact technique for HR and HRV acquisition using a camera. The purposed technique consists in the automatic detection of the users face and utilization of an Independent Component Analysis (ICA) algorithm to separate the necessary signals to determine the HR and HRV. The experiments have shown more than 95% of similarity between the results of the proposed software (HRVCam) in comparison to the results of the photoplethysmography sensor (PPG).

Affective Embedded Systems: a Requirement Engineering Approach

This paper proposes an embedded system development process able to deal with affectivity requirements. This process includes W3C tools, such as EmotionML and SysML, in order to specify cognition and affectivity as system requirements. It combines three different integration models: Sparxs (1) and Wolfs System Development Processes (4) and Russells Intelligent Agent Specification (5). In order to validate our model, a pet robot case study was built. This application can be useful in many homecare applications, such as assistive robots for capturing attention of autistic children and robot therapy for elders. Moreover, it represents a futuristic embedded system design space for applying affective computing.

Including affectivity requirements in embedded systems

This paper proposes an embedded system development process capable to deal with affectivity requirements. It is supported by W3C tools, such as EmotionML and SysML. These tools should be used to specify cognition and affectivity as system requirements. The proposed process is based on three different integration models: Sparxs [1], Wolfs System Development Process [4] and Russells Intelligent Agent Specification [5]. In order to validate this proposal a pet robot case study has been built. This application can be useful in many non-critical domestic applications, such as assistive robots for capturing attention of autistic children and robot therapy for elders. Moreover, it represents a futuristic embedded system design space for applying affective computing.

Building smart sensor nodes according to IEEE 1451.3 standard

A Body Sensor Network (BSN) application requires many software and hardware adaptations to support correctly data exchanges between different sensor node architectures. However, these customizations demand extra time, cost and components. This paper introduces a simple development process in order to customize off-the-shelf BSN sensor nodes according to Transducer Bus Interface Modules (TBIM) standard. IEEE 1451.3 offers technical solutions for interfacing multiple and physically separated transducer allowing self-identification, self-configuration, plug and play and hot swapping capabilities. These are important requirements relating to most BSN applications.

Embedding a Neural Network into WSN furniture

Wireless Sensor Networks (WSN) is an emerging technology that is developed with a large number of useful applications. On the other hand, Artificial Neural Networks (ANN) have found many successful applications in nonlinear system and control, digital communication, pattern recognition, pattern classification, etc. There are many similarities between WSN and ANN. For example, the sensor node itself can be seen as a neuron since the WSN application show characteristics such as distributed processing, massive parallelism, adaptively, inherent contextual information processing, fault tolerance and low computation. This paper examines the possibility of embedding ANN and WSN into a Smart Table. Prototypal results have shown that ANN models are good candidates for using it deployed into low cost System-on-a-Chip (SoC).

Static gestures recognition for Brazilian Sign Language with kinect sensor

Automatic recognition of sign languages to help hearing impaired people is an area that has been explored for quite some time. However, this is still a practical problem due to the complexity involved making it a big challenge. The use of devices, such as the Kinect sensor, has been shown to be promising in gesture recognition. Therefore, is proposes an application for users of the Brazilian Sign Language, which brings a main feature of voice generation based on static hand gestures through Kinect. The preliminary results obtained through testing had an accuracy reaching up to 89% the gesture recognition process.

A hybrid intelligent Body Sensor Networks model

Body Sensor Networks (BSN) have gained interest in recent years from researches. Several promising prototypes are enabling many healthcare services. Despite of technological developments in sensing and monitoring devices, some issues related to BSN have still to be investigated. For example, information routing requires establishment of multi-hop paths which can be done considering the amount of consumed energy at each instant. Previous works addressed this issue by means of blind-routing algorithms such as Dijkstra. In this paper a Genetic Algorithm (GA) is evaluated in order to improve the system scalability. Another important aspect is related to autonomous processing capability expected of each sensor node. This work proposes and evaluates Self Organizing Map (SOM) as an unsupervised learning solution. Thus, the main purpose of this paper is to present a hybrid model combining GA and SOM in order to support intelligent BSNs. At the end of this paper a case study was evaluated in order to support a thermal biofeedback application.