Marco Simões

A serious game with virtual reality for travel training with Autism Spectrum Disorder

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by impairments in social interaction and repetitive patterns of behavior. This article describes the creation of a serious game that prepares individuals with ASD to use buses as a mean of transportation. Virtual reality (VR) support was added, increasing the feeling of presence and the realism of the experience, thus increasing its potential as a learning tool. The game is currently being developed using the Unity game engine and uses the Oculus Rift as virtual reality headset. Preliminary results prove the viability of the experiment and the acceptance from individuals with ASD towards the use of the VR setup. In conclusion, the project aims to understand how games and virtual reality can be used to improve the capabilities of individuals with ASD, and help them live more independently.

Neural Signals Evoked by Stimuli of Increasing Social Scene Complexity Are Detectable at the Single-Trial Level and Right Lateralized

Classification of neural signals at the single-trial level and the study of their relevance in affective and cognitive neuroscience are still in their infancy. Here we investigated the neurophysiological correlates of conditions of increasing social scene complexity using 3D human models as targets of attention, which may also be important in autism research. Challenging single-trial statistical classification of EEG neural signals was attempted for detection of oddball stimuli with increasing social scene complexity. Stimuli had an oddball structure and were as follows: 1) flashed schematic eyes, 2) simple 3D faces flashed between averted and non-averted gaze (only eye position changing), 3) simple 3D faces flashed between averted and non-averted gaze (head and eye position changing), 4) animated avatar alternated its gaze direction to the left and to the right (head and eye position), 5) environment with 4 animated avatars all of which change gaze and one of which is the target of attention. We found a late (> 300 ms) neurophysiological oddball correlate for all conditions irrespective of their complexity as assessed by repeated measures ANOVA. We attempted single-trial detection of this signal with automatic classifiers and obtained a significant balanced accuracy classification of around 79%, which is noteworthy given the amount of scene complexity. Lateralization analysis showed a specific right lateralization only for more complex realistic social scenes. In sum, complex ecological animations with social content elicit neurophysiological events which can be characterized even at the single-trial level. These signals are right lateralized. These finding paves the way for neuroscientific studies in affective neuroscience based on complex social scenes, and given the detectability at the single trial level this suggests the feasibility of brain computer interfaces that can be applied to social cognition disorders such as autism.

Connectedness and local search for bicriteria knapsack problems

This article reports an experimental study on a given structural property of connectedness of optimal solutions for two variants of the bicriteria knapsack problem. A local search algorithm that explores this property is then proposed and its performance is compared against exact algorithms in terms of running time and number of optimal solutions found. The experimental results indicate that this simple local search algorithm is able to find a representative set of optimal solutions in most of the cases, and in much less time than exact approaches.

Virtual Travel Training for Autism Spectrum Disorder: Proof-of-Concept Interventional Study

Background Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by impairments in social interaction and repetitive patterns of behavior, which can lead to deficits in adaptive behavior. In this study, a serious game was developed to train individuals with ASD for an important type of outdoor activity, which is the use of buses as a means of transportation. Objective The aim of this study was to develop a serious game that defines a “safe environment” where the players became familiar with the process of taking a bus and to validate if it could be used effectively to teach bus-taking routines and adaptive procedures to individuals with ASD. Methods In the game, players were placed in a three-dimensional city and were submitted to a set of tasks that involved taking buses to reach specific destinations. Participants with ASD (n=10) underwent between 1 to 3 training sessions. Participants with typical development (n=10) were also included in this study for comparison purposes and received 1 control session. Results We found a statistically significant increase in the measures of knowledge of the process of riding a bus, a reduction in the electrodermal activity (a metric of anxiety) measured inside the bus environments, and a high success rate of their application within the game (93.8%). Conclusions The developed game proved to be potentially useful in the context of emerging immersive virtual reality technologies, of which use in therapies and serious games is still in its infancy. Our findings suggest that serious games, using these technologies, can be used effectively in helping people with ASD become more independent in outdoor activities, specifically regarding the use of buses for transportation.

Feature analysis for correlation studies of simultaneous EEG-fMRI data: A proof of concept for neurofeedback approaches.

The identification and interpretation of facial expressions is an important feature of social cognition. This characteristic is often impaired in various neurodevelopmental disorders. Recent therapeutic approaches to intervene in social communication impairments include neurofeedback (NF). In this study, we present a NF real-time functional Magnetic Resonance Imaging (rt-fMRI), combined with electroencephalography (EEG) to train social communication skills. In this sense, we defined the right Superior Temporal Sulcus as our target region-of-interest. To analyze the correlation between the fMRI regions of interest and the EEG data, we transposed the sources located at the nearest cortical location to the target region. We extracted a set of 75 features from EEG segments and performed a correlation analysis with the brain activations extracted from rt-fMRI in the right pSTS region. The finding of significant correlations of simultaneously measured signals in distinct modalities (EEG and fMRI) is promising. Future studies should address whether the observed correlation levels between local brain activity and scalp measures are sufficient to implement NF approaches.

Specific EEG/ERP Responses to Dynamic Facial Expressions in Virtual Reality Environments

Visual event-related potentials of facial expressions (FEs) have been studied using usually static stimuli after a nonspecific black screen as a baseline. However, when studying social events, the ecology of the environment and stimuli can be a bias. Virtual reality provides a possible approach to improve ecology while keeping stimulus control.

Applying Weightless Neural Networks to a P300-Based Brain-Computer Interface

P300-based Brain Computer Interfaces (BCI) are one of the most used types of BCIs in the literature that make use of the electroencephalogram (EEG) signal to convey commands to the computer. The efficiency of such systems depends drastically on the ability of correctly identifying the P300 wave in the EEG signal. Due to high inter-subject and inter-session variability, single-subject classifiers must be trained every session. In order to achieve fast setup times of the system, only a few trials are available each session for training the classifier. In this scenario, the capacity to learn from few examples is crucial for the performance of the BCI and, therefore, the use of weightless neural networks (WNN) is promising. Despite its possible added value, there are no studies, to our knowledge, applying WNNs to P300 classification. Here we compare the performance of a WNN against the state-of-the-art algorithms when applied to a P300-based BCI for joint-attention training in autism. Our results show that the WNN performs as good as its competitors, outperforming them several times. We also perform an analysis of the WNN hyperparameters, showing that smaller memories achieve better results most of the times. This study demonstrates that the adoption of this type of classifiers might help increase the prediction accuracy of P300-based BCI systems, and should be a valid option for future studies to consider.

Functional Mapping of Inner Speech Areas: A Preliminary Study with Portuguese Speakers

Inner speech can be defined as the act of talking silently with ourselves. Several studies aimed to understand how this process is related to speech organization and language. Despite the advances, some results are still contradictory. Importantly, language dependency is scarcely studied. For this first study of inner speech for Portuguese native speakers using fMRI, we selected a confrontation naming task, consisting of 40 black and white line drawings. Five healthy participants were instructed to name in inner and in overt speech the visually presented image. fMRI data analysis considering the proposed inner speech paradigm identified several brain areas such as the left inferior frontal gyrus, including Broca’s area, supplementary motor area, precentral gyrus and left middle temporal gyrus including Wernicke’s area. Our results also show more pronounced bilateral activations during the overt speech task when compared to inner speech, suggesting that inner and overt speech activate similar areas but stronger activation can be found in the later. However, this difference stems in particular from significant activation differences in the right pre-central gyrus and middle temporal gyrus.

Inner Speech in Portuguese: Acquisition Methods, Database and First Results

In this paper, we present a database developed for studying inner speech brain related areas using functional Magnetic Resonance Imaging (fMRI) in the context of the European Portuguese. First, we addressed the type of stimuli used in inner speech studies. In this sense, considering a preliminary study using a picture naming task, we defined a corpus. The corpus was designed based on cardinal vowels, syllable, disyllabic words and sentences with structure S(ubject)V(erb)O(bject) which were balanced in syllable number (six to ten). All the words used are common words from the Portuguese lexicon and possible ambiguities were excluded. Currently, the dataset includes data from twenty healthy participants native Portuguese speakers. Preliminary, exploratory analysis on the data allowed us to identify the most relevant areas part of the inner speech network, that include inferior frontal gyrus (including Broca’s area), supplementary motor area and precentral gyrus. Ultimately, the better understanding of the inner speech mechanisms will pave way to the development of novel intervention strategies in linguistic disorders.

A NOVEL BIOMARKER OF COMPENSATORY RECRUITMENT OF FACE EMOTIONAL IMAGERY NETWORKS IN AUTISM SPECTRUM DISORDER

Imagery of facial expressions in Autism Spectrum Disorder (ASD) is likely impaired but has been very difficult to capture at a neurophysiological level. We developed an approach that allowed to directly link observation of emotional expressions and imagery in ASD, and to derive biomarkers that are able to classify abnormal imagery in ASD. To provide a handle between perception and action imagery cycles it is important to use visual stimuli exploring the dynamical nature of emotion representation. We conducted a case-control study providing a link between both visualization and mental imagery of dynamic facial expressions and investigated source responses to pure face-expression contrasts. We were able to replicate the same highly group discriminative neural signatures during action observation (dynamical face expressions) and imagery, in the precuneus. Larger activation in regions involved in imagery for the ASD group suggests that this effect is compensatory. We conducted a machine learning procedure to automatically identify these group differences, based on the EEG activity during mental imagery of facial expressions. We compared two classifiers and achieved an accuracy of 81% using 15 features (both linear and non-linear) of the signal from theta, high-beta and gamma bands extracted from right-parietal locations (matching the precuneus region), further confirming the findings regarding standard statistical analysis. This robust classification of signals resulting from imagery of dynamical expressions in ASD is surprising because it far and significantly exceeds the good classification already achieved with observation of neutral face expressions (74%). This novel neural correlate of emotional imagery in autism could potentially serve as a clinical interventional target for studies designed to improve facial expression recognition, or at least as an intervention biomarker.