Pedro Omedas

Network dynamics with BrainX3: a large-scale simulation of the human brain network with real-time interaction

BrainX3 is a large-scale simulation of human brain activity with real-time interaction, rendered in 3D in a virtual reality environment, which combines computational power with human intuition for the exploration and analysis of complex dynamical networks. We ground this simulation on structural connectivity obtained from diffusion spectrum imaging data and model it on neuronal population dynamics. Users can interact with BrainX3 in real-time by perturbing brain regions with transient stimulations to observe reverberating network activity, simulate lesion dynamics or implement network analysis functions from a library of graph theoretic measures. BrainX3 can thus be used as a novel immersive platform for exploration and analysis of dynamical activity patterns in brain networks, both at rest or in a task-related state, for discovery of signaling pathways associated to brain function and/or dysfunction and as a tool for virtual neurosurgery. Our results demonstrate these functionalities and shed insight on the dynamics of the resting-state attractor. Specifically, we found that a noisy network seems to favor a low firing attractor state. We also found that the dynamics of a noisy network is less resilient to lesions. Our simulations on TMS perturbations show that even though TMS inhibits most of the network, it also sparsely excites a few regions. This is presumably due to anti-correlations in the dynamics and suggests that even a lesioned network can show sparsely distributed increased activity compared to healthy resting-state, over specific brain areas.

Inference of human affective states from psychophysiological measurements extracted under ecologically valid conditions

Compared to standard laboratory protocols, the measurement of psychophysiological signals in real world experiments poses technical and methodological challenges due to external factors that cannot be directly controlled. To address this problem, we propose a hybrid approach based on an immersive and human accessible space called the eXperience Induction Machine (XIM), that incorporates the advantages of a laboratory within a life-like setting. The XIM integrates unobtrusive wearable sensors for the acquisition of psychophysiological signals suitable for ambulatory emotion research. In this paper, we present results from two different studies conducted to validate the XIM as a general-purpose sensing infrastructure for the study of human affective states under ecologically valid conditions. In the first investigation, we recorded and classified signals from subjects exposed to pictorial stimuli corresponding to a range of arousal levels, while they were free to walk and gesticulate. In the second study, we designed an experiment that follows the classical conditioning paradigm, a well-known procedure in the behavioral sciences, with the additional feature that participants were free to move in the physical space, as opposed to similar studies measuring physiological signals in constrained laboratory settings. Our results indicate that, by using our sensing infrastructure, it is indeed possible to infer human event-elicited affective states through measurements of psychophysiological signals under ecological conditions.

Connectomics to Semantomics: Addressing the Brain's Big Data Challenge1☆

Abstract Can semantic corpora be coupled to dynamical simulations in such a way so as to extract new associations from the data that were hitherto unapparent? We attempt to do this within neuroscience as an application domain, by introducing the notion of the semantome and coupling it to the connectome of the human brain network. This is implemented using BrainX3, a virtual reality simulation cum data mining platform that can be used for visualization, analysis and feature extraction of neuroscience data. We use this system to explore anatomical, functional and symptomatic semantics associated to simulated neuronal activity of a healthy brain, one with stroke and one perturbed by transcranial magnetic stimulation. In particular, we find that parietal and occipital lesions in stroke affect the visual processing pathway leading to symptoms such as visual neglect, depression and photo-sensitivity seizures. Integrating semantomics with connectomics thus generates hypotheses about symptoms, functions and brain activity that supplement existing tools for diagnosis of mental illness. Our results suggest a new approach to big data with potential applications to other domains.

XIM-engine: a software framework to support the development of interactive applications that uses conscious and unconscious reactions in immersive mixed reality

The development of systems that allow multimodal interpretation of human-machine interaction is crucial to advance our understanding and validation of theoretical models of user behavior. In particular, a system capable of collecting, perceiving and interpreting unconscious behavior can provide rich contextual information for an interactive system. One possible application for such a system is in the exploration of complex data through immersion, where massive amounts of data are generated every day both by humans and computer processes that digitize information at different scales and resolutions thus exceeding our processing capacity. We need tools that accelerate our understanding and generation of hypotheses over the datasets, guide our searches and prevent data overload. We describe XIM-engine, a bio-inspired software framework designed to capture and analyze multi-modal human behavior in an immersive environment. The framework allows performing studies that can advance our understanding on the use of conscious and unconscious reactions in interactive systems.

Advanced interfaces to stem the data deluge in mixed reality: placing human (un)consciousness in the loop

We live in an era of data deluge and this requires novel tools to effectively extract, analyze and understand the massive amounts of data produced by the study of natural and artificial phenomena in many areas of research.

AngioLab-A software tool for morphological analysis and endovascular treatment planning of intracranial aneurysms

Determining whether and how an intracranial aneurysm should be treated is a tough decision that clinicians face everyday. Emerging computational tools could help clinicians analyze clinical data and make these decisions. AngioLab is a single graphical user interface, developed on top of the open source framework GIMIAS, that integrates some of the latest image analysis and computational modeling tools for intracranial aneurysms. Two workflows are available: Advanced Morphological Analysis (AMA) and Endovascular Treatment Planning (ETP). AngioLab has been evaluated by a total of 62 clinicians, who considered the information provided by AngioLab relevant and meaningful. They acknowledged the emerging need of these type of tools and the potential impact they might have on the clinical decision-making process.

BrainX 3 : embodied exploration of neural data

We present BrainX3 as a novel immersive and interactive technology for exploration of large biological data, which in this paper is customized towards brain networks. Unlike traditional machine-inference systems, BrainX3 posits a two-way coupling of human intuition to powerful machine computation to tackle the big data challenge. Furthermore, through unobtrusive wearable sensors, BrainX3 can infer users states in terms of arousal and cognitive workload, thus changing the visualization and the sonification parameters to boost the exploration process.

The dynamic connectome: towards large-scale 3D reconstruction of brain activity in real-time

What does large-scale connectivity tell us about whole-brain activity and neural circuits? In this work, we present a virtual reality based large-scale dynamic simulation for 3D reconstruction of whole-brain activity over the cortical connectome in real-time. Using DTI structural connectivity data from [1] we built an interactive 3D visualization of the human connectome network in an immersive virtual reality environment (Figure ​(Figure1A)1A) using the Unity 3D gaming engine. Further, the virtual reality brain network in Unity is coupled to a real-time neuronal simulator, iqr [2]. As we see, coupling structural connectivity data with detailed enough neuronal population dynamics is sufficient in predicting functional correlations and large-scale activity patterns. We model neuronal dynamics by a linear-threshold filter (as work in progress, we are currently implementing population dynamics from mean-field models [3]). Each population module is stochastic, having Gaussian noise. The user can stimulate any region or simultaneous regions of the network with external input currents. The simulation then reconstructs reverberating neural activity propagating throughout the network in real-time. As an explicit example, we stimulate the superior parietal areas and observe causal activity propagation in the parietal lobe, indicative of visuo-motor integration (Figure ​(Figure1B).1B). This is a first step to simulating and mapping large-scale brain activity in real-time. Figure 1 (A) snapshot of the cortical connectome network in virtual reality. (B) activation in the parietal lobes just after stimulation of the superior parietal area. Activity persists for about 5 secs after stimulation. Results and conclusions As quantitative analysis methods and data-recording technology in neuroscience make improvements, it is becoming evident large-scale dynamics and whole-brain quantitative measures play an important role. For instance, oscillations across large brain regions are precursors to several cognitive functions. Moreover, the causal map in these interactions is crucial. Compared to functional correlations, large-scale temporal activity maps across directionally connected brain structures serve as a more powerful tool to unravel mechanisms of large-scale neural circuits. Our results show that stimulating brain areas triggers a sequence of causal activations in associated network loops that represent cognitively related functions.

The Dynamic Connectome: A Tool For Large-Scale 3D Reconstruction Of Brain Activity In Real-Time.

We present a large-scale simulation tool for real-time 3D reconstruction of brain activity in a virtual reality environment. The 3D interactive visualization of the human cortex connectome in virtual reality is achieved by using a gaming engine (Unity 3D). Further, the visualization is bi-directionally interfaced with a real-time neuronal simulator, iqr. As a result, by stimulating populations of neurons with external input currents, we are able to reconstruct neural activity propagating in 3D and in real-time. We explicitly demonstrate causal activity propagation in the parietal lobe, indicative of visuo-motor integration. This is a first step to simulating and mapping large-scale brain activity.

Towards a synthetic tutor assistant: The EASEL project and its architecture

Robots are gradually but steadily being introduced in our daily lives. A paramount application is that of education, where robots can assume the role of a tutor, a peer or simply a tool to help learners in a specific knowledge domain. Such endeavor posits specific challenges: affective social behavior, proper modelling of the learner’s progress, discrimination of the learner’s utterances, expressions and mental states, which, in turn, require an integrated architecture combining perception, cognition and action. In this paper we present an attempt to improve the current state of robots in the educational domain by introducing the EASEL EU project. Specifically, we introduce the EASEL’s unified robot architecture, an innovative Synthetic Tutor Assistant (STA) whose goal is to interactively guide learners in a science-based learning paradigm, allowing us to achieve such rich multimodal interactions.