Sergi Bermúdez i Badia

Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: methodology, design, psychometrics, usability and validation

BackgroundStroke is a frequent cause of adult disability that can lead to enduring impairments. However, given the life-long plasticity of the brain one could assume that recovery could be facilitated by the harnessing of mechanisms underlying neuronal reorganization. Currently it is not clear how this reorganization can be mobilized. Novel technology based neurorehabilitation techniques hold promise to address this issue. Here we describe a Virtual Reality (VR) based system, the Rehabilitation Gaming System (RGS) that is based on a number of hypotheses on the neuronal mechanisms underlying recovery, the structure of training and the role of individualization. We investigate the psychometrics of the RGS in stroke patients and healthy controls.MethodsWe describe the key components of the RGS and the psychometrics of one rehabilitation scenario called Spheroids. We performed trials with 21 acute/subacute stroke patients and 20 healthy controls to study the effect of the training parameters on task performance. This allowed us to develop a Personalized Training Module (PTM) for online adjustment of task difficulty. In addition, we studied task transfer between physical and virtual environments. Finally, we assessed the usability and acceptance of the RGS as a rehabilitation tool.ResultsWe show that the PTM implemented in RGS allows us to effectively adjust the difficulty and the parameters of the task to the user by capturing specific features of the movements of the arms. The results reported here also show a consistent transfer of movement kinematics between physical and virtual tasks. Moreover, our usability assessment shows that the RGS is highly accepted by stroke patients as a rehabilitation tool.ConclusionsWe introduce a novel VR based paradigm for neurorehabilitation, RGS, which combines specific rehabilitative principles with a psychometric evaluation to provide a personalized and automated training. Our results show that the RGS effectively adjusts to the individual features of the user, allowing for an unsupervised deployment of individualized rehabilitation protocols.

Virtual reality based rehabilitation speeds up functional recovery of the upper extremities after stroke: A randomized controlled pilot study in the acute phase of stroke using the Rehabilitation Gaming System

PURPOSE Given the incidence of stroke, the need has arisen to consider more self-managed rehabilitation approaches. A promising technology is Virtual Reality (VR). Thus far, however, it is not clear what the benefits of VR systems are when compared to conventional methods. Here we investigated the clinical impact of one such system, the Rehabilitation Gaming System (RGS), on the recovery time course of acute stroke. RGS combines concepts of action execution and observation with an automatic individualization of training. METHODS. Acute stroke patients (n = 8) used the RGS during 12 weeks in addition to conventional therapy. A control group (n = 8) performed a time matched alternative treatment, which consisted of intense occupational therapy or non-specific interactive games. RESULTS. At the end of the treatment, between-group comparisons showed that the RGS group displayed significantly improved performance in paretic arm speed that was matched by better performance in the arm subpart of the Fugl-Meyer Assessment Test and the Chedoke Arm and Hand Activity Inventory. In addition, the RGS group presented a significantly faster improvement over time for all the clinical scales during the treatment period. CONCLUSIONS. Our results suggest that rehabilitation with the RGS facilitates the functional recovery of the upper extremities and that this system is therefore a promising tool for stroke neurorehabilitation.

An artificial moth: Chemical source localization using a robot based neuronal model of moth optomotor anemotactic search

Robots have been used to model nature, while nature in turn can contribute to the real-world artifacts we construct. One particular domain of interest is chemical search where a number of efforts are underway to construct mobile chemical search and localization systems. We report on a project that aims at constructing such a system based on our understanding of the pheromone communication system of the moth. Based on an overview of the peripheral processing of chemical cues by the moth and its role in the organization of behavior we emphasize the multimodal aspects of chemical search, i.e. optomotor anemotactic chemical search. We present a model of this behavior that we test in combination with a novel thin metal oxide sensor and custom build mobile robots. We show that the sensor is able to detect the odor cue, ethanol, under varying flow conditions. Subsequently we show that the standard model of insect chemical search, consisting of a surge and cast phases, provides for robust search and localization performance. The same holds when it is augmented with an optomotor collision avoidance model based on the Lobula Giant Movement Detector (LGMD) neuron of the locust. We compare our results to others who have used the moth as inspiration for the construction of odor robots.

The Combined Impact of Virtual Reality Neurorehabilitation and Its Interfaces on Upper Extremity Functional Recovery in Patients With Chronic Stroke

Background and Purpose— Although there is strong evidence on the beneficial effects of virtual reality (VR)-based rehabilitation, it is not yet well understood how the different aspects of these systems affect recovery. Consequently, we do not exactly know what features of VR neurorehabilitation systems are decisive in conveying their beneficial effects. Methods— To specifically address this issue, we developed 3 different configurations of the same VR-based rehabilitation system, the Rehabilitation Gaming System, using 3 different interface technologies: vision-based tracking, haptics, and a passive exoskeleton. Forty-four patients with chronic stroke were randomly allocated to one of the configurations and used the system for 35 minutes a day for 5 days a week during 4 weeks. Results— Our results revealed significant within-subject improvements at most of the standard clinical evaluation scales for all groups. Specifically we observe that the beneficial effects of VR-based training are modulated by the use/nonuse of compensatory movement strategies and the specific sensorimotor contingencies presented to the user, that is, visual feedback versus combined visual haptic feedback. Conclusions— Our findings suggest that the beneficial effects of VR-based neurorehabilitation systems such as the Rehabilitation Gaming System for the treatment of chronic stroke depend on the specific interface systems used. These results have strong implications for the design of future VR rehabilitation strategies that aim at maximizing functional outcomes and their retention. Clinical Trial Registration— This trial was not registered because it is a small clinical study that evaluates the feasibility of prototype devices.

Using a Hybrid Brain Computer Interface and Virtual Reality System to Monitor and Promote Cortical Reorganization through Motor Activity and Motor Imagery Training

Stroke is one of the leading causes of adult disability with high economical and societal costs. In recent years, novel rehabilitation paradigms have been proposed to address the life-long plasticity of the brain to regain motor function. We propose a hybrid brain-computer interface (BCI)-reality (VR) system that combines a personalized motor training in a VR environment, exploiting brain mechanisms for action execution and observation, and a neuro-feedback paradigm using mental imagery as a way to engage secondary or indirect pathways to access undamaged cortico-spinal tracts. Furthermore, we present the development and validation experiments of the proposed system. More specifically, EEG data on nine naïve healthy subjects show that a simultaneous motor activity and motor imagery paradigm is more effective at engaging cortical motor areas and related networks to a larger extent. Additionally, we propose a motor imagery driven BCI-VR version of our system that was evaluated with nine different healthy subjects. Data show that users are capable of controlling a virtual avatar in a motor imagery training task that dynamically adjusts its difficulty to the capabilities of the user. User self-report questionnaires indicate enjoyment and acceptance of the proposed system.

A fly-locust based neuronal control system applied to an unmanned aerial vehicle: the invertebrate neuronal principles for course stabilization, altitude control and collision avoidance

The most versatile and robust flying machines are still those produced by nature through evolution. The solutions to the 6 DOF control problem faced by these machines are implemented in extremely small neuronal structures comprising thousands of neurons. Hence, the biological principles of flight control are not only very effective but also efficient in terms of their implementation. An important question is to what extent these principles can be generalized to man-made flying platforms. Here, this question is investigated in relation to the computational and behavioral principles of the opto-motor system of the fly and locust. The aim is to provide a control infrastructure based only on biologically plausible and realistic neuronal models of the insect opto-motor system. It is shown that relying solely on vision, biologically constrained neuronal models of the fly visual system suffice for course stabilization and altitude control of a blimp-based UAV. Moreover, the system is augmented with a collision avoidance model based on the Lobula Giant Movement Detector neuron of the Locust. It is shown that the biologically constrained course stabilization model is highly robust and that the combined model is able to perform autonomous indoor flight.

The Rehabilitation Gaming System: a Virtual Reality Based System for the Evaluation and Rehabilitation of Motor Deficits

We have developed a virtual reality based system for the rehabilitation of patients suffering from different neuropathologies such as those brought on by stroke and traumatic brain injury. Our Rehabilitation Gaming System, or RGS, uses a vision based motion capture system with gaming technologies and functionally combines active evaluation, with continuous monitoring and intensive training regimes tuned to the needs of individual patients. Here we assess the validity of the evaluation phase of the RGS by comparing the physical and virtual versions of a diagnostic reaching test with 6 stroke patients. Subsequently we illustrate the ability of our system to provide high resolution information at the level of individual performance.

A Biologically Based Chemo-Sensing UAV for Humanitarian Demining:

Antipersonnel mines, weapons of cheap manufacture but lethal effect, have a high impact on the population even decades after the conflicts have finished. Here we investigate the use of a chemo-sensing Unmanned Aerial Vehicle (cUAV) for demining tasks. We developed a blimp based UAV that is equipped with a broadly tuned metal-thin oxide chemo-sensor. A number of chemical mapping strategies were investigated including two biologically based localization strategies derived from the moth chemical search that can optimize the efficiency of the detection and localization of explosives and therefore be used in the demining process. Additionally, we developed a control layer that allows for both fully autonomous and manual controlled flight, as well as for the scheduling of a fleet of cUAVs. Our results confirm the feasibility of this technology for demining in real-world scenarios and give further support to a biologically based approach where the understanding of biological systems is used to solve difficult engineering problems.

PASAR: An integrated model of prediction, anticipation, sensation, attention and response for artificial sensorimotor systems

A wide range of neuroscientific studies suggest the existence of cognitive mechanisms like attention, prediction, anticipation and strong vertical interactions between different hierarchical layers of the brain while performing complex tasks. Despite advances in both cognitive brain research and in the development of brain-inspired artificial cognitive systems, the interplay of these key ingredients of cognition remain largely elusive and unquantified in complex real-world tasks. Furthermore, it has not yet been demonstrated how a self-contained hierarchical cognitive system acting under limited resource constraints can quantifiably benefit from the incorporation of top-down and bottom-up attentional mechanisms. In this context, an open fundamental question is how a data association mechanism can integrate bottom-up sensory information and top-down knowledge. Here, building on the Distributed Adaptive Control (DAC) architecture, we propose a single framework for integrating these different components of cognition and demonstrate the frameworks performance in solving real-world and simulated robot tasks. Using the model we quantify the interactions between prediction, anticipation, attention and memory. Our results support the strength of a complete system that incorporates attention, prediction and anticipation mechanisms compared to incomplete systems for real-world and complex tasks. We unveil the relevance of transient memory that underlines the utility of the above mechanisms for intelligent knowledge management in artificial sensorimotor systems. These findings provide concrete predictions for physiological and psychophysical experiments to validate our model in biological cognitive systems.

The eXperience Induction Machine: A New Paradigm for Mixed-Reality Interaction Design and Psychological Experimentation

The eXperience Induction Machine (XIM) is one of the most advanced mixed-reality spaces available today. XIM is an immersive space that consists of physical sensors and effectors and which is conceptualized as a general-purpose infrastructure for research in the field of psychology and human–artifact interaction. In this chapter, we set out the epistemological rational behind XIM by putting the installation in the context of psychological research. The design and implementation of XIM are based on principles and technologies of neuromorphic control. We give a detailed description of the hardware infrastructure and software architecture, including the logic of the overall behavioral control. To illustrate the approach toward psychological experimentation, we discuss a number of practical applications of XIM. These include the so-called, persistent virtual community, the application in the research of the relationship between human experience and multi-modal stimulation, and an investigation of a mixed-reality social interaction paradigm.