Marnix Nuttin

A Brain-Actuated Wheelchair: Asynchronous and Non-Invasive Brain-Computer Interfaces for Continuous Control of Robots

OBJECTIVE To assess the feasibility and robustness of an asynchronous and non-invasive EEG-based Brain-Computer Interface (BCI) for continuous mental control of a wheelchair. METHODS In experiment 1 two subjects were asked to mentally drive both a real and a simulated wheelchair from a starting point to a goal along a pre-specified path. Here we only report experiments with the simulated wheelchair for which we have extensive data in a complex environment that allows a sound analysis. Each subject participated in five experimental sessions, each consisting of 10 trials. The time elapsed between two consecutive experimental sessions was variable (from 1h to 2months) to assess the system robustness over time. The pre-specified path was divided into seven stretches to assess the system robustness in different contexts. To further assess the performance of the brain-actuated wheelchair, subject 1 participated in a second experiment consisting of 10 trials where he was asked to drive the simulated wheelchair following 10 different complex and random paths never tried before. RESULTS In experiment 1 the two subjects were able to reach 100% (subject 1) and 80% (subject 2) of the final goals along the pre-specified trajectory in their best sessions. Different performances were obtained over time and path stretches, what indicates that performance is time and context dependent. In experiment 2, subject 1 was able to reach the final goal in 80% of the trials. CONCLUSIONS The results show that subjects can rapidly master our asynchronous EEG-based BCI to control a wheelchair. Also, they can autonomously operate the BCI over long periods of time without the need for adaptive algorithms externally tuned by a human operator to minimize the impact of EEG non-stationarities. This is possible because of two key components: first, the inclusion of a shared control system between the BCI system and the intelligent simulated wheelchair; second, the selection of stable user-specific EEG features that maximize the separability between the mental tasks. SIGNIFICANCE These results show the feasibility of continuously controlling complex robotics devices using an asynchronous and non-invasive BCI.

Omnidirectional Vision Based Topological Navigation

In this work we present a novel system for autonomous mobile robot navigation. With only an omnidirectional camera as sensor, this system is able to build automatically and robustly accurate topologically organised environment maps of a complex, natural environment. It can localise itself using such a map at each moment, including both at startup (kidnapped robot) or using knowledge of former localisations. The topological nature of the map is similar to the intuitive maps humans use, is memory-efficient and enables fast and simple path planning towards a specified goal. We developed a real-time visual servoing technique to steer the system along the computed path.A key technology making this all possible is the novel fast wide baseline feature matching, which yields an efficient description of the scene, with a focus on man-made environments.

Adaptive Shared Control of a Brain-Actuated Simulated Wheelchair

The use of shared control techniques has a profound impact on the performance of a robotic assistant controlled by human brain signals. However, this shared control usually provides assistance to the user in a constant and identical manner each time. Creating an adaptive level of assistance, thereby complementing the users capabilities at any moment, would be more appropriate. The better the user can do by himself, the less assistance he receives from the shared control system; and vice versa. In order to do this, we need to be able to detect when and in what way the user needs assistance. An appropriate assisting behaviour would then be activated for the time the user requires help, thereby adapting the level of assistance to the specific situation. This paper presents such a system, helping a brain-computer interface (BCI) subject perform goal-directed navigation of a simulated wheelchair in an adaptive manner. Whenever the subject has more difficulties in driving the wheelchair, more assistance will be given. Experimental results of two subjects show that this adaptive shared control increases the task performance. Also, it shows that a subject with a lower BCI performance has more need for extra assistance in difficult situations, such as manoeuvring in a narrow corridor.

Asynchronous non-invasive brain-actuated control of an intelligent wheelchair

In this paper we present further results of our asynchronous and non-invasive BMI for the continuous control of an intelligent wheelchair. Three subjects participated in two experiments where they steered the wheelchair spontaneously, without any external cue. To do so the users learn to voluntary modulate EEG oscillatory rhythms by executing three mental tasks (i.e., mental imagery) that are associated to different steering commands. Importantly, we implement shared control techniques between the BMI and the intelligent wheelchair to assist the subject in the driving task. The results show that the three subjects could achieve a significant level of mental control, even if far from optimal, to drive an intelligent wheelchair.

Context-based filtering for assisted brain-actuated wheelchair driving

Controlling a robotic device by using human brain signals is an interesting and challenging task. The device may be complicated to control and the nonstationary nature of the brain signals provides for a rather unstable input. With the use of intelligent processing algorithms adapted to the task at hand, however, the performance can be increased. This paper introduces a shared control system that helps the subject in driving an intelligent wheelchair with a noninvasive brain interface. The subjects steering intentions are estimated from electroencephalogram (EEG) signals and passed through to the shared control system before being sent to the wheelchair motors. Experimental results show a possibility for significant improvement in the overall driving performance when using the shared control system compared to driving without it. These results have been obtained with 2 healthy subjects during their first day of training with the brain-actuated wheelchair.

User-adapted plan recognition and user-adapted shared control: A Bayesian approach to semi-autonomous wheelchair driving

Abstract Many elderly and physically impaired people experience difficulties when maneuvering a powered wheelchair. In order to ease maneuvering, powered wheelchairs have been equipped with sensors, additional computing power and intelligence by various research groups. This paper presents a Bayesian approach to maneuvering assistance for wheelchair driving, which can be adapted to a specific user. The proposed framework is able to model and estimate even complex user intents, i.e. wheelchair maneuvers that the driver has in mind. Furthermore, it explicitly takes the uncertainty on the user’s intent into account. Besides during intent estimation, user-specific properties and uncertainty on the user’s intent are incorporated when taking assistive actions, such that assistance is tailored to the user’s driving skills. This decision making is modeled as a greedy Partially Observable Markov Decision Process (POMDP). Benefits of this approach are shown using experimental results in simulation and on our wheelchair platform Sharioto.

Feature based omnidirectional sparse visual path following

Vision sensors are attractive for autonomous robots because they are a rich source of environment information. The main challenge in using images for mobile robots is managing this wealth of information. A relatively recent approach is the use of fast wide baseline local features, which we developed and used in the novel approach to sparse visual path following described in this paper. These local features have the great advantage that they can be recognized even if the viewpoint differs significantly. This opens the door to a memory efficient description of a path by descriptors of sparse images. We propose a method for re-execution of these paths by a series of visual homing operations which yield a navigation method with unique properties: it is accurate, robust, fast, and without odometry error build-up.

Pruning and regularization in reservoir computing

Reservoir computing is a new paradigm for using recurrent neural network with a much simpler training method. The key idea is to use a large but fixed recurrent part as a reservoir of dynamic features and to train only the output layer to extract the desired information. We propose to study how pruning some connections from the reservoir to the output layer can help on the one hand to increase the generalization ability, in much the same way as regularization techniques do, and on the other hand to improve the implementability of reservoirs in hardware.

Bayesian Estimation of Wheelchair Driver Intents: Modeling Intents as Geometric Paths Tracked by the Driver

Many elderly and disabled people today experience difficulties when manoeuvring an electric wheelchair. In order to help these people, several robotic assistance platforms have been devised in the past. In most cases, these platforms consist of separate assistance modes, and heuristic rules are used to automatically decide which assistance mode should be selected in each time step. As these decision rules are often hard-coded and do not take uncertainty regarding the users intent into account, assistive actions may lead to confusion or even irritation if the users actual plans do not correspond to the assistive systems behavior. In contrast to previous approaches, this paper presents a more user-centered approach for recognizing the intent of wheelchair drivers, which explicitly estimates the uncertainty on the users intent. The paper shows the benefit of estimating this uncertainty using experimental results with our wheelchair platform Sharioto

Lino, the user-interface robot

This paper reports on the development of a domestic user-interface robot that is able to have a natural human interaction by speech and emotional feedback and is able to navigate in a home environment. The natural interaction with the user is achieved by means of a mechanical head able to express emotions. The robot is aware of the position and identities of the users, both from visual and auditory information. The robot estimates its location in the environment with an appearance-based localization method using a stereo camera system. The navigation to the goal is achieved with a hybrid method, combining planning with reactive control. The robot is designed to operate in an intelligent environment, such that external information can be used to localize users and their intentions (context awareness), and that additional information can be retrieved from various databases in the environment. The result is a service robot that can have a simple dialogue with the user, provide information in a natural way (speech and expressions) and can be instructed to navigate to any specific goal in the environment.