Daniel Bacher

Reach and grasp by people with tetraplegia using a neurally controlled robotic arm

Paralysis following spinal cord injury, brainstem stroke, amyotrophic lateral sclerosis and other disorders can disconnect the brain from the body, eliminating the ability to perform volitional movements. A neural interface system could restore mobility and independence for people with paralysis by translating neuronal activity directly into control signals for assistive devices. We have previously shown that people with long-standing tetraplegia can use a neural interface system to move and click a computer cursor and to control physical devices. Able-bodied monkeys have used a neural interface system to control a robotic arm, but it is unknown whether people with profound upper extremity paralysis or limb loss could use cortical neuronal ensemble signals to direct useful arm actions. Here we demonstrate the ability of two people with long-standing tetraplegia to use neural interface system-based control of a robotic arm to perform three-dimensional reach and grasp movements. Participants controlled the arm and hand over a broad space without explicit training, using signals decoded from a small, local population of motor cortex (MI) neurons recorded from a 96-channel microelectrode array. One of the study participants, implanted with the sensor 5 years earlier, also used a robotic arm to drink coffee from a bottle. Although robotic reach and grasp actions were not as fast or accurate as those of an able-bodied person, our results demonstrate the feasibility for people with tetraplegia, years after injury to the central nervous system, to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.

Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface

Individuals with tetraplegia are able to type self-paced for hours across multiple days using a self-calibrating point-and-click intracortical brain-computer interface. Prolonged typing with refined BCI The fact that the brain can be hooked up to a computer to allow paralyzed individuals to type is already a technological feat. But, these so-called brain-computer interface technologies can be tiring and burdensome for users, requiring frequent disruptions for recalibration when the decoded neural signals change. Jarosiewicz and colleagues therefore combined three calibration methods—retrospective target interference, velocity bias correction, and adaptive tracking of neural features—for seamless typing and stable neural control. This combination allowed two individuals with tetraplegia and with cortical microelectrode arrays to compose long texts at their own paces, with no need to interrupt typing for recalibration. Brain-computer interfaces (BCIs) promise to restore independence for people with severe motor disabilities by translating decoded neural activity directly into the control of a computer. However, recorded neural signals are not stationary (that is, can change over time), degrading the quality of decoding. Requiring users to pause what they are doing whenever signals change to perform decoder recalibration routines is time-consuming and impractical for everyday use of BCIs. We demonstrate that signal nonstationarity in an intracortical BCI can be mitigated automatically in software, enabling long periods (hours to days) of self-paced point-and-click typing by people with tetraplegia, without degradation in neural control. Three key innovations were included in our approach: tracking the statistics of the neural activity during self-timed pauses in neural control, velocity bias correction during neural control, and periodically recalibrating the decoder using data acquired during typing by mapping neural activity to movement intentions that are inferred retrospectively based on the user’s self-selected targets. These methods, which can be extended to a variety of neurally controlled applications, advance the potential for intracortical BCIs to help restore independent communication and assistive device control for people with paralysis.

Neural Point-and-Click Communication by a Person With Incomplete Locked-In Syndrome

A goal of brain–computer interface research is to develop fast and reliable means of communication for individuals with paralysis and anarthria. We evaluated the ability of an individual with incomplete locked-in syndrome enrolled in the BrainGate Neural Interface System pilot clinical trial to communicate using neural point-and-click control. A general-purpose interface was developed to provide control of a computer cursor in tandem with one of two on-screen virtual keyboards. The novel BrainGate Radial Keyboard was compared to a standard QWERTY keyboard in a balanced copy-spelling task. The Radial Keyboard yielded a significant improvement in typing accuracy and speed—enabling typing rates over 10 correct characters per minute. The participant used this interface to communicate face-to-face with research staff by using text-to-speech conversion, and remotely using an internet chat application. This study demonstrates the first use of an intracortical brain–computer interface for neural point-and-click communication by an individual with incomplete locked-in syndrome.

An assistive decision-and-control architecture for force-sensitive hand-arm systems driven by human-machine interfaces

Fully autonomous applications of modern robotic systems are still constrained by limitations in sensory data processing, scene interpretation, and automated reasoning. However, their use as assistive devices for people with upper-limb disabilities has become possible with recent advances in “soft robotics”, that is, interaction control, physical human–robot interaction, and reflex planning. In this context, impedance and reflex-based control has generally been understood to be a promising approach to safe interaction robotics. To create semi-autonomous assistive devices, we propose a decision-and-control architecture for hand–arm systems with “soft robotics” capabilities, which can then be used via human–machine interfaces (HMIs). We validated the functionality of our approach within the BrainGate2 clinical trial, in which an individual with tetraplegia used our architecture to control a robotic hand–arm system under neural control via a multi-electrode array implanted in the motor cortex. The neuroscience results of this research have previously been published by Hochberg et al. In this paper we present our assistive decision-and-control architecture and demonstrate how the semi-autonomous assistive behavior can help the user. In our framework the robot is controlled through a multi-priority Cartesian impedance controller and its behavior is extended with collision detection and reflex reaction. Furthermore, virtual workspaces are added to ensure safety. On top of this we employ a decision-and-control architecture that uses sensory information available from the robotic system to evaluate the current state of task execution. Based on a set of available assistive skills, our architecture provides support in object interaction and manipulation and thereby enhances the usability of the robotic system for use with HMIs. The goal of our development is to provide an easy-to-use robotic system for people with physical disabilities and thereby enable them to perform simple tasks of daily living. In an exemplary real-world task, the participant was able to serve herself a beverage autonomously for the first time since her brainstem stroke, which she suffered approximately 14 years prior to this research.

Continuous Control of the DLR Light-Weight Robot III by a Human with Tetraplegia Using the BrainGate2 Neural Interface System

We have investigated control of the DLR Light-Weight Robot III with DLR Five-Finger Hand by a person with tetraplegia using the BrainGate2 Neural Interface System. The goal of this research is to develop assistive technologies for people with severe physical disabilities. A BrainGate-enabled DLR LWR III would potentially permit a person with tetraplegia to gain improved control over their environment, e.g. to drink a glass of water. First results of the developed control loop are very encouraging and allow the participant to perform simple interaction tasks with her environment, e.g., pick up a bottle and move it around. To this end, only a few minutes of system training are required, after which the system can be used.

Towards the optimal design of an assistive communication interface with neural input

Brain Computer Interfaces (BCIs) directly connect the brain to an external device. Since neural signals encoding movement in tetraplegic individuals may remain robust, BCIs can utilize this signal for external device control. By recording activity in the motor cortex, intracortically-based BCIs have converted this neural signal to perform useful and intended motor functions [1]. The BrainGate2 Neural Interface System (NIS) is an investigational device that has allowed individuals with physical disability to interact directly with a computer, enabling them to navigate the screen with a cursor, click, and type. Simulating neural cursor control is useful for comparing the potential user performance of different communication interfaces. These simulations can be used to improve neurally controlled communication interface design towards restoring reliable communication for individuals with tetraplegia.