Dawn M. Taylor

Continuous neuronal ensemble control of simulated arm reaching by a human with tetraplegia

Functional electrical stimulation (FES), the coordinated electrical activation of multiple muscles, has been used to restore arm and hand function in people with paralysis. User interfaces for such systems typically derive commands from mechanically unrelated parts of the body with retained volitional control, and are unnatural and unable to simultaneously command the various joints of the arm. Neural interface systems, based on spiking intracortical signals recorded from the arm area of motor cortex, have shown the ability to control computer cursors, robotic arms and individual muscles in intact non-human primates. Such neural interface systems may thus offer a more natural source of commands for restoring dexterous movements via FES. However, the ability to use decoded neural signals to control the complex mechanical dynamics of a reanimated human limb, rather than the kinematics of a computer mouse, has not been demonstrated. This study demonstrates the ability of an individual with long-standing tetraplegia to use cortical neuron recordings to command the real-time movements of a simulated dynamic arm. This virtual arm replicates the dynamics associated with arm mass and muscle contractile properties, as well as those of an FES feedback controller that converts user commands into the required muscle activation patterns. An individual with long-standing tetraplegia was thus able to control a virtual, two-joint, dynamic arm in real time using commands derived from an existing human intracortical interface technology. These results show the feasibility of combining such an intracortical interface with existing FES systems to provide a high-performance, natural system for restoring arm and hand function in individuals with extensive paralysis.

Training in cortical control of neuroprosthetic devices improves signal extraction from small neuronal ensembles

We have recently developed a closed-loop environment in which we can test the ability of primates to control the motion of a virtual device using ensembles of simultaneously recorded neurons /29/. Here we use a maximum likelihood method to assess the information about task performance contained in the neuronal ensemble. We trained two animals to control the motion of a computer cursor in three dimensions. Initially the animals controlled cursor motion using arm movements, but eventually they learned to drive the cursor directly from cortical activity. Using a population vector (PV) based upon the relation between cortical activity and arm motion, the animals were able to control the cursor directly from the brain in a closed-loop environment, but with difficulty. We added a supervised learning method that modified the parameters of the PV according to task performance (adaptive PV), and found that animals were able to exert much finer control over the cursor motion from brain signals. Here we describe a maximum likelihood method (ML) to assess the information about target contained in neuronal ensemble activity. Using this method, we compared the information about target contained in the ensemble during arm control, during brain control early in the adaptive PV, and during brain control after the adaptive PV had settled and the animal could drive the cursor reliably and with fine gradations. During the arm-control task, the ML was able to determine the target of the movement in as few as 10% of the trials, and as many as 75% of the trials, with an average of 65%. This average dropped when the animals used a population vector to control motion of the cursor. On average we could determine the target in around 35% of the trials. This low percentage was also reflected in poor control of the cursor, so that the animal was unable to reach the target in a large percentage of trials. Supervised adjustment of the population vector parameters produced new weighting coefficients and directional tuning parameters for many neurons. This produced a much better performance of the brain-controlled cursor motion. It was also reflected in the maximum likelihood measure of cell activity, producing the correct target based only on neuronal activity in over 80% of the trials on average. The changes in maximum likelihood estimates of target location based on ensemble firing show that an animals ability to regulate the motion of a cortically controlled device is not crucially dependent on the experimenters ability to estimate intention from neuronal activity.

Brain-Computer Interfaces: An international assessment of research and development trends

Brain-computer interface (BCI) research deals with establishing communication pathways between the brain and external devices where such pathways do not otherwise exist. Throughout the world, such research is surprisingly extensive and expanding. BCI research is rapidly approaching a level of first-generation medical practice for use by individuals whose neural pathways are damaged, and use of BCI technologies is accelerating rapidly in nonmedical arenas of commerce as well, particularly in the gaming, automotive, and robotics industries. The technologies used for BCI purposes are cutting-edge, enabling, and synergistic in many interrelated arenas, including signal processing, neural tissue engineering, multiscale modeling, systems integration, and robotics. This WTEC study gathered information on worldwide status and trends in BCI research to disseminate to government decisionmakers and the research community. The study reviewed and assessed the state of the art in sensor technology, the biotic-abiotic interface and biocompatibility, data analysis and modeling, hardware implementation, systems engineering, functional electrical stimulation, noninvasive communication systems, and cognitive and emotional neuroprostheses in academic research and industry. The study also compared the distinctly different foci, range, and investment levels of BCI research programs in the United States, Canada, China, Europe, and Japan.

Signal acquisition and analysis for cortical control of neuroprosthetics

Work in cortically controlled neuroprosthetic systems has concentrated on decoding natural behaviors from neural activity, with the idea that if the behavior could be fully decoded it could be duplicated using an artificial system. Initial estimates from this approach suggested that a high-fidelity signal comprised of many hundreds of neurons would be required to control a neuroprosthetic system successfully. However, recent studies are showing hints that these systems can be controlled effectively using only a few tens of neurons. Attempting to decode the pre-existing relationship between neural activity and natural behavior is not nearly as important as choosing a decoding scheme that can be more readily deployed and trained to generate the desired actions of the artificial system. These artificial systems need not resemble or behave similarly to any natural biological system. Effective matching of discrete and continuous neural command signals to appropriately configured device functions will enable effective control of both natural and abstract artificial systems using compatible thought processes.

The general utility of a neuroprosthetic device under direct cortical control

We have described an adaptive signal processing method that allows fine graded control of a cursor in three-dimensions from cortical signals. Here we describe application of the same signal processing method to direct cortical control of a robotic arm for a variety of tasks. Our subject was extensively trained in controlling a computer cursor in a 3D virtual environment. We applied the mapping between cortical activity and cursor motion to endpoint control of a robotic arm. This algorithm was refined further as the animal continued to make 3D point-to-point movements of the brain-controlled robot. The animal then used the cortically-controlled robot to retrieve food placed at arbitrary locations within the workspace and deliver the food to a hopper. Finally, the animal learned to use the cortically-controlled robot to deliver food directly to its mouth.

Real-Time Control of the Hand by Intracortically Controlled Functional Neuromuscular Stimulation

The purpose of this study was to develop an animal model to evaluate the efficacy of a brain machine interface (BMI) to control a neuroprosthesis intended to restore hand function via functional neuromuscular stimulation (FNS). We have implemented the system in a single primate, whose limb could be temporarily paralyzed by a reversible peripheral nerve block Recordings from the primary motor cortex were obtained from a 100-electrode array in the intact monkey, and used to predict the activity of a variety of wrist and hand muscles. These predictions were calculated in real-time, and used as inputs to a 4 channel neuromuscular stimulator for electrically activating the paralyzed muscles. Here we demonstrate that the BMI can be used to restore voluntary control of wrist flexion following muscle paralysis.

Discreet Discrete Commands for Assistive and Neuroprosthetic Devices

Many new assistive devices are available for individuals paralyzed below the neck due to spinal cord injury. Severely paralyzed individuals must be able to command their complex assistive devices using remaining activity from the neck up. Electromyographic (EMG) sensors enable people to use contractions of head and neck muscles to generate multiple proportional command signals. Electroencephalographic (EEG) signals can also be used to generate commands for assistive device control by conveying information about imagined or attempted movements. Fully-implanted wireless biopotential detection systems are now being developed to reliably detect EMGs, EEGs, or a mixture of the two from recording electrodes implanted just under the skin or scalp thus eliminating the need for externally worn hardware on the head or face. This present study shows how novel patterns of jaw muscle contractions, detected via biopotential sensors on the scalp surface or implanted just under the scalp, can be used to generate reliable discrete EMG commands, which can be differentiated from patterns generated during normal activities, such as chewing. These jaw contractions can be detected with sensors already in place to detect other muscle- or brain-based command signals thus adding to the functionality of current device control systems.

Virtual reality hardware and graphic display options for brain-machine interfaces.

Virtual reality hardware and graphic displays are reviewed here as a development environment for brain-machine interfaces (BMIs). Two desktop stereoscopic monitors and one 2D monitor were compared in a visual depth discrimination task and in a 3D target-matching task where able-bodied individuals used actual hand movements to match a virtual hand to different target hands. Three graphic representations of the hand were compared: a plain sphere, a sphere attached to the fingertip of a realistic hand and arm, and a stylized pacman-like hand. Several subjects had great difficulty using either stereo monitor for depth perception when perspective size cues were removed. A mismatch in stereo and size cues generated inappropriate depth illusions. This phenomenon has implications for choosing target and virtual hand sizes in BMI experiments. Target-matching accuracy was about as good with the 2D monitor as with either 3D monitor. However, users achieved this accuracy by exploring the boundaries of the hand in the target with carefully controlled movements. This method of determining relative depth may not be possible in BMI experiments if movement control is more limited. Intuitive depth cues, such as including a virtual arm, can significantly improve depth perception accuracy with or without stereo viewing.

Use of Intracortical Recordings to Control a Hand Neuroprosthesis

The purpose of this study was to develop an animal model for evaluating the efficacy of a brain machine interface (BMI) for controlling functional neuromuscular stimulation (FNS). We have implemented such a system in a single primate. Our experimental model includes a 100-electrode implanted micro-array for recording intracortical signals, a reversible peripheral nerve block for temporarily paralyzing wrist and hand muscles, and a neuromuscular stimulator for electrically activating the paralyzed muscles. To date, we have evaluated the efficacy of each of these components and demonstrated how the BMI can be used to restore voluntary control of wrist flexion following muscle paralysis

Predicting reach goal in a continuous workspace for command of a brain-controlled upper-limb neuroprosthesis

A controller for an upper-limb functional electrical stimulation system could use intended reach goal to generate a set of stimulation patterns that would move the hand to the desired location via a reasonably naturalistic velocity profile. Although discrete classifiers have been successfully used to predict movement goal from a fixed number of possible reach locations using neural activity recorded during movement planning, practical implementation of this paradigm for use in upper-limb neuroprostheses requires the ability to predict a reach goal anywhere within a persons workspace. Using neural data collected from monkeys during brain-controlled movements of a virtual cursor and robotic arm, we evaluated how well the direction versus magnitude of the final movement goal could be predicted from varying lengths of neural data collected after the target appeared. Although a majority of the channels were significantly modulated with intended movement direction, only 10-20% showed any significant modulation related to the magnitude of the movement goal. We propose a method of trajectory generation that could use the more reliably encoded directional information in the neural activity to control both magnitude and direction of a goal oriented reaching movement