Andrew B. Schwartz

DIRECT CORTICAL CONTROL OF 3D NEUROPROSTHETIC DEVICES

Three-dimensional (3D) movement of neuroprosthetic devices can be controlled by the activity of cortical neurons when appropriate algorithms are used to decode intended movement in real time. Previous studies assumed that neurons maintain fixed tuning properties, and the studies used subjects who were unaware of the movements predicted by their recorded units. In this study, subjects had real-time visual feedback of their brain-controlled trajectories. Cell tuning properties changed when used for brain-controlled movements. By using control algorithms that track these changes, subjects made long sequences of 3D movements using far fewer cortical units than expected. Daily practice improved movement accuracy and the directional tuning of these units.

Cortical control of a prosthetic arm for self-feeding

Arm movement is well represented in populations of neurons recorded from the motor cortex. Cortical activity patterns have been used in the new field of brain–machine interfaces to show how cursors on computer displays can be moved in two- and three-dimensional space. Although the ability to move a cursor can be useful in its own right, this technology could be applied to restore arm and hand function for amputees and paralysed persons. However, the use of cortical signals to control a multi-jointed prosthetic device for direct real-time interaction with the physical environment (‘embodiment’) has not been demonstrated. Here we describe a system that permits embodied prosthetic control; we show how monkeys (Macaca mulatta) use their motor cortical activity to control a mechanized arm replica in a self-feeding task. In addition to the three dimensions of movement, the subjects’ cortical signals also proportionally controlled a gripper on the end of the arm. Owing to the physical interaction between the monkey, the robotic arm and objects in the workspace, this new task presented a higher level of difficulty than previous virtual (cursor-control) experiments. Apart from an example of simple one-dimensional control, previous experiments have lacked physical interaction even in cases where a robotic arm or hand was included in the control loop, because the subjects did not use it to interact with physical objects—an interaction that cannot be fully simulated. This demonstration of multi-degree-of-freedom embodied prosthetic control paves the way towards the development of dexterous prosthetic devices that could ultimately achieve arm and hand function at a near-natural level.

High-performance neuroprosthetic control by an individual with tetraplegia.

BACKGROUND Paralysis or amputation of an arm results in the loss of the ability to orient the hand and grasp, manipulate, and carry objects, functions that are essential for activities of daily living. Brain-machine interfaces could provide a solution to restoring many of these lost functions. We therefore tested whether an individual with tetraplegia could rapidly achieve neurological control of a high-performance prosthetic limb using this type of an interface. METHODS We implanted two 96-channel intracortical microelectrodes in the motor cortex of a 52-year-old individual with tetraplegia. Brain-machine-interface training was done for 13 weeks with the goal of controlling an anthropomorphic prosthetic limb with seven degrees of freedom (three-dimensional translation, three-dimensional orientation, one-dimensional grasping). The participants ability to control the prosthetic limb was assessed with clinical measures of upper limb function. This study is registered with ClinicalTrials.gov, NCT01364480. FINDINGS The participant was able to move the prosthetic limb freely in the three-dimensional workspace on the second day of training. After 13 weeks, robust seven-dimensional movements were performed routinely. Mean success rate on target-based reaching tasks was 91·6% (SD 4·4) versus median chance level 6·2% (95% CI 2·0-15·3). Improvements were seen in completion time (decreased from a mean of 148 s [SD 60] to 112 s [6]) and path efficiency (increased from 0·30 [0·04] to 0·38 [0·02]). The participant was also able to use the prosthetic limb to do skilful and coordinated reach and grasp movements that resulted in clinically significant gains in tests of upper limb function. No adverse events were reported. INTERPRETATION With continued development of neuroprosthetic limbs, individuals with long-term paralysis could recover the natural and intuitive command signals for hand placement, orientation, and reaching, allowing them to perform activities of daily living. FUNDING Defense Advanced Research Projects Agency, National Institutes of Health, Department of Veterans Affairs, and UPMC Rehabilitation Institute.

Brain-Controlled Interfaces: Movement Restoration with Neural Prosthetics

Brain-controlled interfaces are devices that capture brain transmissions involved in a subjects intention to act, with the potential to restore communication and movement to those who are immobilized. Current devices record electrical activity from the scalp, on the surface of the brain, and within the cerebral cortex. These signals are being translated to command signals driving prosthetic limbs and computer displays. Somatosensory feedback is being added to this control as generated behaviors become more complex. New technology to engineer the tissue-electrode interface, electrode design, and extraction algorithms to transform the recorded signal to movement will help translate exciting laboratory demonstrations to patient practice in the near future.

Motor cortical activity during drawing movements: population representation during lemniscate tracing.

Activity was recorded extracellularly from single cells in motor and premotor cortex as monkeys traced figure-eights on a touch-sensitive computer monitor using the index finger. Each unit was recorded individually, and the responses collected from four hemispheres (3 primary motor and 1 dorsal premotor) were analyzed as a population. Population vectors constructed from this activity accurately and isomorphically represented the shape of the drawn figures showing that they represent the spatial aspect of the task well. These observations were extended by examining the temporal relation between this neural representation and finger displacement. Movements generated during this task were made in four kinematic segments. This segmentation was clearly evident in a time series of population vectors. In addition, the (2)/(3) power law described for human drawing was also evident in the neural correlate of the monkey hand trajectory. Movement direction and speed changed continuously during the task. Within each segment, speed and direction changed reciprocally. The prediction interval between the population vector and movement direction increased in the middle of the segments where curvature was high, but decreased in straight portions at the beginning and end of each segment. In contrast to direction, prediction intervals between the movement speed and population vector length were near-constant with only a modest modulation in each segment. Population vectors predicted direction (vector angle) and speed (vector length) throughout the drawing task. Joint angular velocity and arm muscle EMG were well correlated to hand direction, suggesting that kinematic and kinetic parameters are correlated in these tasks.

Functional network reorganization during learning in a brain-computer interface paradigm

Efforts to study the neural correlates of learning are hampered by the size of the network in which learning occurs. To understand the importance of learning-related changes in a network of neurons, it is necessary to understand how the network acts as a whole to generate behavior. Here we introduce a paradigm in which the output of a cortical network can be perturbed directly and the neural basis of the compensatory changes studied in detail. Using a brain-computer interface, dozens of simultaneously recorded neurons in the motor cortex of awake, behaving monkeys are used to control the movement of a cursor in a three-dimensional virtual-reality environment. This device creates a precise, well-defined mapping between the firing of the recorded neurons and an expressed behavior (cursor movement). In a series of experiments, we force the animal to relearn the association between neural firing and cursor movement in a subset of neurons and assess how the network changes to compensate. We find that changes in neural activity reflect not only an alteration of behavioral strategy but also the relative contributions of individual neurons to the population error signal.

An electrocorticographic brain interface in an individual with tetraplegia.

Brain-computer interface (BCI) technology aims to help individuals with disability to control assistive devices and reanimate paralyzed limbs. Our study investigated the feasibility of an electrocorticography (ECoG)-based BCI system in an individual with tetraplegia caused by C4 level spinal cord injury. ECoG signals were recorded with a high-density 32-electrode grid over the hand and arm area of the left sensorimotor cortex. The participant was able to voluntarily activate his sensorimotor cortex using attempted movements, with distinct cortical activity patterns for different segments of the upper limb. Using only brain activity, the participant achieved robust control of 3D cursor movement. The ECoG grid was explanted 28 days post-implantation with no adverse effect. This study demonstrates that ECoG signals recorded from the sensorimotor cortex can be used for real-time device control in paralyzed individuals.

Information conveyed through brain-control: cursor versus robot

Microwire electrode arrays were implanted in the motor and premotor cortical areas of rhesus macaques. The recorded activity was used to control the three-dimensional movements of a virtual cursor and of a robotic arm in real time. The goal was to move the cursor or robot to one of eight targets. Average information conveyed about the intended target was calculated from the observed trajectories at 30-ms intervals throughout the movements. Most of the information about intended target was conveyed within the first second of the movement. For the brain-controlled cursor, the instantaneous information transmission rate was at its maximum at the beginning of each movement (averaged 4.8 to 5.5 bits/s depending on the calculation method used). However, this instantaneous rate quickly slowed down as the movement progressed and additional information became redundant. Information was conveyed more slowly through the brain-controlled robot due to the dynamics and noise of the robot system. The brain-controlled cursor data was also used to demonstrate a method for optimizing information transmission rate in the case where repeated cursor movements are used to make long strings of sequential choices such as in a typing task.

Extraction algorithms for cortical control of arm prosthetics.

Now that recordings of multiple, individual action potentials are being made with chronic electrodes, it seems that previous work showing simple encoding of movement parameters in these spike trains can be used as a real-time control signal for prosthetic arms. Efficient extraction algorithms can compensate for the limited ensemble sample acquired with this emerging technology.

Neural Interface Technology for Rehabilitation: Exploiting and Promoting Neuroplasticity

This article reviews neural interface technology and its relationship with neuroplasticity. Two types of neural interface technology are reviewed, highlighting specific technologies that the authors directly work with: (1) neural interface technology for neural recording, such as the micro-ECoG BCI system for hand prosthesis control, and the comprehensive rehabilitation paradigm combining MEG-BCI, action observation, and motor imagery training; (2) neural interface technology for functional neural stimulation, such as somatosensory neural stimulation for restoring somatosensation, and non-invasive cortical stimulation using rTMS and tDCS for modulating cortical excitability and stroke rehabilitation. The close interaction between neural interface devices and neuroplasticity leads to increased efficacy of neural interface devices and improved functional recovery of the nervous system. This symbiotic relationship between neural interface technology and the nervous system is expected to maximize functional gain for individuals with various sensory, motor, and cognitive impairments, eventually leading to better quality of life.