Eric A. Pohlmeyer

Toward the Restoration of Hand Use to a Paralyzed Monkey: Brain-Controlled Functional Electrical Stimulation of Forearm Muscles

Loss of hand use is considered by many spinal cord injury survivors to be the most devastating consequence of their injury. Functional electrical stimulation (FES) of forearm and hand muscles has been used to provide basic, voluntary hand grasp to hundreds of human patients. Current approaches typically grade pre-programmed patterns of muscle activation using simple control signals, such as those derived from residual movement or muscle activity. However, the use of such fixed stimulation patterns limits hand function to the few tasks programmed into the controller. In contrast, we are developing a system that uses neural signals recorded from a multi-electrode array implanted in the motor cortex; this system has the potential to provide independent control of multiple muscles over a broad range of functional tasks. Two monkeys were able to use this cortically controlled FES system to control the contraction of four forearm muscles despite temporary limb paralysis. The amount of wrist force the monkeys were able to produce in a one-dimensional force tracking task was significantly increased. Furthermore, the monkeys were able to control the magnitude and time course of the force with sufficient accuracy to track visually displayed force targets at speeds reduced by only one-third to one-half of normal. Although these results were achieved by controlling only four muscles, there is no fundamental reason why the same methods could not be scaled up to control a larger number of muscles. We believe these results provide an important proof of concept that brain-controlled FES prostheses could ultimately be of great benefit to paralyzed patients with injuries in the mid-cervical spinal cord.

In a Blink of an Eye and a Switch of a Transistor: Cortically Coupled Computer Vision

Our societys information technology advancements have resulted in the increasingly problematic issue of information overload-i.e., we have more access to information than we can possibly process. This is nowhere more apparent than in the volume of imagery and video that we can access on a daily basis-for the general public, availability of YouTube video and Google Images, or for the image analysis professional tasked with searching security video or satellite reconnaissance. Which images to look at and how to ensure we see the images that are of most interest to us, begs the question of whether there are smart ways to triage this volume of imagery. Over the past decade, computer vision research has focused on the issue of ranking and indexing imagery. However, computer vision is limited in its ability to identify interesting imagery, particularly as ¿interesting¿ might be defined by an individual. In this paper we describe our efforts in developing brain-computer interfaces (BCIs) which synergistically integrate computer vision and human vision so as to construct a system for image triage. Our approach exploits machine learning for real-time decoding of brain signals which are recorded noninvasively via electroencephalography (EEG). The signals we decode are specific for events related to imagery attracting a users attention. We describe two architectures we have developed for this type of cortically coupled computer vision and discuss potential applications and challenges for the future.

Prediction of upper limb muscle activity from motor cortical discharge during reaching

Movement representation by the motor cortex (M1) has been a theoretical interest for many years, but in the past several years it has become a more practical question, with the advent of the brain-machine interface. An increasing number of groups have demonstrated the ability to predict a variety of kinematic signals on the basis of M1 recordings and to use these predictions to control the movement of a cursor or robotic limb. We, on the other hand, have undertaken the prediction of myoelectric (EMG) signals recorded from various muscles of the arm and hand during button pressing and prehension movements. We have shown that these signals can be predicted with accuracy that is similar to that of kinematic signals, despite their stochastic nature and greater bandwidth. The predictions were made using a subset of 12 or 16 neural signals selected in the order of each signals unique, output-related information content. The accuracy of the resultant predictions remained stable through a typical experimental session. Accuracy remained above 80% of its initial level for most muscles even across periods as long as two weeks. We are exploring the use of these predictions as control signals for neuromuscular electrical stimulation in quadriplegic patients.

Biomimetic Brain Machine Interfaces for the Control of Movement

Quite recently, it has become possible to use signals recorded simultaneously from large numbers of cortical neurons for real-time control. Such brain machine interfaces (BMIs) have allowed animal subjects and human patients to control the position of a computer cursor or robotic limb under the guidance of visual feedback. Although impressive, such approaches essentially ignore the dynamics of the musculoskeletal system, and they lack potentially critical somatosensory feedback. In this mini-symposium, we will initiate a discussion of systems that more nearly mimic the control of natural limb movement. The work that we will describe is based on fundamental observations of sensorimotor physiology that have inspired novel BMI approaches. We will focus on what we consider to be three of the most important new directions for BMI development related to the control of movement. (1) We will present alternative methods for building decoders, including structured, nonlinear models, the explicit incorporation of limb state information, and novel approaches to the development of decoders for paralyzed subjects unable to generate an output signal. (2) We will describe the real-time prediction of dynamical signals, including joint torque, force, and EMG, and the real-time control of physical plants with dynamics like that of the real limb. (3) We will discuss critical factors that must be considered to incorporate somatosensory feedback to the BMI user, including its potential benefits, the differing representations of sensation and perception across cortical areas, and the changes in the cortical representation of tactile events after spinal injury.

Identification of Multiple-Input Systems with Highly Coupled Inputs: Application to EMG Prediction from Multiple Intracortical Electrodes

A robust identification algorithm has been developed for linear, time-invariant, multiple-input single-output systems, with an emphasis on how this algorithm can be used to estimate the dynamic relationship between a set of neural recordings and related physiological signals. The identification algorithm provides a decomposition of the system output such that each component is uniquely attributable to a specific input signal, and then reduces the complexity of the estimation problem by discarding those input signals that are deemed to be insignificant. Numerical difficulties due to limited input bandwidth and correlations among the inputs are addressed using a robust estimation technique based on singular value decomposition. The algorithm has been evaluated on both simulated and experimental data. The latter involved estimating the relationship between up to 40 simultaneously recorded motor cortical signals and peripheral electromyograms (EMGs) from four upper limb muscles in a freely moving primate.The algorithm performed well in both cases:it provided reliable estimates of the system output and significantly reduced the number of inputs needed for output prediction. For example, although physiological recordings from up to 40 different neuronal signals were available, the input selection algorithm reduced this to 10 neuronal signals that made signicant contributions to the recorded EMGs.

Brain state decoding for rapid image retrieval

Human visual perception is able to recognize a wide range of targets under challenging conditions, but has limited throughput. Machine vision and automatic content analytics can process images at a high speed, but suffers from inadequate recognition accuracy for general target classes. In this paper, we propose a new paradigm to explore and combine the strengths of both systems. A single trial EEG-based brain machine interface (BCI) subsystem is used to detect objects of interest of arbitrary classes from an initial subset of images. The EEG detection outcomes are used as input to a graph-based pattern mining subsystem to identify, refine, and propagate the labels to retrieve relevant images from a much larger pool. The combined strategy is unique in its generality, robustness, and high throughput. It has great potential for advancing the state of the art in media retrieval applications. We have evaluated and demonstrated significant performance gains of the proposed system with multiple and diverse image classes over several data sets, including those from Internet (Caltech 101) and remote sensing images. In this paper, we will also present insights learned from the experiments and discuss future research directions.

Using reinforcement learning to provide stable brain-machine interface control despite neural input reorganization.

Brain-machine interface (BMI) systems give users direct neural control of robotic, communication, or functional electrical stimulation systems. As BMI systems begin transitioning from laboratory settings into activities of daily living, an important goal is to develop neural decoding algorithms that can be calibrated with a minimal burden on the user, provide stable control for long periods of time, and can be responsive to fluctuations in the decoder’s neural input space (e.g. neurons appearing or being lost amongst electrode recordings). These are significant challenges for static neural decoding algorithms that assume stationary input/output relationships. Here we use an actor-critic reinforcement learning architecture to provide an adaptive BMI controller that can successfully adapt to dramatic neural reorganizations, can maintain its performance over long time periods, and which does not require the user to produce specific kinetic or kinematic activities to calibrate the BMI. Two marmoset monkeys used the Reinforcement Learning BMI (RLBMI) to successfully control a robotic arm during a two-target reaching task. The RLBMI was initialized using random initial conditions, and it quickly learned to control the robot from brain states using only a binary evaluative feedback regarding whether previously chosen robot actions were good or bad. The RLBMI was able to maintain control over the system throughout sessions spanning multiple weeks. Furthermore, the RLBMI was able to quickly adapt and maintain control of the robot despite dramatic perturbations to the neural inputs, including a series of tests in which the neuron input space was deliberately halved or doubled.

The effects of targeted muscle reinnervation on neuromas in a rabbit rectus abdominis flap model

PURPOSE A targeted muscle reinnervation (TMR) model was created using a pedicled rabbit rectus abdominis (RA) flap to receive the input from previously amputated forelimb neuromas. We hypothesize that a segmental muscle flap can undergo TMR and that it is possible to differentiate the signal from 3 independent nerves. In addition, by virtue of the nerve coaptation, the morphology of the previous amputation neuroma would become more like that of an in-continuity neuroma. METHODS Five New Zealand white rabbits had a forelimb amputation. In a second-stage surgery, an RA flap was transposed onto the chest wall. After neuroma excision, 3 neurorrhaphies were made between the median nerve, radial nerve, and ulnar nerves, and 3 motor nerves of the RA. After 10 weeks, the electrophysiologic properties of the reinnervated flap were tested. Nerve specimens from the median, radial, and ulnar nerves were harvested before and after TMR to quantify the histomorphometric changes effected by TMR on the mixed nerve neuromas. RESULTS Of the 12 nerve coaptations performed in the 4 viable flaps, all 12 were grossly successful. Muscle surface EMG data demonstrated that the RA retained its segmental innervation pattern after TMR. Similarly, prolonged stimulation of 1 nerve reinnervating the RA resulted in the depletion of glycogen specific to the territory of the muscle stimulated by that nerve. TMR was found to favorably alter the histomorphometric characteristics of the neuroma by decreasing myelinated fiber counts and increasing fascicle diameter in the transferred nerves. CONCLUSIONS This study demonstrates that 1 segmented muscle having TMR by multiple nerve ingrowth and in turn generate discrete EMG signals. During this process, the previous amputation neuroma undergoes favorable morphologic alteration. CLINICAL RELEVANCE Based on these preclinical results, this technique might be useful in upper extremity amputees to recruit target muscles to have reinnervation to drive myoelectric prostheses and to treat symptomatic neuromas.

Towards autonomous neuroprosthetic control using Hebbian reinforcement learning

OBJECTIVE Our goal was to design an adaptive neuroprosthetic controller that could learn the mapping from neural states to prosthetic actions and automatically adjust adaptation using only a binary evaluative feedback as a measure of desirability/undesirability of performance. APPROACH Hebbian reinforcement learning (HRL) in a connectionist network was used for the design of the adaptive controller. The method combines the efficiency of supervised learning with the generality of reinforcement learning. The convergence properties of this approach were studied using both closed-loop control simulations and open-loop simulations that used primate neural data from robot-assisted reaching tasks. MAIN RESULTS The HRL controller was able to perform classification and regression tasks using its episodic and sequential learning modes, respectively. In our experiments, the HRL controller quickly achieved convergence to an effective control policy, followed by robust performance. The controller also automatically stopped adapting the parameters after converging to a satisfactory control policy. Additionally, when the input neural vector was reorganized, the controller resumed adaptation to maintain performance. SIGNIFICANCE By estimating an evaluative feedback directly from the user, the HRL control algorithm may provide an efficient method for autonomous adaptation of neuroprosthetic systems. This method may enable the user to teach the controller the desired behavior using only a simple feedback signal.

Real-Time Control of the Hand by Intracortically Controlled Functional Neuromuscular Stimulation

The purpose of this study was to develop an animal model to evaluate the efficacy of a brain machine interface (BMI) to control a neuroprosthesis intended to restore hand function via functional neuromuscular stimulation (FNS). We have implemented the system in a single primate, whose limb could be temporarily paralyzed by a reversible peripheral nerve block Recordings from the primary motor cortex were obtained from a 100-electrode array in the intact monkey, and used to predict the activity of a variety of wrist and hand muscles. These predictions were calculated in real-time, and used as inputs to a 4 channel neuromuscular stimulator for electrically activating the paralyzed muscles. Here we demonstrate that the BMI can be used to restore voluntary control of wrist flexion following muscle paralysis.