Nicolas Y. Masse

Reach and grasp by people with tetraplegia using a neurally controlled robotic arm

Paralysis following spinal cord injury, brainstem stroke, amyotrophic lateral sclerosis and other disorders can disconnect the brain from the body, eliminating the ability to perform volitional movements. A neural interface system could restore mobility and independence for people with paralysis by translating neuronal activity directly into control signals for assistive devices. We have previously shown that people with long-standing tetraplegia can use a neural interface system to move and click a computer cursor and to control physical devices. Able-bodied monkeys have used a neural interface system to control a robotic arm, but it is unknown whether people with profound upper extremity paralysis or limb loss could use cortical neuronal ensemble signals to direct useful arm actions. Here we demonstrate the ability of two people with long-standing tetraplegia to use neural interface system-based control of a robotic arm to perform three-dimensional reach and grasp movements. Participants controlled the arm and hand over a broad space without explicit training, using signals decoded from a small, local population of motor cortex (MI) neurons recorded from a 96-channel microelectrode array. One of the study participants, implanted with the sensor 5 years earlier, also used a robotic arm to drink coffee from a bottle. Although robotic reach and grasp actions were not as fast or accurate as those of an able-bodied person, our results demonstrate the feasibility for people with tetraplegia, years after injury to the central nervous system, to recreate useful multidimensional control of complex devices directly from a small sample of neural signals.

Olfactory Information Processing in Drosophila

In both insect and vertebrate olfactory systems only two synapses separate the sensory periphery from brain areas required for memory formation and the organisation of behaviour. In the Drosophila olfactory system, which is anatomically very similar to its vertebrate counterpart, there has been substantial recent progress in understanding the flow of information from experiments using molecular genetic, electrophysiological and optical imaging techniques. In this review, we shall focus on how olfactory information is processed and transformed in order to extract behaviourally relevant information. We follow the progress from olfactory receptor neurons, through the first processing area, the antennal lobe, to higher olfactory centres. We address both the underlying anatomy and mechanisms that govern the transformation of neural activity. We emphasise our emerging understanding of how different elementary computations, including signal averaging, gain control, decorrelation and integration, may be mapped onto different circuit elements.

Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface

Individuals with tetraplegia are able to type self-paced for hours across multiple days using a self-calibrating point-and-click intracortical brain-computer interface. Prolonged typing with refined BCI The fact that the brain can be hooked up to a computer to allow paralyzed individuals to type is already a technological feat. But, these so-called brain-computer interface technologies can be tiring and burdensome for users, requiring frequent disruptions for recalibration when the decoded neural signals change. Jarosiewicz and colleagues therefore combined three calibration methods—retrospective target interference, velocity bias correction, and adaptive tracking of neural features—for seamless typing and stable neural control. This combination allowed two individuals with tetraplegia and with cortical microelectrode arrays to compose long texts at their own paces, with no need to interrupt typing for recalibration. Brain-computer interfaces (BCIs) promise to restore independence for people with severe motor disabilities by translating decoded neural activity directly into the control of a computer. However, recorded neural signals are not stationary (that is, can change over time), degrading the quality of decoding. Requiring users to pause what they are doing whenever signals change to perform decoder recalibration routines is time-consuming and impractical for everyday use of BCIs. We demonstrate that signal nonstationarity in an intracortical BCI can be mitigated automatically in software, enabling long periods (hours to days) of self-paced point-and-click typing by people with tetraplegia, without degradation in neural control. Three key innovations were included in our approach: tracking the statistics of the neural activity during self-timed pauses in neural control, velocity bias correction during neural control, and periodically recalibrating the decoder using data acquired during typing by mapping neural activity to movement intentions that are inferred retrospectively based on the user’s self-selected targets. These methods, which can be extended to a variety of neurally controlled applications, advance the potential for intracortical BCIs to help restore independent communication and assistive device control for people with paralysis.

The Effect of Microsaccades on the Correlation between Neural Activity and Behavior in Middle Temporal, Ventral Intraparietal, and Lateral Intraparietal Areas

It is widely reported that the activity of single neurons in visual cortex is correlated with the perceptual decision of the subject. The strength of this correlation has implications for the neuronal populations generating the percepts. Here we asked whether microsaccades, which are small, involuntary eye movements, contribute to the correlation between neural activity and behavior. We analyzed data from three different visual detection experiments, with neural recordings from the middle temporal (MT), lateral intraparietal (LIP), and ventral intraparietal (VIP) areas. All three experiments used random dot motion stimuli, with the animals required to detect a transient or sustained change in the speed or strength of motion. We found that microsaccades suppressed neural activity and inhibited detection of the motion stimulus, contributing to the correlation between neural activity and detection behavior. Microsaccades accounted for as much as 19% of the correlation for area MT, 21% for area LIP, and 17% for VIP. While microsaccades only explain part of the correlation between neural activity and behavior, their effect has implications when considering the neuronal populations underlying perceptual decisions.

Task-specific versus generalized mnemonic representations in parietal and prefrontal cortices

Our ability to learn a wide range of behavioral tasks is essential for responding appropriately to sensory stimuli according to behavioral demands, but the underlying neural mechanism has been rarely examined by neurophysiological recordings in the same subjects across learning. To understand how learning new behavioral tasks affects neuronal representations, we recorded from posterior parietal cortex (PPC) before and after training on a visual motion categorization task. We found that categorization training influenced cognitive encoding in PPC, with a marked enhancement of memory-related delay-period encoding during the categorization task that was absent during a motion discrimination task before categorization training. In contrast, the prefrontal cortex (PFC) exhibited strong delay-period encoding during both discrimination and categorization tasks. This reveals a dissociation between PFCs and PPCs roles in working memory, with general engagement of PFC across multiple tasks, in contrast with more task-specific mnemonic encoding in PPC.

Neural Point-and-Click Communication by a Person With Incomplete Locked-In Syndrome

A goal of brain–computer interface research is to develop fast and reliable means of communication for individuals with paralysis and anarthria. We evaluated the ability of an individual with incomplete locked-in syndrome enrolled in the BrainGate Neural Interface System pilot clinical trial to communicate using neural point-and-click control. A general-purpose interface was developed to provide control of a computer cursor in tandem with one of two on-screen virtual keyboards. The novel BrainGate Radial Keyboard was compared to a standard QWERTY keyboard in a balanced copy-spelling task. The Radial Keyboard yielded a significant improvement in typing accuracy and speed—enabling typing rates over 10 correct characters per minute. The participant used this interface to communicate face-to-face with research staff by using text-to-speech conversion, and remotely using an internet chat application. This study demonstrates the first use of an intracortical brain–computer interface for neural point-and-click communication by an individual with incomplete locked-in syndrome.

The Effect of Middle Temporal Spike Phase on Sensory Encoding and Correlates with Behavior during a Motion-Detection Task

Previous studies have shown that sensory neurons that are the most informative of the stimulus tend to be the best correlated with the subjects perceptual decision. We wanted to know whether this relationship might also apply to short time segments of a neurons response. We asked whether spikes that conveyed more information about a motion stimulus were also more tightly linked to the perceptual behavior. We examined single-neuron activity in middle temporal (MT) area while monkeys performed a motion-detection task. Because of a slow stimulus update (every 27 ms), activity in many MT neurons was entrained and phase-locked to the stimulus. These stimulus-entrained neuronal oscillations allowed us to separate spikes based on phase. We observed a large amount of variability in how spikes at different phases of the oscillation encoded the stimulus, as revealed by the spike-triggered average of the motion. Spikes during certain phases of the cycle were much more informative about the presence of coherent motion than others. Importantly, we found that the phases that were the most informative about the motion stimulus were also more correlated with the behavioral performance and reaction time of the animal. Our results suggest that the relationship between a neurons spikes, the stimulus, and behavior can vary on a time scale of tens of milliseconds.

Behavioral Time Course of Microstimulation in Cortical Area MT

Electrical stimulation of the brain is a valuable research tool and has shown therapeutic promise in the development of new sensory neural prosthetics. Despite its widespread use, we still do not fully understand how current passed through a microelectrode interacts with functioning neural circuits. Past behavioral studies have suggested that weak electrical stimulation (referred to as microstimulation) of sensory areas of cortex produces percepts that are similar to those generated by normal sensory stimuli. In contrast, electrophysiological studies using in vitro or anesthetized preparations have shown that neural activity produced by brief microstimulation is radically different and longer lasting than normal responses. To help reconcile these two aspects of microstimulation, we examined the temporal properties that microstimulation has on visual perception. We found that brief application of subthreshold microstimulation in the middle temporal (MT) area of visual cortex produced smaller and longer-lasting effects on motion perception compared with an equivalent visual stimulus. In agreement with past electrophysiological studies, a computer simulation reproduced our behavioral effects when the time course of a single microstimulation pulse was modeled with three components: an immediate fast strong excitatory component, followed by a weaker inhibitory component, and then followed by a long duration weak excitatory component. Overall, these results suggest the behavioral effects of microstimulation in our experiments were caused by the unique and long-lasting temporal effects microstimulation has on functioning cortical circuits.

A Comparison of Lateral and Medial Intraparietal Areas during a Visual Categorization Task

Categorization is essential for interpreting sensory stimuli and guiding our actions. Recent studies have revealed robust neuronal category representations in the lateral intraparietal area (LIP). Here, we examine the specialization of LIP for categorization and the roles of other parietal areas by comparing LIP and the medial intraparietal area (MIP) during a visual categorization task. MIP is involved in goal-directed arm movements and visuomotor coordination but has not been implicated in non-motor cognitive functions, such as categorization. As expected, we found strong category encoding in LIP. Interestingly, we also observed category signals in MIP. However, category signals were stronger and appeared with a shorter latency in LIP than MIP. In this task, monkeys indicated whether a test stimulus was a category match to a previous sample with a manual response. Test-period activity in LIP showed category encoding and distinguished between matches and non-matches. In contrast, MIP primarily reflected the match/non-match status of test stimuli, with a strong preference for matches (which required a motor response). This suggests that, although category representations are distributed across parietal cortex, LIP and MIP play distinct roles: LIP appears more involved in the categorization process itself, whereas MIP is more closely tied to decision-related motor actions.

Mnemonic encoding and cortical organization in parietal and prefrontal cortices

Persistent activity within the frontoparietal network is consistently observed during tasks that require working memory. However, the neural circuit mechanisms underlying persistent neuronal encoding within this network remain unresolved. Here, we ask how neural circuits support persistent activity by examining population recordings from posterior parietal (PPC) and prefrontal (PFC) cortices in two male monkeys that performed spatial and motion direction-based tasks that required working memory. While spatially selective persistent activity was observed in both areas, robust selective persistent activity for motion direction was only observed in PFC. Crucially, we find that this difference between mnemonic encoding in PPC and PFC is associated with the presence of functional clustering: PPC and PFC neurons up to ∼700 μm apart preferred similar spatial locations, and PFC neurons up to ∼700 μm apart preferred similar motion directions. In contrast, motion-direction tuning similarity between nearby PPC neurons was much weaker and decayed rapidly beyond ∼200 μm. We also observed a similar association between persistent activity and functional clustering in trained recurrent neural network models embedded with a columnar topology. These results suggest that functional clustering facilitates mnemonic encoding of sensory information. SIGNIFICANCE STATEMENT Working memory refers to our ability to temporarily store and manipulate information. Numerous studies have observed that, during working memory, neurons in higher cortical areas, such as the parietal and prefrontal cortices, mnemonically encode the remembered stimulus. However, several recent studies have failed to observe mnemonic encoding during working memory, raising the question as to why mnemonic encoding is observed during some, but not all, conditions. In this study, we show that mnemonic encoding occurs when a cortical area is organized such that nearby neurons preferentially respond to the same stimulus. This result provides plausible neuronal conditions that allow for mnemonic encoding, and gives us further understanding of the brains mechanisms that support working memory.