Naotaka Fujii

Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkeys.

Brain–machine interfaces (BMIs) employ the electrical activity generated by cortical neurons directly for controlling external devices and have been conceived as a means for restoring human cognitive or sensory-motor functions. The dominant approach in BMI research has been to decode motor variables based on single-unit activity (SUA). Unfortunately, this approach suffers from poor long-term stability and daily recalibration is normally required to maintain reliable performance. A possible alternative is BMIs based on electrocorticograms (ECoGs), which measure population activity and may provide more durable and stable recording. However, the level of long-term stability that ECoG-based decoding can offer remains unclear. Here we propose a novel ECoG-based decoding paradigm and show that we have successfully decoded hand positions and arm joint angles during an asynchronous food-reaching task in monkeys when explicit cues prompting the onset of movement were not required. Performance using our ECoG-based decoder was comparable to existing SUA-based systems while evincing far superior stability and durability. In addition, the same decoder could be used for months without any drift in accuracy or recalibration. These results were achieved by incorporating the spatio-spectro-temporal integration of activity across multiple cortical areas to compensate for the lower fidelity of ECoG signals. These results show the feasibility of high-performance, chronic and versatile ECoG-based neuroprosthetic devices for real-life applications. This new method provides a stable platform for investigating cortical correlates for understanding motor control, sensory perception, and high-level cognitive processes.

Extension of corticocortical afferents into the anterior bank of the intraparietal sulcus by tool-use training in adult monkeys

When humans use a tool, it becomes an extension of the hand physically and perceptually. Common introspection might occur in monkeys trained in tool-use, which should depend on brain operations that constantly update and automatically integrate information about the current intrinsic (somatosensory) and the extrinsic (visual) status of the body parts and the tools. The parietal cortex plays an important role in using tools. Intraparietal neurones of naïve monkeys mostly respond unimodally to somatosensory stimuli; however, after training these neurones become bimodally active and respond to visual stimuli. The response properties of these neurones change to code the body images modified by assimilation of the tool to the hand holding it. In this study, we compared the projection patterns between visually related areas and the intraparietal cortex in trained and naïve monkeys using tracer techniques. Light microscopy analyses revealed the emergence of novel projections from the higher visual centres in the vicinity of the temporo-parietal junction and the ventrolateral prefrontal areas to the intraparietal area in monkeys trained in tool-use, but not in naïve monkeys. Functionally active synapses of intracortical afferents arising from higher visual centres to the intraparietal cortex of the trained monkeys were confirmed by electron microscopy. These results provide the first concrete evidence for the induction of novel neural connections in the adult monkey cerebral cortex, which accompanies a process of demanding behaviour in these animals.

Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method

A new generalized multilinear regression model, termed the higher order partial least squares (HOPLS), is introduced with the aim to predict a tensor (multiway array) Y from a tensor X through projecting the data onto the latent space and performing regression on the corresponding latent variables. HOPLS differs substantially from other regression models in that it explains the data by a sum of orthogonal Tucker tensors, while the number of orthogonal loadings serves as a parameter to control model complexity and prevent overfitting. The low-dimensional latent space is optimized sequentially via a deflation operation, yielding the best joint subspace approximation for both X and Y. Instead of decomposing X and Y individually, higher order singular value decomposition on a newly defined generalized cross-covariance tensor is employed to optimize the orthogonal loadings. A systematic comparison on both synthetic data and real-world decoding of 3D movement trajectories from electrocorticogram signals demonstrate the advantages of HOPLS over the existing methods in terms of better predictive ability, suitability to handle small sample sizes, and robustness to noise.

Multidimensional recording (MDR) and data sharing: an ecological open research and educational platform for neuroscience.

Primate neurophysiology has revealed various neural mechanisms at the single-cell level and population level. However, because recording techniques have not been updated for several decades, the types of experimental design that can be applied in the emerging field of social neuroscience are limited, in particular those involving interactions within a realistic social environment. To address these limitations and allow more freedom in experimental design to understand dynamic adaptive neural functions, multidimensional recording (MDR) was developed. MDR obtains behavioral, neural, eye position, and other biological data simultaneously by using integrated multiple recording systems. MDR gives a wide degree of freedom in experimental design because the level of behavioral restraint is adjustable depending on the experimental requirements while still maintaining the signal quality. The biggest advantage of MDR is that it can provide a stable neural signal at higher temporal resolution at the network level from multiple subjects for months, which no other method can provide. Conventional event-related analysis of MDR data shows results consistent with previous findings, whereas new methods of analysis can reveal network mechanisms that could not have been investigated previously. MDR data are now shared in the public server Neurotycho.org. These recording and sharing methods support an ecological system that is open to everyone and will be a valuable and powerful research/educational platform for understanding the dynamic mechanisms of neural networks.

Dynamic social adaptation of motion-related neurons in primate parietal cortex

Social brain function, which allows us to adapt our behavior to social context, is poorly understood at the single-cell level due largely to technical limitations. But the questions involved are vital: How do neurons recognize and modulate their activity in response to social context? To probe the mechanisms involved, we developed a novel recording technique, called multi-dimensional recording, and applied it simultaneously in the left parietal cortices of two monkeys while they shared a common social space. When the monkeys sat near each other but did not interact, each monkeys parietal activity showed robust response preference to action by his own right arm and almost no response to action by the others arm. But the preference was broken if social conflict emerged between the monkeys—specifically, if both were able to reach for the same food item placed on the table between them. Under these circumstances, parietal neurons started to show complex combinatorial responses to motion of self and other. Parietal cortex adapted its response properties in the social context by discarding and recruiting different neural populations. Our results suggest that parietal neurons can recognize social events in the environment linked with current social context and form part of a larger social brain network.

Measuring integrated information from the decoding perspective

Accumulating evidence indicates that the capacity to integrate information in the brain is a prerequisite for consciousness. Integrated Information Theory (IIT) of consciousness provides a mathematical approach to quantifying the information integrated in a system, called integrated information, Φ. Integrated information is defined theoretically as the amount of information a system generates as a whole, above and beyond the amount of information its parts independently generate. IIT predicts that the amount of integrated information in the brain should reflect levels of consciousness. Empirical evaluation of this theory requires computing integrated information from neural data acquired from experiments, although difficulties with using the original measure Φ precludes such computations. Although some practical measures have been previously proposed, we found that these measures fail to satisfy the theoretical requirements as a measure of integrated information. Measures of integrated information should satisfy the lower and upper bounds as follows: The lower bound of integrated information should be 0 and is equal to 0 when the system does not generate information (no information) or when the system comprises independent parts (no integration). The upper bound of integrated information is the amount of information generated by the whole system. Here we derive the novel practical measure Φ* by introducing a concept of mismatched decoding developed from information theory. We show that Φ* is properly bounded from below and above, as required, as a measure of integrated information. We derive the analytical expression of Φ* under the Gaussian assumption, which makes it readily applicable to experimental data. Our novel measure Φ* can generally be used as a measure of integrated information in research on consciousness, and also as a tool for network analysis on diverse areas of biology.

Decoding continuous three-dimensional hand trajectories from epidural electrocorticographic signals in Japanese macaques

Brain–machine interface (BMI) technology captures brain signals to enable control of prosthetic or communication devices with the goal of assisting patients who have limited or no ability to perform voluntary movements. Decoding of inherent information in brain signals to interpret the user’s intention is one of main approaches for developing BMI technology. Subdural electrocorticography (sECoG)-based decoding provides good accuracy, but surgical complications are one of the major concerns for this approach to be applied in BMIs. In contrast, epidural electrocorticography (eECoG) is less invasive, thus it is theoretically more suitable for long-term implementation, although it is unclear whether eECoG signals carry sufficient information for decoding natural movements. We successfully decoded continuous three-dimensional hand trajectories from eECoG signals in Japanese macaques. A steady quantity of information of continuous hand movements could be acquired from the decoding system for at least several months, and a decoding model could be used for ∼10 days without significant degradation in accuracy or recalibration. The correlation coefficients between observed and predicted trajectories were lower than those for sECoG-based decoding experiments we previously reported, owing to a greater degree of chewing artifacts in eECoG-based decoding than is found in sECoG-based decoding. As one of the safest invasive recording methods available, eECoG provides an acceptable level of performance. With the ease of replacement and upgrades, eECoG systems could become the first-choice interface for real-life BMI applications.

Social cognition in premotor and parietal cortex

Abstract Socially correct behavior requires constant observation of the social environment. Behavior that was appropriate a few seconds ago is not guaranteed to be appropriate now. The brain keeps the eyes focused on the current social space and constantly updates its internal representation of the environment and social context. Monitoring the behavior of others is essential for this updating. The neural systems involved in perceiving the actions of others have been explored extensively, but the detailed, quantitative character of the system at the single-cell level remains poorly understood. To address this question, we used the new technique of multidimensional recording to record neuronal activity in monkeys simultaneously from ventral premotor cortex (PM) and parietal cortex in the left hemisphere while they performed a food grab task. Motion-related (MR) response was shown by 35% (52/148) of PM neurons and 54% (94/174) of parietal neurons, meaning their activity increased in response to various combinations of arm motions made by self and/or other. Both areas showed robust lateralized preference to Self–Right action. When it came to recognizing the actions of the other monkey, PM-MR neurons showed the same kind of right-arm preference as self-action while parietal-MR neurons, in contrast, did not show arm preference. And while both areas discriminated self-action from other, a significantly larger proportion of PM-MR neurons did so. These results suggest that PM neurons provide information about an actions agent and effector as primitives of action cognition within the mirror neuron network, while parietal neurons represent social space and participate in the recognition of another agents actions in relation to ones own actions within the parieto-prefrontal network.

Social state representation in prefrontal cortex

Abstract One of the cardinal mental faculties of humans and other primates is social brain function, the collective name assigned to the distributed system of social cognitive processes that orchestrate our sophisticated adaptive social behavior. These must include processes for recognizing current social context and maintaining an internal representation of the current social state as a reference for decision-making. But how and where the brain processes such social-state information is unknown. To home in on the neural substrates of social-state representation, the activity of 196 prefrontal (PFC) neurons was recorded from two monkeys simultaneously during a food-grab task under varying social conditions. Of PFC neurons, 39% showed activity modulation during movement-free periods and seemed to be representing current social state. The direction of modulation was opposite between the dominant and submissive monkeys: During social engagement, PFC activity increased in the dominant monkey and was suppressed in the submissive monkey. The modulation was consistently observed in additional PFC neurons (27/72) in additional pairings with two other monkeys. Notably, PFC activity in one formerly submissive monkey switched to dominant modulation mode when he was paired with a new monkey of lower social status. These findings suggest that PFC, as part of a larger social brain network, maintains a multistate classification of social context for use as a behavioral reference for social decision-making.

Loss of Consciousness Is Associated with Stabilization of Cortical Activity

What aspects of neuronal activity distinguish the conscious from the unconscious brain? This has been a subject of intense interest and debate since the early days of neurophysiology. However, as any practicing anesthesiologist can attest, it is currently not possible to reliably distinguish a conscious state from an unconscious one on the basis of brain activity. Here we approach this problem from the perspective of dynamical systems theory. We argue that the brain, as a dynamical system, is self-regulated at the boundary between stable and unstable regimes, allowing it in particular to maintain high susceptibility to stimuli. To test this hypothesis, we performed stability analysis of high-density electrocorticography recordings covering an entire cerebral hemisphere in monkeys during reversible loss of consciousness. We show that, during loss of consciousness, the number of eigenmodes at the edge of instability decreases smoothly, independently of the type of anesthetic and specific features of brain activity. The eigenmodes drift back toward the unstable line during recovery of consciousness. Furthermore, we show that stability is an emergent phenomenon dependent on the correlations among activity in different cortical regions rather than signals taken in isolation. These findings support the conclusion that dynamics at the edge of instability are essential for maintaining consciousness and provide a novel and principled measure that distinguishes between the conscious and the unconscious brain. SIGNIFICANCE STATEMENT What distinguishes brain activity during consciousness from that observed during unconsciousness? Answering this question has proven difficult because neither consciousness nor lack thereof have universal signatures in terms of most specific features of brain activity. For instance, different anesthetics induce different patterns of brain activity. We demonstrate that loss of consciousness is universally and reliably associated with stabilization of cortical dynamics regardless of the specific activity characteristics. To give an analogy, our analysis suggests that loss of consciousness is akin to depressing the damper pedal on the piano, which makes the sounds dissipate quicker regardless of the specific melody being played. This approach may prove useful in detecting consciousness on the basis of brain activity under anesthesia and other settings.