Hiroyuki Nakahara

Higher-Order Interactions Characterized in Cortical Activity

In the cortex, the interactions among neurons give rise to transient coherent activity patterns that underlie perception, cognition, and action. Recently, it was actively debated whether the most basic interactions, i.e., the pairwise correlations between neurons or groups of neurons, suffice to explain those observed activity patterns. So far, the evidence reported is controversial. Importantly, the overall organization of neuronal interactions and the mechanisms underlying their generation, especially those of high-order interactions, have remained elusive. Here we show that higher-order interactions are required to properly account for cortical dynamics such as ongoing neuronal avalanches in the alert monkey and evoked visual responses in the anesthetized cat. A Gaussian interaction model that utilizes the observed pairwise correlations and event rates and that applies intrinsic thresholding identifies those higher-order interactions correctly, both in cortical local field potentials and spiking activities. This allows for accurate prediction of large neuronal population activities as required, e.g., in brain–machine interface paradigms. Our results demonstrate that higher-order interactions are inherent properties of cortical dynamics and suggest a simple solution to overcome the apparent formidable complexity previously thought to be intrinsic to those interactions.

Learning to Simulate Others' Decisions

A fundamental challenge in social cognition is how humans learn another persons values to predict their decision-making behavior. This form of learning is often assumed to require simulation of the other by direct recruitment of ones own valuation process to model the others process. However, the cognitive and neural mechanism of simulation learning is not known. Using behavior, modeling, and fMRI, we show that simulation involves two learning signals in a hierarchical arrangement. A simulated-others reward prediction error processed in ventromedial prefrontal cortex mediated simulation by direct recruitment, being identical for valuation of the self and simulated-other. However, direct recruitment was insufficient for learning, and also required observation of the others choices to generatexa0a simulated-others action prediction error encoded in dorsomedial/dorsolateral prefrontal cortex. These findings show that simulation uses a core prefrontal circuit for modeling the others valuation to generate prediction and an adjunct circuit for tracking behavioral variation to refine prediction.

Multiple timescales of memory in lateral habenula and dopamine neurons.

Midbrain dopamine neurons are thought to signal predictions about future rewards based on the memory of past rewarding experience. Little is known about the source of their reward memory and the factors that control its timescale. Here we recorded from dopamine neurons, as well as one of their sources of input, the lateral habenula, while animals predicted upcoming rewards based on the past reward history. We found that lateral habenula and dopamine neurons accessed two distinct reward memories: a short-timescale memory expressed at the start of the task and a near-optimal long-timescale memory expressed when a future reward outcome was revealed. The short- and long-timescale memories were expressed in different forms of reward-oriented eye movements. Our data show that the habenula-dopamine pathway contains multiple timescales of memory and provide evidence for their role in motivated behavior.

Differential Reward Coding in the Subdivisions of the Primate Caudate during an Oculomotor Task

The basal ganglia play a pivotal role in reward-oriented behavior. The striatum, an input channel of the basal ganglia, is composed of subdivisions that are topographically connected with different cortical and subcortical areas. To test whether reward information is differentially processed in the different parts of the striatum, we compared reward-related neuronal activity along the dorsolateral–ventromedial axis in the caudate nucleus of monkeys performing an asymmetrically rewarded oculomotor task. In a given block, a target in one position was associated with a large reward, whereas the other target was associated with a small reward. The target position–reward value contingency was switched between blocks. We found the following: (1) activity that reflected the block-wise reward contingency emerged before the appearance of a visual target, and it was more prevalent in the dorsal, rather than central and ventral, caudate; (2) activity that was positively related to the reward size of the current trial was evident, especially after reward delivery, and it was more prevalent in the ventral and central, rather than dorsal, caudate; and (3) activity that was modulated by the memory of the outcomes of the previous trials was evident in the dorsal and central caudate. This multiple reward information, together with the target-direction information, was represented primarily by individual caudate neurons, and the different reward information was represented in caudate subpopulations with distinct electrophysiological properties, e.g., baseline firing and spike width. These results suggest parallel processing of different reward information by the basal ganglia subdivisions defined by extrinsic connections and intrinsic properties.

Encoding of social state information by neuronal activities in the macaque caudate nucleus.

Social animals adjust their behavior according to social relationships and momentary circumstances. Dominant–submissive relationships modulate, but do not completely determine, their competitive behaviors. For example, a submissive monkeys decision to retrieve food depends not only on the presence of dominant partners but also on their observed behavior. Thus, behavioral expression requires a dynamic evaluation of reward outcome and momentary social states. The neural mechanisms underlying this evaluation remain elusive. The caudate nucleus (CN) plays a pivotal role in representing reward expectation and translating it into action selection. To investigate whether their activities encode social state information, we recorded from CN neurons in monkeys while they performed a competitive food-grab task against a dominant competitor. We found two groups of CN neurons: one primarily responded to reward outcome, while the other primarily tracked the monkeys social state. These social state-dependent neurons showed greater activity when the monkeys freely retrieved food without active challenges from the competitor and reduced activity when the monkeys were in a submissive state due to the competitors active behavior. These results indicate that different neuronal activities in the CN encode social state information and reward-related information, which may contribute to adjusting competitive behavior in dynamic social contexts.

Internal-time temporal difference model for neural value-based decision making

The temporal difference (TD) learning framework is a major paradigm for understanding value-based decision making and related neural activities (e.g., dopamine activity). The representation of time in neural processes modeled by a TD framework, however, is poorly understood. To address this issue, we propose a TD formulation that separates the time of the operator (neural valuation processes), which we refer to as internal time, from the time of the observer (experiment), which we refer to as conventional time. We provide the formulation and theoretical characteristics of this TD model based on internal time, called internal-time TD, and explore the possible consequences of the use of this model in neural value-based decision making. Due to the separation of the two times, internal-time TD computations, such as TD error, are expressed differently, depending on both the time frame and time unit. We examine this operator-observer problem in relation to the time representation used in previous TD models. An internal time TD value function exhibits the co-appearance of exponential and hyperbolic discounting at different delays in intertemporal choice tasks. We further examine the effects of internal time noise on TD error, the dynamic construction of internal time, and the modulation of internal time with the internal time hypothesis of serotonin function. We also relate the internal TD formulation to research on interval timing and subjective time.

Hierarchical Interaction Structure of Neural Activities in Cortical Slice Cultures

Recent advances in the analysis of neuronal activities suggest that the instantaneous activity patterns can be mostly explained by considering only first-order and pairwise interactions between recorded elements, i.e., action potentials or local field potentials (LFP), and do not require higher-than-pairwise-order interactions. If generally applicable, this pairwise approach greatly simplifies the description of network interactions. However, an important question remains: are the recorded elements the units of interaction that best describe neuronal activity patterns? To explore this, we recorded spontaneous LFP peak activities in cortical organotypic cultures using planar, integrated 60-microelectrode arrays. We compared predictions obtained using a pairwise approach with those using a hierarchical approach that uses two different spatial units for describing the activity interactions: single electrodes and electrode clusters. In this hierarchical model, short-range interactions within each cluster were modeled by pairwise interactions of electrode activities and long-range interactions were modeled by pairwise interactions of cluster activities. Despite the relatively low number of parameters used, the hierarchical model provided a more accurate description of the activity patterns than the pairwise model when applied to ensembles of 10 electrodes. Furthermore, the hierarchical model was successfully applied to a larger-scale data of ∼60 electrodes. Electrode activities within clusters were highly correlated and spatially contiguous. In contrast, long-range interactions were diffuse, suggesting the presence of higher-than-pairwise-order interactions involved in the LFP peak activities. Thus, the identification of appropriate units of interaction may allow for the successful characterization of neuronal activities in large-scale networks.

A demonstration of the GUINNESS: A GUI based neural NEtwork SyntheSizer for an FPGA

The GUINNESS is a tool flow for the deep neural network toward FPGA implementation [3,4,5] based on the GUI (Graphical User Interface) including both the binarized deep neural network training on GPUs and the inference on an FPGA. It generates the trained the Binarized deep neural network [2] on the desktop PC, then, it generates the bitstream by using standard the FPGA CAD tool flow. All the operation is done on the GUI, thus, the designer is not necessary to write any scripts to descript the neural network structure, training behaviour, only specify the values for hyper parameters. After finished the training, it automatically generates C++ codes to synthesis the bitstream using the Xilinx SDSoC system design tool flow. Thus, our tool flow is suitable for the software programmers who are not familiar with the FPGA design.

Neural correlates of the emulated-other's prediction errors in value-based decision making

The ability to estimate elapse time is crucial to anticipate upcoming events and prepare appropriate actions. In this study, we examined whether this ability improved with training on a reaction-time (RT) task in which human subjects were required to initiate wrist movements quickly after a ‘Go’ signal. The probability that the Go signal would occur (i.e., the hazard rate) increased with the amount of time that elapsed (e.g., 1–2 s) before the onset of the signal (i.e., the foreperiod). We calculated a blurred version of hazard rate (i.e., subjective hazard rate), which was formalized based on the assumption that uncertainty in time estimation scales with time (i.e., Weber’s law). The waveform of the subjective hazard function depended on the Weber fraction for time estimation. Subjects performed the task for 480 trials/day for 12 days. Reaction time decreased as the duration of the foreperiod increased, suggesting that subjects estimated elapsed time and calculated time-dependent probability associated with the Go signal. Training affected the pattern of RT decrease. The decrease became consistent, independent of the foreperiod, throughout training. Reaction time was inversely related to, and well fit by, a weighted sum of subjective hazard rates. The fitting procedure revealed that the consistent decrease in RT involved a decrease in the Weber fraction. The results indicate that the ability to process time improves with experience using elapsed time to anticipate the onset of behaviorally relevant events and to prepare appropriate actions.

Modulation of caudate activity by social dominance

Social dominance is a powerful modulator of behavior in animals that form hierarchical societies, such as monkeys. Social rank significantly affects priority over food so that subordinate monkeys often suppress reaching it when with dominant conspecifics. This behavior requires integrating reward and social rank information to determine motor behavior, but its underlying neural mechanisms are still elusive. The caudate nucleus (CD) of the basal ganglia is known to play a pivotal role in representing reward expectation and translating it to actions. To investigate whether CD activities also encode social rank, we recorded from the CD as monkeys reached for food in a naturalistic setting designed to evoke dominant and subordinate behaviors. A monkey and a human competitor were seated side-by-side facing a round table, on which food pellets were placed at various locations and retrieved by monkey or human. When food was placed between competitors, human behavior was either competitive (actively retrieving the food) or passive, alternating across blocks. During competitive blocks, monkeys exhibited subordinate behavior and suppressed hand movements; otherwise, monkeys exhibited dominant behavior and readily retrieved the food. Analyzing recorded activities, we found a group of CD neurons with activities primarily related to reward expectation, as described in previous studies. We further found another group of CD neurons that had higher activity when the animal was in a dominant state; their activities had significant preference to trials of the non-competitive block, irrespective of the location of food placement, and were not modulated by reward expectation as tested in a control, non-social experiment. These results suggest that the neurons of the second group signal a state of lower social inhibition the dominant state. With these activities, the CD can contribute to determining a final motor behavior based on both reward expectation and social rank information.