Takemasa Satoh

Dopamine neurons learn to encode the long-term value of multiple future rewards

Midbrain dopamine neurons signal reward value, their prediction error, and the salience of events. If they play a critical role in achieving specific distant goals, long-term future rewards should also be encoded as suggested in reinforcement learning theories. Here, we address this experimentally untested issue. We recorded 185 dopamine neurons in three monkeys that performed a multistep choice task in which they explored a reward target among alternatives and then exploited that knowledge to receive one or two additional rewards by choosing the same target in a set of subsequent trials. An analysis of anticipatory licking for reward water indicated that the monkeys did not anticipate an immediately expected reward in individual trials; rather, they anticipated the sum of immediate and multiple future rewards. In accordance with this behavioral observation, the dopamine responses to the start cues and reinforcer beeps reflected the expected values of the multiple future rewards and their errors, respectively. More specifically, when monkeys learned the multistep choice task over the course of several weeks, the responses of dopamine neurons encoded the sum of the immediate and expected multiple future rewards. The dopamine responses were quantitatively predicted by theoretical descriptions of the value function with time discounting in reinforcement learning. These findings demonstrate that dopamine neurons learn to encode the long-term value of multiple future rewards with distant rewards discounted.

What does the habenula tell dopamine neurons

Reward-related events activate dopamine and other neurons in many brain areas. A report in Nature, however, now suggests that neurons in the lateral habenula signal to dopamine neurons when no reward is expected.

Cortical activity regulates corticothalamic synapses in dorsal lateral geniculate nucleus of rats.

In the visual system, the afferent axons from the dorsal lateral geniculate nucleus (dLGN) to the primary visual cortex (V1) show significant activity-dependent plasticity in early postnatal life. To determine whether activity-dependent plasticity operates also in feedback projections from V1 to dLGN, we inactivated cortical inputs pharmacologically and examined possible changes in the density of synaptic proteins, vesicular glutamate transporter 1 (VGluT1) and type 1 metabotropic glutamate receptor alpha (mGluR1alpha), which locate pre- and postsynaptically at feedback projections, respectively in dLGN of rats. The intensity of the immunohistochemical signal of mGluR1alpha in dLGN significantly decreased following the cortical inactivation for at least 2 days, and the decrease was maintained under cortical inactivation until 28 days. On the other hand, the signal intensity of VGluT1 showed a significant increase following 14 or 28 days of cortical inactivation. In adult rats, however, we found no significant change in VGluT1 signal intensity and only a small and transient downregulation of mGluR1alpha following 7-day inactivation. Thus, the decrease in presynaptic activity induces a rapid downregulation of postsynaptic mGluR1alpha followed by a delayed upregulation of presynaptic VGluT1 in young rats. These results suggest that feedback synapses are regulated by neural activity during development.

In vivo electroporation to physiologically identified deep brain regions in postnatal mammals

Genetic manipulation is widely used to research the central nervous system (CNS). The manipulation of molecular expression in a small number of neurons permits the detailed investigation of the role of specific molecules on the function and morphology of the neurons. Electroporation is a broadly used technique for gene transfer in the CNS. However, the targeting of gene transfer using electroporation in postnatal animals was restricted to the cortex, hippocampus, or the region facing the ventricle in previous reports. Electroporation targeting of deep brain structures, such as the thalamus, has been difficult. We introduce a novel electroporation technique that enables gene transfer to a physiologically identified deep brain region using a glass pipette. We recorded neural activity in young-adult mice to identify the location of the lateral geniculate nucleus (LGN) of the thalamus, using a glass pipette electrode containing the plasmid DNA encoding enhanced green fluorescent protein (EGFP). The location of the LGN was confirmed by monitoring visual responses, and the plasmid solution was pressure-injected into the recording site. Voltage pulses were delivered through the glass pipette electrode. Several EGFP-labeled somata and dendrites were observed in the LGN after a few weeks, and labeled axons were found in the visual cortex. The EGFP-expressing structures were observed in detail sufficient to reconstruct their morphology in three dimensions. We further confirmed the applicability of this technique in cats. This method should be useful for the transfer of various genes into cells in physiologically identified brain regions in rodents and gyrencephalic mammals.

Dopamine neurons encode teaching signals for learning reward-based decision strategy

Abstract Although a critical involvement of dopamine has been implicated in the functions of the basal ganglia, especially in learning, little is known about its mechanisms. In two macaque monkeys, the nigrostriate dopamine system was unilaterally depleted by neurotoxin, 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP). We asked the monkeys to learn sequential button press tasks. In addition to the slowness of movement, they showed specific deficits in learning action strategy and reward prediction when using contralateral arm to the dopamine depletion, but with ipsilateral arm, they learned the tasks efficiently. Activity of single, midbrain dopamine neurons was recorded from two monkeys in a task in which they chose one correct, rewarding button among three alternatives in a trial and error basis. It was found that the magnitudes of responses to beep sound informing correct and incorrect choices precisely reward prediction error signals. We present a hypothetical scheme for the mechanisms of action learning in the basal ganglia, in which the nigrostriate dopamine system provides the striatum, locus of learning, with reward prediction errors as a teaching signal for learning, and excitability of the striate projection neurons to specific cortical inputs is modified by the teaching signal so as to achieve goal-directed action selection.

Identification of NeuN immunopositive cells in the adult mouse subventricular zone

In the adult rodent subventricular zone (SVZ), there are neural stem cells (NSCs) and the specialized neurogenic niche is critical to maintain their stemness. To date, many cellular and noncellular factors that compose the neurogenic niche and markers to identify subpopulations of Type A cells have been confirmed. In particular, neurotransmitters regulate adult neurogenesis and mature neurons in the SVZ have been only partially analyzed. Moreover, Type A cells, descendants of NSCs, are highly heterogeneous and more molecular markers are still needed to identify them. In the present study, we systematically classified NeuN, commonly used as a marker of mature and immature post‐mitotic neurons, immunopositive (+) cells within the adult mouse SVZ. These SVZ‐NeuN+ cells (SVZ‐Ns) were mainly classified into two types. One was mature SVZ‐Ns (M‐SVZ‐Ns). Neurochemical properties of M‐SVZ‐Ns were similar to those of striatal neurons, but their birth date and morphology were different. M‐SVZ‐Ns were generated during embryonic and early postnatal stages with bipolar peaks and extended their processes along the wall of the lateral ventricle. The second type was small SVZ‐Ns (S‐SVZ‐Ns) with features of Type A cells. They expressed not only markers of Type A cells, but also proliferated and migrated from the SVZ to the olfactory bulb. Furthermore, S‐SVZ‐Ns could be classified into two types by their spatial locations and glutamic acid decarboxylase 67 expression. Our data indicate that M‐SVZ‐Ns are a new component of the neurogenic niche and S‐SVZ‐Ns are newly identified subpopulations of Type A cells.

Involvement of the Basal Ganglia and Dopamine System in Learning and Execution of Goal-Directed Behavior

To perform any task of our own volition, we execute multiple movements in a specific order based on the likelihood of obtaining a successful outcome. Neurons in the supplementary motor area (SMA), pre-SMA and those in the basal ganglia encode the temporal order, or the sequence of movements used in the tasks (Mushiake and Strick, 1995; Kermadi and Joseph, 1995; Nakamura et al., 1998; Shima and Tanji, 1998, 2000). These neurons must play a crucial role in the mechanisms of planning and execution of temporally organized multiple movements, action.

Activity of midbrain dopamine neurons during learning sequential motor tasks in monkeys

By performing optical measurement in the rat brain slices, we have previously reported that a part of deep layers (layer V/VI) of perirhinal cortex (PC) near the border of entorhinal cortex (EC) and PC plays a key role for the neural activity to propagate from the EC to PC. To reveal the functional organization of this region, electrophysiological properties of neurons were studied with intracellular recording techniques. Thereby three types of cells were observed. This classification of neurons was derived from the studies in the neocortex by McCormic. That is, three types of cells were termed as a regular spiking (RS), a bursting (BS), and a fast spiking (FS) cell. In both RS and BS cell, the fast component of IPSP evoked by electrical stimulation to deep layer of EC was completely blocked by 5pM bicuculline. RS cells were bicuculline sensitive, and generated the seizure like firing. On the other hand. the BS cells were insensitive for bicucullinc,