Henry H. Yin

The role of the basal ganglia in habit formation

Many organisms, especially humans, are characterized by their capacity for intentional, goal-directed actions. However, similar behaviours often proceed automatically, as habitual responses to antecedent stimuli. How are goal-directed actions transformed into habitual responses? Recent work combining modern behavioural assays and neurobiological analysis of the basal ganglia has begun to yield insights into the neural basis of habit formation.

Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning

Habits are controlled by antecedent stimuli rather than by goal expectancy. Interval schedules of feedback have been shown to generate habits, as revealed by the insensitivity of behaviour acquired under this schedule to outcome devaluation treatments. Two experiments were conducted to assess the role of the dorsolateral striatum in habit learning. In Experiment 1, sham operated controls and rats with dorsolateral striatum lesions were trained to press a lever for sucrose under interval schedules. After training, the sucrose was devalued by inducing taste aversion to it using lithium chloride, whereas saline injections were given to the controls. Only rats given the devaluation treatment reduced their consumption of sucrose and this reduction was similar in both the sham and the lesioned groups. All rats were then returned to the instrumental chamber for an extinction test, in which the lever was extended but no sucrose was delivered. In contrast to sham operated controls, rats with dorsolateral striatum lesions refrained from pressing the lever if the outcome was devalued. To assess the specificity of the role of dorsolateral striatum in this effect a second experiment was conducted in which a group with lesions of dorsomedial striatum was added. In relation now to both the sham and the dorsomedial lesioned groups, only rats with lesions of dorsolateral striatum significantly reduced responding after outcome devaluation. In conclusion, this study provides direct evidence that the dorsolateral striatum is necessary for habit formation. Furthermore, it suggests that, when the habit system is disrupted, control over instrumental performance reverts to the system controlling the performance of goal‐directed instrumental actions.

Dopaminergic Control of Corticostriatal Long-Term Synaptic Depression in Medium Spiny Neurons Is Mediated by Cholinergic Interneurons

Long-term depression (LTD) of the synapse formed between cortical pyramidal neurons and striatal medium spiny neurons is central to many theories of motor plasticity and associative learning. The induction of LTD at this synapse is thought to depend upon D(2) dopamine receptors localized in the postsynaptic membrane. If this were true, LTD should be inducible in neurons from only one of the two projection systems of the striatum. Using transgenic mice in which neurons that contribute to these two systems are labeled, we show that this is not the case. Rather, in both cell types, the D(2) receptor dependence of LTD induction reflects the need to lower M(1) muscarinic receptor activity-a goal accomplished by D(2) receptors on cholinergic interneurons. In addition to reconciling discordant tracts of the striatal literature, these findings point to cholinergic interneurons as key mediators of dopamine-dependent striatal plasticity and learning.

Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill

The learning of new skills is characterized by an initial phase of rapid improvement in performance and a phase of more gradual improvements as skills are automatized and performance asymptotes. Using in vivo striatal recordings, we observed region-specific changes in neural activity during the different phases of skill learning, with the associative or dorsomedial striatum being preferentially engaged early in training and the sensorimotor or dorsolateral striatum being engaged later in training. Ex vivo recordings from medium spiny striatal neurons in brain slices of trained mice revealed that the changes observed in vivo corresponded to regional- and training-specific changes in excitatory synaptic transmission in the striatum. Furthermore, the potentiation of glutamatergic transmission observed in dorsolateral striatum after extensive training was preferentially expressed in striatopallidal neurons, rather than striatonigral neurons. These findings demonstrate that region- and pathway-specific plasticity sculpts the circuits involved in the performance of the skill as it becomes automatized.

Reward-guided learning beyond dopamine in the nucleus accumbens: The integrative functions of cortico-basal ganglia networks

Here we challenge the view that reward‐guided learning is solely controlled by the mesoaccumbens pathway arising from dopaminergic neurons in the ventral tegmental area and projecting to the nucleus accumbens. This widely accepted view assumes that reward is a monolithic concept, but recent work has suggested otherwise. It now appears that, in reward‐guided learning, the functions of ventral and dorsal striata, and the cortico‐basal ganglia circuitry associated with them, can be dissociated. Whereas the nucleus accumbens is necessary for the acquisition and expression of certain appetitive Pavlovian responses and contributes to the motivational control of instrumental performance, the dorsal striatum is necessary for the acquisition and expression of instrumental actions. Such findings suggest the existence of multiple independent yet interacting functional systems that are implemented in iterating and hierarchically organized cortico‐basal ganglia networks engaged in appetitive behaviors ranging from Pavlovian approach responses to goal‐directed instrumental actions controlled by action‐outcome contingencies.

Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning

Although there is consensus that instrumental conditioning depends on the encoding of action–outcome associations, it is not known where this learning process is localized in the brain. Recent research suggests that the posterior dorsomedial striatum (pDMS) may be the critical locus of these associations. We tested this hypothesis by examining the contribution of N‐methyl‐d‐aspartate receptors (NMDARs) in the pDMS to action–outcome learning. Rats with bilateral cannulae in the pDMS were first trained to perform two actions (left and right lever presses), for sucrose solution. After the pre‐training phase, they were given an infusion of the NMDA antagonist 2‐amino‐5‐phosphonopentanoic acid (APV, 1 mg/mL) or artificial cerebral spinal fluid (ACSF) before a 30‐min session in which pressing one lever delivered food pellets and pressing the other delivered fruit punch. Learning during this session was tested the next day by sating the animals on either the pellets or fruit punch before assessing their performance on the two levers in extinction. The ACSF group selectively reduced responding on the lever that, in training, had earned the now devalued outcome, whereas the APV group did not. Experiment 2 replicated the effect of APV during the critical training session but found no effect of APV given after acquisition and before test. Furthermore, Experiment 3 showed that the effect of APV on instrumental learning was restricted to the pDMS; infusion into the dorsolateral striatum did not prevent learning. These experiments provide the first direct evidence that, in instrumental conditioning, NMDARs in the dorsomedial striatum are involved in encoding action–outcome associations.

Disrupted motor learning and long-term synaptic plasticity in mice lacking NMDAR1 in the striatum

Much research has implicated the striatum in motor learning, but the underlying mechanisms have not been identified. Although NMDA receptor (NMDAR)-dependent long-term potentiation has been observed in the striatum, its involvement in motor learning remains unclear. To examine the role of striatal NMDAR in motor learning, we created striatum-specific NMDAR1 subunit knockout mice, analyzed the striatal anatomy and neuronal morphology of these mice, evaluated their performance on well established motor tasks, and performed electrophysiological recordings to assay striatal NMDAR function and long-term synaptic plasticity. Our results show that deleting the NMDAR1 subunit of the NMDAR specifically in the striatum, which virtually abolished NMDAR-mediated currents, resulted in only small changes in striatal neuronal morphology but severely impaired motor learning and disrupted dorsal striatal long-term potentiation and ventral striatal long-term depression.

Endocannabinoid Signaling is Critical for Habit Formation

Extended training can induce a shift in behavioral control from goal-directed actions, which are governed by action-outcome contingencies and sensitive to changes in the expected value of the outcome, to habits which are less dependent on action-outcome relations and insensitive to changes in outcome value. Previous studies in rats have shown that interval schedules of reinforcement favor habit formation while ratio schedules favor goal-directed behavior. However, the molecular mechanisms underlying habit formation are not well understood. Endocannabinoids, which can function as retrograde messengers acting through presynaptic CB1 receptors, are highly expressed in the dorsolateral striatum, a key region involved in habit formation. Using a reversible devaluation paradigm, we confirmed that in mice random interval schedules also favor habit formation compared with random ratio schedules. We also found that training with interval schedules resulted in a preference for exploration of a novel lever, whereas training with ratio schedules resulted in less generalization and more exploitation of the reinforced lever. Furthermore, mice carrying either a heterozygous or a homozygous null mutation of the cannabinoid receptor type I (CB1) showed reduced habit formation and enhanced exploitation. The impaired habit formation in CB1 mutant mice cannot be attributed to chronic developmental or behavioral abnormalities because pharmacological blockade of CB1 receptors specifically during training also impairs habit formation. Taken together our data suggest that endocannabinoid signaling is critical for habit formation.

Ethanol reverses the direction of long-term synaptic plasticity in the dorsomedial striatum

The striatum is a critical structure for the control of voluntary behaviour, and striatal synaptic plasticity has been implicated in instrumental learning. As ethanol consumption can cause impairments in cognition, learning, and action selection, it is important to understand the effects of this drug on striatal function. In this study we examined the effects of ethanol on long‐term synaptic plasticity in the dorsomedial striatum (DMS), a striatal subregion that plays a central role in the acquisition and selection of goal‐directed actions. Ethanol was found to impair N‐methyl‐d‐aspartic acid receptor (NMDAR)‐dependent long‐term potentiation (LTP) dose‐dependently in the DMS, and to promote long‐term depression (LTD) at the highest concentration (50 mm) used. These results suggest that ethanol, at a concentration usually associated with mild intoxication, could significantly change experience‐dependent modification of corticostriatal circuits underlying the learning of goal‐directed instrumental actions.

A wireless multi-channel recording system for freely behaving mice and rats.

To understand the neural basis of behavior, it is necessary to record brain activity in freely moving animals. Advances in implantable multi-electrode array technology have enabled researchers to record the activity of neuronal ensembles from multiple brain regions. The full potential of this approach is currently limited by reliance on cable tethers, with bundles of wires connecting the implanted electrodes to the data acquisition system while impeding the natural behavior of the animal. To overcome these limitations, here we introduce a multi-channel wireless headstage system designed for small animals such as rats and mice. A variety of single unit and local field potential signals were recorded from the dorsal striatum and substantia nigra in mice and the ventral striatum and prefrontal cortex simultaneously in rats. This wireless system could be interfaced with commercially available data acquisition systems, and the signals obtained were comparable in quality to those acquired using cable tethers. On account of its small size, light weight, and rechargeable battery, this wireless headstage system is suitable for studying the neural basis of natural behavior, eliminating the need for wires, commutators, and other limitations associated with traditional tethered recording systems.