Jon C. Horvitz

Cortical and basal ganglia contributions to habit learning and automaticity

In the 20th century it was thought that novel behaviors are mediated primarily in cortex and that the development of automaticity is a process of transferring control to subcortical structures. However, evidence supports the view that subcortical structures, such as the striatum, make significant contributions to initial learning. More recently, there has been increasing evidence that neurons in the associative striatum are selectively activated during early learning, whereas those in the sensorimotor striatum are more active after automaticity has developed. At the same time, other recent reports indicate that automatic behaviors are striatum- and dopamine-independent, and might be mediated entirely within cortex. Resolving this apparent conflict should be a major goal of future research.

Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat.

In light of evidence implicating dopamine in the pathophysiology of attention deficit disorder and schizophrenia, diseases involving attentional or sensory processing abnormalities, it was of interest to determine whether and how dopamine neurons in the ventral tegmental area respond to sensory stimuli. The single-unit responses of ventral tegmental dopamine neurons were recorded in freely-moving cats during the presentation of brief, non-conditioned auditory and visual stimuli. Both auditory and visual stimuli produced neuronal excitation, involving a greater than 5-fold increase in the probability of burst firing followed by a period of burst inhibition. The burst nature of the single-unit response suggests that sensory-induced dopamine release at target sites was disproportionally large relative to the discharge frequency. While characteristics of the dopaminergic sensory response were similar for auditory and visual stimuli, the response latency was longer for visual stimuli. The results demonstrate that dopamine neurons in the ventral tegmental area, the site of origin for mesolimbocortical dopamine neurons, are reliably activated by non-conditioned auditory and visual stimuli.

Dopaminergic Mechanisms in Actions and Habits

Recent studies suggest new ways to interpret dopaminergic actions in goal-directed performance and habitual responding. In the early stages of learning dopamine plays an essential role, but with extended training dopamine appears to play a decreasing role in response expression. Experimental manipulation of dopamine levels alters the correlation of cortical and striatal neural activity in behaving animals, and these dopamine-dependent changes in corticostriatal correlations may be reflected in changes in action selection in the basal ganglia. Consistent with this hypothesis, changes in dopamine signaling brought about by sensitization with amphetamine mimic the transition from goal-directed to habit-based instrumental performance. At the cellular level, dopamine-dependent synaptic plasticity may be important initially, and subsequently lead to more persistent changes that no longer require dopamine. The locus of these actions within the cortical and corticostriatal circuitry is a focus on ongoing research.

Dopamine gating of glutamatergic sensorimotor and incentive motivational input signals to the striatum

Dopamine (DA) neurons of the substantia nigra (SN) and ventral tegmental area (VTA) respond to a wide category of salient stimuli. Activation of SN and VTA DA neurons, and consequent release of nigrostriatal and mesolimbic DA, modulates the processing of concurrent glutamate inputs to dorsal and ventral striatal target regions. According to the view described here, this occurs under conditions of unexpected environmental change regardless of whether that change is rewarding or aversive. Nigrostriatal and mesolimbic DA activity gates the input of sensory, motor, and incentive motivational (e.g. reward) signals to the striatum. In light of recent single-unit and brain imaging data, it is suggested that the striatal reward signals originate in the orbitofrontal cortex and basolateral amygdala (BLA), regions that project strongly to the striatum. A DA signal of salient unexpected event occurrence, from this framework, gates the throughput of the orbitofrontal glutamate reward input to the striatum just as it gates the throughput of corticostriatal sensory and motor signals needed for normal response execution. Processing of these incoming signals is enhanced when synaptic DA levels are high, because DA enhances the synaptic efficacy of strong concurrent glutamate inputs while reducing the efficacy of weak glutamate inputs. The impairments in motor performance and incentive motivational processes that follow from nigrostriatal and mesolimbic DA loss can be understood in terms of a single mechanism: abnormal processing of sensorimotor and incentive motivation-related glutamate input signals to the striatum.

Effects of dopamine antagonists on the timing of two intervals.

Rats were trained on a two-interval (12 and 36 s) temporal production task (the peak procedure). Test sessions were conducted in which either the D(1) antagonist SCH-23390 (SCH; 0.02, 0.04, 0.06 mg/kg) or the D(2) antagonist haloperidol (HAL; 0.05, 0.1, 0.2 mg/kg) were injected prior to testing. Both drugs affected the amount of responding, but only HAL affected timing. Under HAL, both intervals were overestimated, consistent with a HAL-induced decrease in clock speed. Drug-induced decreases in response output were more profound for the long interval than the short. In addition, there was evidence of HAL- and SCH-induced delays in response initiation that were more severe for the long interval, perhaps owing to its status as a weaker conditioned stimulus.

Extended Habit Training Reduces Dopamine Mediation of Appetitive Response Expression

A wide range of behaviors is impaired after disruption of dopamine (DA) transmission, yet behaviors that are reflexive, automatic, or elicited by salient cues often remain intact. Responses triggered by strong external cues appear to be DA independent. Here, we examined the possibility that a single behavior may become DA independent as a result of extended training. Rats were trained to execute a head-entry response to a cue signaling food delivery. Vulnerability of the response to D1 or D2 receptor blockade was assessed on day 3, 7, or 17 of 28-trial-per-day training. During the early stages of training, the D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH 23390) increased response latencies; however, the same behavior was unaffected by SCH 23390 in animals tested during the later stages of training. Other aspects of behavior such as locomotion and head-entry responses during the uncued intertrial interval remained vulnerable to SCH 23390 throughout the experiment. This D1-mediated response was unaffected by the D2 antagonist raclopride, even at a dose that strongly suppressed locomotion. The results provide strong evidence that a D1-dependent behavior becomes less dependent on DA with extended training. A number of fundamental neurobiological changes occur as behaviors become learned habits; at least for some responses, this change involves a shift from D1-mediated to D1-independent responding.

Haloperidol blocks the response-reinstating effects of food reward: A methodology for separating neuroleptic effects on reinforcement and motor processes

To test the hypothesis that dopamine antagonist drugs attenuate the reinforcing properties of food, rats previously trained to traverse a straight runway for food reward subsequently underwent extinction sessions. After running speeds had substantially decreased, rats received a single food-rewarded trial either in the presence or absence of haloperidol (0, 0.15 or 0.30 mg/kg IP). Twenty-four hours later, animals were tested for reinstatement of the running response during a drug-free test trial. Animals that were nondrugged during the food-rewarded trial showed increases in running speed on the test trial relative to extinction baseline speeds. In contrast, animals under the influence of haloperidol during the food-rewarded trial failed to show test day increases in running speed. Additional control groups ruled out the possibility that the haloperidol results were due to either motor or state-dependent learning effects. The findings support the view that dopamine systems play a role in the neural substrates underlying food reinforcement. In addition, the study demonstrates a simple and effective methodology for separating neuroleptic effects on motor and reinforcement processes.

Amphetamine affects the start of responding in the peak interval timing task

In this paper we investigate how amphetamine affects performance in a PI task by comparing two analyses of responding during peak trials. After training on 24 s fixed interval (FI-24) with 96 s peak trials, rats were given amphetamine for 4 consecutive days at doses of .5 and 1.0 mg/kg. Responses during peak trials were fitted with a Gaussian distribution to estimate the expected time of reinforcement from the peak time. A single trials analysis was also performed to determine the start time and stop time of the transition into and out of a high rate of responding on each peak trial. Amphetamine significantly decreased peak times as measured with the Gaussian curve fitting. However, in the single trials analysis, animals initiated responding significantly earlier, but did not stop responding earlier. Thus, fitting a Gaussian to the average performance across trials sometimes provides a different characterization of the timing process than does analyzing the start and stop of responding on individual trials. In the current experiment, the latter approach provided a more precise characterization of the effects of amphetamine on response timing.

Conditioned incentive properties of a food-paired conditioned stimulus remain intact during dopamine receptor blockade.

Hungary rats were exposed to a conditioned stimulus (CS) event (either light onset or offset) before food delivery. After several weeks of contingent CS+/food pairings, animals were pretreated with either 0, 0.5, 0.75, or 1.0 mg/kg pimozide and exposed to the CS+ alone. Both vehicle- and neuroleptic-treated rats showed large elevations in locomotor activity immediately after CS+ presentation, in relation to pre-CS+ activity levels. This elevation in activity was apparently due to the conditioned motivational properties of the stimulus because animals that had previously received unpaired presentations of the CS and food failed to show similar responsiveness to the CS. Although pimozide did not affect responsiveness to the CS+, the neuroleptic did produce an overall suppression of locomotor activity during both pre- and post-CS+ periods. The results suggest that neuroleptic treatment produces a suppression of general activity but leaves the motivational properties of food-paired stimuli intact.

Stimulus-response and response-outcome learning mechanisms in the striatum.

While midbrain DA neurons show phasic activations in response to both reward-predicting and salient non-reward events, activation responses to primary and conditioned rewards are sustained for several hundreds of milliseconds beyond those elicited by salient non-reward-related stimuli. The longer-duration DA reward response and corresponding elevated DA release in striatal target sites may selectively strengthen currently-active corticostriatal synapses, i.e., those associated with the successful reward-procuring behavior. This paper describes how similar models of DA-mediated plasticity of corticostriatal synapses may describe both stimulus-response and response-outcome learning. DA-mediated strengthening of corticostriatal synapses in regions of the dorsolateral striatum receiving afferents from primary sensorimotor cortex is likely to bind corticostriatal inputs representing the previously-emitted movement to striatal outputs contributing to the selection of the next movement segment in a behavioral sequence. Within the striatum, more generally, inputs from distinct regions of the frontal cortex that code independently for movement direction and reward expectation send convergent projections to striatal output cells. DA-mediated strengthening of active corticostriatal synapses promotes the future output of the striatal cell under similar input conditions. This is postulated to promote persistence of neuronal activity in the very cortical cells that drive corticostriatal input, leading to the establishment of sustained reverberatory loops that permit cortical movement-related cells to maintain activity until the appropriate time of movement initiation.