Shelly B. Flagel

A selective role for dopamine in stimulus-reward learning

Individuals make choices and prioritize goals using complex processes that assign value to rewards and associated stimuli. During Pavlovian learning, previously neutral stimuli that predict rewards can acquire motivational properties, becoming attractive and desirable incentive stimuli. However, whether a cue acts solely as a predictor of reward, or also serves as an incentive stimulus, differs between individuals. Thus, individuals vary in the degree to which cues bias choice and potentially promote maladaptive behaviour. Here we use rats that differ in the incentive motivational properties they attribute to food cues to probe the role of the neurotransmitter dopamine in stimulus–reward learning. We show that intact dopamine transmission is not required for all forms of learning in which reward cues become effective predictors. Rather, dopamine acts selectively in a form of stimulus–reward learning in which incentive salience is assigned to reward cues. In individuals with a propensity for this form of learning, reward cues come to powerfully motivate and control behaviour. This work provides insight into the neurobiology of a form of stimulus–reward learning that confers increased susceptibility to disorders of impulse control.

Neuroendocrine and psychophysiologic responses in PTSD : A symptom provocation study

Biological research on post-traumatic stress disorder (PTSD) has focused on autonomic, sympatho-adrenal, and hypothalamo-pituitary-adrenal (HPA) axis systems. Interactions among these response modalities have not been well studied and may be illuminating. We examined subjective, autonomic, adrenergic, and HPA axis responses in a trauma-cue paradigm and explored the hypothesis that the ability of linked stress-response systems to mount integrated responses to environmental threat would produce strong correlations across systems. Seventeen veterans with PTSD, 11 veteran controls without PTSD, and 14 nonveteran controls were exposed to white noise and combat sounds on separate days. Subjective distress, heart rate, skin conductance, plasma catecholamines, ACTH, and cortisol, at baseline and in response to the auditory stimuli, were analyzed for group differences and for patterns of interrelationships. PTSD patients exhibited higher skin conductance, heart rate, plasma cortisol, and catecholamines at baseline, and exaggerated responses to combat sounds in skin conductance, heart rate, plasma epinephrine, and norepinephrine, but not ACTH. The control groups did not differ on any measure. In canonical correlation analyses, no significant correlations were found between response systems. Thus, PTSD patients showed heightened responsivity to trauma-related cues in some, but not all, response modalities. The data did not support the integrated, multisystem stress response in PTSD that had been hypothesized. Individual response differences or differing pathophysiological processes may determine which neurobiological system is affected in any given patient.

An animal model of genetic vulnerability to behavioral disinhibition and responsiveness to reward-related cues: implications for addiction.

Rats selectively bred based on high or low reactivity to a novel environment were characterized for other behavioral and neurobiological traits thought to be relevant to addiction vulnerability. The two lines of animals, which differ in their propensity to self-administer drugs, also differ in the value they attribute to cues associated with reward, in impulsive behavior, and in their dopamine system. When a cue was paired with food or cocaine reward bred high-responder rats (bHRs) learned to approach the cue, whereas bred low-responder rats (bLRs) learned to approach the location of food delivery, suggesting that bHRs but not bLRs attributed incentive value to the cue. Moreover, although less impulsive on a measure of ‘impulsive choice’, bHRs were more impulsive on a measure of ‘impulsive action’— ie, they had difficulty withholding an action to receive a reward, indicative of ‘behavioral disinhibition’. The dopamine agonist quinpirole caused greater psychomotor activation in bHRs relative to bLRs, suggesting dopamine supersensitivity. Indeed, relative to bLRs, bHRs also had a greater proportion of dopamine D2high receptors, the functionally active form of the receptor, in the striatum, in spite of lower D2 mRNA levels and comparable total D2 binding. In addition, fast-scan cyclic voltammetry revealed that bHRs had more spontaneous dopamine ‘release events’ in the core of the nucleus accumbens than bLRs. Thus, bHRs exhibit parallels to ‘externalizing disorders’ in humans, representing a genetic animal model of addiction vulnerability associated with a propensity to attribute incentive salience to reward-related cues, behavioral disinhibition, and increased dopaminergic ‘tone.’

Quantifying Individual Variation in the Propensity to Attribute Incentive Salience to Reward Cues

If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties (“incentive salience”) to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue (“sign-trackers”) vs. others that approached the location of reward delivery (“goal-trackers”). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.

Persistent Alterations in Cognitive Function and Prefrontal Dopamine D2 Receptors Following Extended, but Not Limited, Access to Self-Administered Cocaine

Drug addicts have deficits in frontocortical function and cognition even long after the discontinuation of drug use. It is not clear, however, whether the cognitive deficits are a consequence of drug use, or are present prior to drug use, and thus are a potential predisposing factor for addiction. To determine if self-administration of cocaine is capable of producing long-lasting alterations in cognition, rats were allowed access to cocaine for either 1 h/day (short access, ShA) or 6 h/day (long access, LgA) for 3 weeks. Between 1 and 30 days after the last self-administration session, we examined performance on a cognitively demanding test of sustained attention that requires an intact medial prefrontal cortex. The expression levels of dopamine D1 and D2 receptor mRNA and D2 protein in the prefrontal cortex were also examined. Early after discontinuation of drug use, LgA (but not ShA) animals were markedly impaired on the sustained attention task. Although the LgA animals improved over time, they continued to show a persistent pattern of performance deficits indicative of a disruption of cognitive flexibility up to 30 days after the discontinuation of drug use. This was accompanied by a significant decrease in DA D2 (but not D1) mRNA in the medial and orbital prefrontal cortex, and D2 receptor protein in the medial prefrontal cortex of LgA (but not ShA) animals. These findings establish that repeated cocaine use is capable of producing persistent alterations in the prefrontal cortex and in cognitive function, and illustrate the usefulness of extended access self-administration procedures for studying the neurobiology of addiction.

Cocaine self-administration produces a persistent increase in dopamine D2High receptors

Cocaine addicts are reported to have decreased numbers of striatal dopamine D2 receptors. However, in rodents, repeated cocaine administration consistently produces hypersensitivity to the psychomotor activating effects of both indirect dopamine agonists, such as cocaine itself, and importantly, to direct-acting D2 receptor agonists. The current study reports a possible resolution to this long-standing paradox. The dopamine D2 receptor exists in both a low and a high-affinity state, and dopamine exerts its effects via the more functionally relevant high-affinity D2 receptor (D2 High). We report here that cocaine self-administration experience produces a large (approximately 150%) increase in the proportion of D2 High receptors in the striatum with no change in the total number of D2 receptors, and this effect is evident both 3 and 30 days after the discontinuation of cocaine self-administration. Changes in D2 High receptors would not be evident with the probes used in human (and non-human primate) imaging studies. We suggest, therefore, that cocaine addicts and animals previously treated with cocaine may be hyper-responsive to dopaminergic drugs in part because an increase in D2 High receptors results in dopamine supersensitivity. This may also help explain why stimuli that increase dopamine neurotransmission, including drugs themselves, are so effective in producing relapse in individuals with a history of exposure to cocaine.

Variation in the form of Pavlovian conditioned approach behavior among outbred male Sprague-Dawley rats from different vendors and colonies: sign-tracking vs. goal-tracking.

Even when trained under exactly the same conditions outbred male Sprague-Dawley (SD) rats vary in the form of the Pavlovian conditioned approach response (CR) they acquire. The form of the CR (i.e. sign-tracking vs. goal-tracking) predicts to what degree individuals attribute incentive salience to cues associated with food or drugs. However, we have noticed variation in the incidence of these two phenotypes in rats obtained from different vendors. In this study, we quantified sign- and goal-tracking behavior in a reasonably large sample of SD rats obtained from two vendors (Harlan or Charles River), as well as from individual colonies operated by both vendors. Our sample of rats acquired from Harlan had, on average, more sign-trackers than goal-trackers, and vice versa for our sample of rats acquired from Charles River. Furthermore, there were significant differences among colonies of the same vendor. Although it is impossible to rule out environmental variables, SD rats at different vendors and barriers may have reduced phenotypic heterogeneity as a result of genetic variables, such as random genetic drift or population bottlenecks. Consistent with this hypothesis, we identified marked population structure among colonies from Harlan. Therefore, despite sharing the same name, investigators should be aware that important genetic and phenotypic differences exist among SD rats from different vendors or even from different colonies of the same vendor. If used judiciously this can be an asset to experimental design, but it can also be a pitfall for those unaware of the issue.

Modelling Individual Differences in the Form of Pavlovian Conditioned Approach Responses: A Dual Learning Systems Approach with Factored Representations

Reinforcement Learning has greatly influenced models of conditioning, providing powerful explanations of acquired behaviour and underlying physiological observations. However, in recent autoshaping experiments in rats, variation in the form of Pavlovian conditioned responses (CRs) and associated dopamine activity, have questioned the classical hypothesis that phasic dopamine activity corresponds to a reward prediction error-like signal arising from a classical Model-Free system, necessary for Pavlovian conditioning. Over the course of Pavlovian conditioning using food as the unconditioned stimulus (US), some rats (sign-trackers) come to approach and engage the conditioned stimulus (CS) itself – a lever – more and more avidly, whereas other rats (goal-trackers) learn to approach the location of food delivery upon CS presentation. Importantly, although both sign-trackers and goal-trackers learn the CS-US association equally well, only in sign-trackers does phasic dopamine activity show classical reward prediction error-like bursts. Furthermore, neither the acquisition nor the expression of a goal-tracking CR is dopamine-dependent. Here we present a computational model that can account for such individual variations. We show that a combination of a Model-Based system and a revised Model-Free system can account for the development of distinct CRs in rats. Moreover, we show that revising a classical Model-Free system to individually process stimuli by using factored representations can explain why classical dopaminergic patterns may be observed for some rats and not for others depending on the CR they develop. In addition, the model can account for other behavioural and pharmacological results obtained using the same, or similar, autoshaping procedures. Finally, the model makes it possible to draw a set of experimental predictions that may be verified in a modified experimental protocol. We suggest that further investigation of factored representations in computational neuroscience studies may be useful.

The role of learning-related dopamine signals in addiction vulnerability

Dopaminergic signals play a mathematically precise role in reward-related learning, and variations in dopaminergic signaling have been implicated in vulnerability to addiction. Here, we provide a detailed overview of the relationship between theoretical, mathematical, and experimental accounts of phasic dopamine signaling, with implications for the role of learning-related dopamine signaling in addiction and related disorders. We describe the theoretical and behavioral characteristics of model-free learning based on errors in the prediction of reward, including step-by-step explanations of the underlying equations. We then use recent insights from an animal model that highlights individual variation in learning during a Pavlovian conditioning paradigm to describe overlapping aspects of incentive salience attribution and model-free learning. We argue that this provides a computationally coherent account of some features of addiction.

Quantifying the psychomotor activating effects of cocaine in the rat

Studies that involve analysis of the psychomotor activating effects of drugs often use locomotor activity as the sole measure of psychomotor activation. At low doses, psychostimulant drugs typically produce primarily locomotor hyperactivity. As dose is increased, behavior, however, changes in complex ways, in part because of a transition to behavior progressively dominated by more and more stereotyped actions, such as repetitive head movements. Thus, at some doses an increase in a drug effect is reflected by an increase in locomotion and at others by a decrease, making the interpretation of changes in locomotor activity difficult. Using an automated video analysis system (Clever Sys., Inc. Reston, Virginia, USA), we quantified various components of the psychomotor response to cocaine in the rat, including locomotor activity and lateral head movements, as well as the velocity and/or frequency of these behaviors. We report that the combination of these measures provides an especially sensitive measure of the psychomotor activating effects of cocaine, and how behavior changes as a consequence of repeated drug treatment.