Rudolf N. Cardinal

Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex.

Emotions are multifaceted, but a key aspect of emotion involves the assessment of the value of environmental stimuli. This article reviews the many psychological representations, including representations of stimulus value, which are formed in the brain during Pavlovian and instrumental conditioning tasks. These representations may be related directly to the functions of cortical and subcortical neural structures. The basolateral amygdala (BLA) appears to be required for a Pavlovian conditioned stimulus (CS) to gain access to the current value of the specific unconditioned stimulus (US) that it predicts, while the central nucleus of the amygdala acts as a controller of brainstem arousal and response systems, and subserves some forms of stimulus-response Pavlovian conditioning. The nucleus accumbens, which appears not to be required for knowledge of the contingency between instrumental actions and their outcomes, nevertheless influences instrumental behaviour strongly by allowing Pavlovian CSs to affect the level of instrumental responding (Pavlovian-instrumental transfer), and is required for the normal ability of animals to choose rewards that are delayed. The prelimbic cortex is required for the detection of instrumental action-outcome contingencies, while insular cortex may allow rats to retrieve the values of specific foods via their sensory properties. The orbitofrontal cortex, like the BLA, may represent aspects of reinforcer value that govern instrumental choice behaviour. Finally, the anterior cingulate cortex, implicated in human disorders of emotion and attention, may have multiple roles in responding to the emotional significance of stimuli and to errors in performance, preventing responding to inappropriate stimuli.

Contrasting Roles of Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice

The orbitofrontal cortex (OFC) and basolateral nucleus of the amygdala (BLA) share many reciprocal connections, and a functional interaction between these regions is important in controlling goal-directed behavior. However, their relative roles have proved hard to dissociate. Although injury to these brain regions can cause similar effects, it has been suggested that the resulting impairments arise through damage to different, yet converging, cognitive processes. Patients with OFC or amygdala lesions exhibit maladaptive decision making and aberrant social behavior often described as impulsive. Impulsive choice may be measured in both humans and rodents by evaluating intolerance to delay of reinforcement. Rats with excitotoxic lesions of the BLA and OFC were tested on such a delay-discounting procedure. Although lesions of the BLA increased choice of the small immediate reward, indicating greater impulsivity, OFC lesions had the opposite effect, increasing preference for the larger but delayed reward. The fact that the delay did not devalue the large reward to such an extent in OFC-lesioned animals supports the suggestion that the OFC is involved in updating the incentive value of outcomes in response to devaluation. In contrast, the BLA-lesioned animals markedly decreased their preference for the large reward when it was delayed, potentially because of an inability to maintain a representation of the reward in its absence. This is the first time that lesions to these two structures have produced opposite behavioral effects, indicating their distinct contributions to cognition.

Neural systems implicated in delayed and probabilistic reinforcement

This review considers the theoretical problems facing agents that must learn and choose on the basis of reward or reinforcement that is uncertain or delayed, in implicit or procedural (stimulus-response) representational systems and in explicit or declarative (action-outcome-value) representational systems. Individual differences in sensitivity to delays and uncertainty may contribute to impulsivity and risk taking. Learning and choice with delayed and uncertain reinforcement are related but in some cases dissociable processes. The contributions to delay and uncertainty discounting of neuromodulators including serotonin, dopamine, and noradrenaline, and of specific neural structures including the nucleus accumbens core, nucleus accumbens shell, orbitofrontal cortex, basolateral amygdala, anterior cingulate cortex, medial prefrontal (prelimbic/infralimbic) cortex, insula, subthalamic nucleus, and hippocampus are examined.

Limbic Corticostriatal Systems and Delayed Reinforcement

Abstract: Impulsive choice, one aspect of impulsivity, is characterized by an abnormally high preference for small, immediate rewards over larger delayed rewards, and can be a feature of adolescence, but also attention‐deficit/hyperactivity disorder (ADHD), addiction, and other neuropsychiatric disorders. Both the serotonin and dopamine neuromodulator systems are implicated in impulsivity; manipulations of these systems affect animal models of impulsive choice, though these effects may depend on the receptor subtype and whether or not the reward is signaled. These systems project to limbic cortical and striatal structures shown to be abnormal in animal models of ADHD. Damage to the nucleus accumbens core (AcbC) causes rats to exhibit impulsive choice. These rats are also hyperactive, but are unimpaired in tests of visuospatial attention; they may therefore represent an animal model of the hyperactive‐impulsive subtype of ADHD. Lesions to the anterior cingulate or medial prefrontal cortex, two afferents to the AcbC, do not induce impulsive choice, but lesions of the basolateral amygdala do, while lesions to the orbitofrontal cortex have had opposite effects in different tasks measuring impulsive choice. In theory, impulsive choice may emerge as a result of abnormal processing of the magnitude of rewards, or as a result of a deficit in the effects of delayed reinforcement. Recent evidence suggests that AcbC‐lesioned rats perceive reward magnitude normally, but exhibit a selective deficit in learning instrumental responses using delayed reinforcement, suggesting that the AcbC is a reinforcement learning system that mediates the effects of delayed rewards.

Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function.

The involvement of mesoaccumbens dopamine in adaptive learning and behaviour is unclear. For example, dopamine may act as a teaching signal to enable learning, or more generally modulate the behavioural expression, or selection, of an already-learned response. The present study investigated the involvement of the mesoaccumbens dopamine system in a fundamental form of learning: Pavlovian conditioning. In this case, the temporal association of a previously neutral visual stimulus and a biologically significant unconditioned stimulus (US), subsequently led to the production of the conditioned response (CR) of discriminated approach behaviour directed toward the conditioned stimulus (CS+), relative to a control (CS-) stimulus. 6-hydroxydopamine lesions of the nucleus accumbens (NAcc), leading to approximately 80% reductions in tissue dopamine, were made at varying time points in four experimental groups of rats, either before or subsequent to the acquisition of the CR. NAcc dopamine depletion produced long-term neuroadaptations in dopamine function 2 months after surgery, and profoundly impaired discriminated Pavlovian approach regardless of when the lesion was made. Thus, NAcc dopamine not only plays a role in conditioned behavioural activation, but also in making the appropriate discriminated response i.e. the direction of response. Further, acquisition lesions produced a far greater impact on discriminated approach than performance lesions. This difference in lesion-induced impairment implies that mesoaccumbens dopamine may play differential roles in the learning and performance of preparatory Pavlovian conditioning.

Neural and psychological mechanisms underlying appetitive learning: links to drug addiction.

The complexity of drug addiction mirrors the complexity of the psychological processes that motivate animals to work for any reinforcer, be it a natural reward or a drug. Here, we review the role of the nucleus accumbens, together with its dopaminergic and cortical innervation, in responding to reinforcement. One important contribution made by the nucleus accumbens is to the process through which neutral stimuli, once paired with a reinforcer such as a drug, have the capacity to motivate behaviour. This process may be one of several contributing to addiction, and it may be amenable to pharmacological intervention.

Effects of Selective Excitotoxic Lesions of the Nucleus Accumbens Core, Anterior Cingulate Cortex, and Central Nucleus of the Amygdala on Autoshaping Performance in Rats

The nucleus accumbens core (AcbC), anterior cingulate cortex (ACC), and central nucleus of the amygdala (CeA) are required for normal acquisition of tasks based on stimulus-reward associations. However, it is not known whether they are involved purely in the learning process or are required for behavioral expression of a learned response. Rats were trained preoperatively on a Pavlovian autoshaping task in which pairing a visual conditioned stimulus (CS+) with food causes subjects to approach the CS+ while not approaching an unpaired stimulus (CS-). Subjects then received lesions of the AcbC, ACC, or CeA before being retested. AcbC lesions severely impaired performance; lesioned subjects approached the CS+ significantly less often than controls, failing to discriminate between the CS+ and CS-. ACC lesions also impaired performance but did not abolish discrimination entirely. CeA lesions had no effect on performance. Thus, the CeA is required for learning, but not expression, of a conditioned approach response, implying that it makes a specific contribution to the learning of stimulus-reward associations.

Decision making and neuropsychiatry

Abnormal decision making is a central feature of neuropsychiatric disorders. Recent investigations of the neural substrates underlying decision making have involved qualitative assessment of the cognition of decision making in clinical lesion studies (in patients with frontal lobe dementia) and neuropsychiatric disorders such as mania, substance abuse and personality disorders. A neural network involving the orbitofrontal cortex, ventral striatum and modulatory ascending neurotransmitter systems has been identified as having a fundamental role in decision making and in the neural basis of neuropsychiatric diseases. This network accounts for the dissociations among decision-making deficits in different clinical populations. Ultimately, a more refined and sophisticated characterization of such deficits might guide the early diagnosis and cognitive and therapeutic rehabilitation of these patients.

Hippocampal lesions facilitate instrumental learning with delayed reinforcement but induce impulsive choice in rats

BackgroundAnimals must frequently act to influence the world even when the reinforcing outcomes of their actions are delayed. Learning with action-outcome delays is a complex problem, and little is known of the neural mechanisms that bridge such delays. When outcomes are delayed, they may be attributed to (or associated with) the action that caused them, or mistakenly attributed to other stimuli, such as the environmental context. Consequently, animals that are poor at forming context-outcome associations might learn action-outcome associations better with delayed reinforcement than normal animals. The hippocampus contributes to the representation of environmental context, being required for aspects of contextual conditioning. We therefore hypothesized that animals with hippocampal lesions would be better than normal animals at learning to act on the basis of delayed reinforcement. We tested the ability of hippocampal-lesioned rats to learn a free-operant instrumental response using delayed reinforcement, and what is potentially a related ability – the ability to exhibit self-controlled choice, or to sacrifice an immediate, small reward in order to obtain a delayed but larger reward.ResultsRats with sham or excitotoxic hippocampal lesions acquired an instrumental response with different delays (0, 10, or 20 s) between the response and reinforcer delivery. These delays retarded learning in normal rats. Hippocampal-lesioned rats responded slightly less than sham-operated controls in the absence of delays, but they became better at learning (relative to shams) as the delays increased; delays impaired learning less in hippocampal-lesioned rats than in shams. In contrast, lesioned rats exhibited impulsive choice, preferring an immediate, small reward to a delayed, larger reward, even though they preferred the large reward when it was not delayed.ConclusionThese results support the view that the hippocampus hinders action-outcome learning with delayed outcomes, perhaps because it promotes the formation of context-outcome associations instead. However, although lesioned rats were better at learning with delayed reinforcement, they were worse at choosing it, suggesting that self-controlled choice and learning with delayed reinforcement tax different psychological processes.

Explaining the escalation of drug use in substance dependence: models and appropriate animal laboratory tests.

Escalation of drug use, a hallmark of drug dependence, has traditionally been interpreted as reflecting the development of tolerance to the drug’s effects. However, on the basis of animal behavioral data, several groups have recently proposed alternative explanations, i.e. that such an escalation of drug use might not be based on (1) tolerance, but rather be indicative of (2) sensitization to the drug’s reinforcing effect, (3) reward allostasis, (4) an increase in the incentive salience of drug-associated stimuli, (5) an increase in the reinforcing strength of the drug reinforcer relative to alternative reinforcers, or (6) habit formation. From the pharmacological perspective, models 1–3 allow predictions about the change in the shape of drug dose-effect curves that are based on mathematically defined models governing receptor-ligand interaction and signal transduction. These predictions are tested in the present review, which also describes the other currently championed models for drug use escalation and other components of apparent ‘reinforcement’ (in its original meaning, like ‘tolerance’ or ‘sensitization’, a purely descriptive term). It evaluates the animal experimental approaches employed to support or prove the existence of each of the models and reinforcement components, and recapitulates the clinical evidence, which strongly suggests that escalation of drug use is predominantly based on an increase in the frequency of intoxication events rather than an increase in the dose taken at each intoxication event. Two apparent discrepancies in animal experiments are that (a) sensitization to overall reinforcement has been found more often for psychostimulants than for opioids, and that (b) tolerance to the reinforcing and other effects has been observed more often for opioids than for cocaine. These discrepancies are resolved by the finding that cocaine levels seem to be more tightly regulated at submaximum reinforcing levels than opioid levels are. Consequently, animals self-administering opioids are more likely to expose themselves to higher above-threshold doses than animals self-administering psychostimulants, rendering the development of tolerance to opioids more likely than tolerance to psychostimulants. The review concludes by making suggestions on how to improve the current behavioral experimental approaches.