Donna J. Calu

Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards.

The dopamine system is thought to be involved in making decisions about reward. Here we recorded from the ventral tegmental area in rats learning to choose between differently delayed and sized rewards. As expected, the activity of many putative dopamine neurons reflected reward prediction errors, changing when the value of the reward increased or decreased unexpectedly. During learning, neural responses to reward in these neurons waned and responses to cues that predicted reward emerged. Notably, this cue-evoked activity varied with size and delay. Moreover, when rats were given a choice between two differently valued outcomes, the activity of the neurons initially reflected the more valuable option, even when it was not subsequently selected.

Opiate versus psychostimulant addiction: the differences do matter

The publication of the psychomotor stimulant theory of addiction in 1987 and the finding that addictive drugs increase dopamine concentrations in the rat mesolimbic system in 1988 have led to a predominance of psychobiological theories that consider addiction to opiates and addiction to psychostimulants as essentially identical phenomena. Indeed, current theories of addiction — hedonic allostasis, incentive sensitization, aberrant learning and frontostriatal dysfunction — all argue for a unitary account of drug addiction. This view is challenged by behavioural, cognitive and neurobiological findings in laboratory animals and humans. Here, we argue that opiate addiction and psychostimulant addiction are behaviourally and neurobiologically distinct and that the differences have important implications for addiction treatment, addiction theories and future research.

The reinstatement model of drug relapse: recent neurobiological findings, emerging research topics, and translational research.

Background and rationaleResults from many clinical studies suggest that drug relapse and craving are often provoked by acute exposure to the self-administered drug or related drugs, drug-associated cues or contexts, or certain stressors. During the last two decades, this clinical scenario has been studied in laboratory animals by using the reinstatement model. In this model, reinstatement of drug seeking by drug priming, drug cues or contexts, or certain stressors is assessed following drug self-administration training and subsequent extinction of the drug-reinforced responding.ObjectiveIn this review, we first summarize recent (2009–present) neurobiological findings from studies using the reinstatement model. We then discuss emerging research topics, including the impact of interfering with putative reconsolidation processes on cue- and context-induced reinstatement of drug seeking, and similarities and differences in mechanisms of reinstatement across drug classes. We conclude by discussing results from recent human studies that were inspired by results from rat studies using the reinstatement model.ConclusionsMain conclusions from the studies reviewed highlight: (1) the ventral subiculum and lateral hypothalamus as emerging brain areas important for reinstatement of drug seeking, (2) the existence of differences in brain mechanisms controlling reinstatement of drug seeking across drug classes, (3) the utility of the reinstatement model for assessing the effect of reconsolidation-related manipulations on cue-induced drug seeking, and (4) the encouraging pharmacological concordance between results from rat studies using the reinstatement model and human laboratory studies on cue- and stress-induced drug craving.

The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes

Humans and other animals change their behavior in response to unexpected outcomes. The orbitofrontal cortex (OFC) is implicated in such adaptive responding, based on evidence from reversal tasks. Yet these tasks confound using information about expected outcomes with learning when those expectations are violated. OFC is critical for the former function; here we show it is also critical for the latter. In a Pavlovian overexpectation task, inactivation of OFC prevented learning driven by unexpected outcomes, even when performance was assessed later. We propose this reflects a critical contribution of outcome signaling by OFC to encoding of reward prediction errors elsewhere. In accord with this proposal, we report that signaling of reward predictions by OFC neurons was related to signaling of prediction errors by dopamine neurons in ventral tegmental area (VTA). Furthermore, bilateral inactivation of VTA or contralateral inactivation of VTA and OFC disrupted learning driven by unexpected outcomes.

Neural Correlates of Variations in Event Processing during Learning in Basolateral Amygdala

The discovery that dopamine neurons signal errors in reward prediction has demonstrated that concepts empirically derived from the study of animal behavior can be used to understand the neural implementation of reward learning. Yet the learning theory models linked to phasic dopamine activity treat attention to events such as cues and rewards as static quantities; other models, such as Pearce–Hall, propose that learning might be influenced by variations in processing of these events. A key feature of these accounts is that event processing is modulated by unsigned rather than signed reward prediction errors. Here we tested whether neural activity in rat basolateral amygdala conforms to this pattern by recording single units in a behavioral task in which rewards were unexpectedly delivered or omitted. We report that neural activity at the time of reward is providing an unsigned error signal with characteristics consistent with those postulated by these models. This neural signal increased immediately after a change in reward, and stronger firing was evident whether the value of the reward increased or decreased. Further, as predicted by these models, the change in firing developed over several trials as expectations for reward were repeatedly violated. This neural signal was correlated with faster orienting to predictive cues after changes in reward, and abolition of the signal by inactivation of basolateral amygdala disrupted this change in orienting and retarded learning in response to changes in reward. These results suggest that basolateral amygdala serves a critical function in attention for learning.

All That Glitters … Dissociating Attention and Outcome Expectancy From Prediction Errors Signals

Initially reported in dopamine neurons, neural correlates of prediction errors have now been shown in a variety of areas, including orbitofrontal cortex, ventral striatum, and amygdala. Yet changes in neural activity to an outcome or cues that precede it can reflect other processes. We review the recent literature and show that although activity in dopamine neurons appears to signal prediction errors, similar activity in orbitofrontal cortex, basolateral amygdala, and ventral striatum does not. Instead, increased firing in basolateral amygdala to unexpected outcomes likely reflects attention, whereas activity in orbitofrontal cortex and ventral striatum is unaffected by prior expectations and may provide information on outcome expectancy. These results have important implications for how these areas interact to facilitate learning and guide behavior.

Should I stay or should I go? : Transformation of time-discounted rewards in orbitofrontal cortex and associated brain circuits

Abstract:  Animals prefer a small, immediate reward over a larger delayed reward (time discounting). Lesions of the orbitofrontal cortex (OFC) can either increase or decrease the breakpoint at which animals abandon the large delayed reward for the more immediate reward as the delay becomes longer. Here we argue that the varied effects of OFC lesions on delayed discounting reflect two different patterns of activity in OFC; one that bridges the gap between a response and an outcome and another that discounts delayed reward. These signals appear to reflect the spatial location of the reward and/or the action taken to obtain it, and are encoded independently from representations of absolute value. We suggest a dual role for output from OFC in both discounting delayed reward, while at the same time supporting new learning for them.

Optogenetic Inhibition of Dorsal Medial Prefrontal Cortex Attenuates Stress-Induced Reinstatement of Palatable Food Seeking in Female Rats

Relapse to maladaptive eating habits during dieting is often provoked by stress. Recently, we identified a role of dorsal medial prefrontal cortex (mPFC) neurons in stress-induced reinstatement of palatable food seeking in male rats. It is unknown whether endogenous neural activity in dorsal mPFC drives stress-induced reinstatement in female rats. Here, we used an optogenetic approach, in which female rats received bilateral dorsal mPFC microinjections of viral constructs coding light-sensitive eNpHR3.0–eYFP or control eYFP protein and intracranial fiber optic implants. Rats were food restricted and trained to lever press for palatable food pellets. Subsequently, pellets were removed, and lever pressing was extinguished; then the effect of bilateral dorsal mPFC light delivery on reinstatement of food seeking was assessed after injections of the pharmacological stressor yohimbine (an α-2 andrenoceptor antagonist) or pellet priming, a manipulation known to provoke food seeking in hungry rats. Dorsal mPFC light delivery attenuated yohimbine-induced reinstatement of food seeking in eNpHR3.0-injected but not eYFP-injected rats. This optical manipulation had no effect on pellet-priming-induced reinstatement or ongoing food-reinforced responding. Dorsal mPFC light delivery attenuated yohimbine-induced Fos immunoreactivity and disrupted neural activity during in vivo electrophysiological recording in awake rats. Optical stimulation caused significant outward currents and blocked electrically evoked action potentials in eNpHR3.0-injected but not eYFP-injected mPFC hemispheres. Light delivery alone caused no significant inflammatory response in mPFC. These findings indicate that intracranial light delivery in eNpHR3.0 rats disrupts endogenous dorsal mPFC neural activity that plays a role in stress-induced relapse to food seeking in female rats.

Medial Prefrontal Cortex Neuronal Activation and Synaptic Alterations after Stress-Induced Reinstatement of Palatable Food Seeking: A Study Using c-fos-GFP Transgenic Female Rats

Relapse to maladaptive eating habits during dieting is often provoked by stress and there is evidence for a role of ovarian hormones in stress responses and feeding. We studied the role of these hormones in stress-induced reinstatement of food seeking and medial prefrontal cortex (mPFC) neuronal activation in c-fos-GFP transgenic female rats, which express GFP in strongly activated neurons. Food-restricted ovariectomized or sham-operated c-fos-GFP rats were trained to lever-press for palatable food pellets. Subsequently, lever-pressing was extinguished and reinstatement of food seeking and mPFC neuronal activation was assessed after injections of the pharmacological stressor yohimbine (0.5–2 mg/kg) or pellet priming (1–4 noncontingent pellets). Estrous cycle effects on reinstatement were also assessed in wild-type rats. Yohimbine- and pellet-priming-induced reinstatement was associated with Fos and GFP induction in mPFC; both reinstatement and neuronal activation were minimally affected by ovarian hormones in both c-fos-GFP and wild-type rats. c-fos-GFP transgenic rats were then used to assess glutamatergic synaptic alterations within activated GFP-positive and nonactivated GFP-negative mPFC neurons following yohimbine-induced reinstatement of food seeking. This reinstatement was associated with reduced AMPA receptor/NMDA receptor current ratios and increased paired-pulse facilitation in activated GFP-positive but not GFP-negative neurons. While ovarian hormones do not appear to play a role in stress-induced relapse of food seeking in our rat model, this reinstatement was associated with unique synaptic alterations in strongly activated mPFC neurons. Our paper introduces the c-fos-GFP transgenic rat as a new tool to study unique synaptic changes in activated neurons during behavior.

Neural Correlates of Variations in Event Processing during Learning in Central Nucleus of Amygdala

Attention or variations in event processing help drive learning. Lesion studies have implicated the central nucleus of the amygdala (CeA) in this process, particularly when expected rewards are omitted. However, lesion studies cannot specify how information processing in CeA supports such learning. To address these questions, we recorded CeA neurons in rats performing a task in which rewards were delivered or omitted unexpectedly. We found that activity in CeA neurons increased selectively at the time of omission and declined again with learning. Increased firing correlated with CeA-inactivation sensitive measures of attention. Notably CeA neurons did not fire to the cues or in response to unexpected rewards. These results indicate that CeA contributes to learning in response to reward omission due to a specific role in signaling actual omission rather than a more general involvement in signaling expectancies, errors, or reward value.