Melissa J. Sharpe

Dopamine transients are sufficient and necessary for acquisition of model-based associations

Associative learning is driven by prediction errors. Dopamine transients correlate with these errors, which current interpretations limit to endowing cues with a scalar quantity reflecting the value of future rewards. We tested whether dopamine might act more broadly to support learning of an associative model of the environment. Using sensory preconditioning, we show that prediction errors underlying stimulus–stimulus learning can be blocked behaviorally and reinstated by optogenetically activating dopamine neurons. We further show that suppressing the firing of these neurons across the transition prevents normal stimulus–stimulus learning. These results establish that the acquisition of model-based information about transitions between nonrewarding events is also driven by prediction errors and that, contrary to existing canon, dopamine transients are both sufficient and necessary to support this type of learning. Our findings open new possibilities for how these biological signals might support associative learning in the mammalian brain in these and other contexts.

The Prelimbic Cortex Contributes to the Down-Regulation of Attention Toward Redundant Cues

Previous research suggests disruption of activity in the prelimbic (PL) cortex produces deficits in tasks requiring preferential attention toward cues that are good predictors of an event. By manipulating cue predictive power, we clarify this role using Pavlovian conditioning. Experiment 1a showed pretraining excitotoxic lesions of the PL cortex disrupted the ability of animals to distribute attention across stimuli conditioned in compound. Experiment 1b demonstrated that these lesions did not affect the ability to block learning about a stimulus when it was presented simultaneously with another stimulus that was previously paired with the outcome. However, in a subsequent test, PL-lesioned animals learnt about this blocked cue faster than sham-lesioned animals when this stimulus alone was paired with reinforcement, suggesting these animals did not down-regulate attention toward the redundant cue during blocking. Experiment 2 tested this hypothesis using an unblocking procedure designed to explicitly reveal a down-regulation of attention during blocking. In this, sham-lesioned animals were shown to down-regulate attention during blocking. PL-lesioned animals did not exhibit this effect. We propose that observed deficits are the result of a specific deficit in down-regulating attention toward redundant cues, indicating the disruption of an attentional process described in Mackintoshs (Mackintosh NJ. 1975. Psychol Review. 82:276) attentional theory.

The prelimbic cortex uses contextual cues to modulate responding towards predictive stimuli during fear renewal

Previous research suggests the prelimbic (PL) cortex is involved in expression of conditioned fear (Burgos-Robles, Vidal-Gonzalez, & Quirk, 2009; Corcoran & Quirk, 2007). However, there is a long history of research in the appetitive domain which implicates this region in using higher-order cues to modulate a behavioural response (Birrell & Brown, 2000; Floresco, Block, & Tse, 2008; Marquis, Killcross, & Haddon, 2007; Sharpe & Killcross, 2014). For example, the PL cortex is necessary to allow animals to use contextual cues to disambiguate response conflict in ambiguous circumstances (Marquis et al., 2007). Using an ABA fear renewal procedure, we assessed the role of the PL cortex in using contextual cues to modulate a response towards a conditioned stimulus (CS) in an aversive setting. We found that pre-training lesions of the PL cortex did not impact on the expression or extinction of conditioned fear. Rather, they selectively abolished renewal. Functional inactivation of the PL cortex during extinction did not disrupt the subsequent renewal of conditioned fear or the ability of animals to exhibit fear towards a CS during the extinction session. However, PL inactivation during the renewal test session disrupted the ability of animals to demonstrate a reinstatement of responding in the renewal context. An analysis of orienting responses showed that renewal deficits were accompanied by a lack of change in attentional responding towards the CS. These data suggest the PL cortex uses contextual cues to modulate both a behavioural and an attentional response during aversive procedures. We argue that the role of the PL cortex in the expression of conditioned fear is to use higher-order information to modulate responding towards predictive cues in ambiguous circumstance.

The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning

Phasic activity of midbrain dopamine neurons is currently thought to encapsulate the prediction-error signal described in Sutton and Barto’s (1981) model-free reinforcement learning algorithm. This phasic signal is thought to contain information about the quantitative value of reward, which transfers to the reward-predictive cue after learning. This is argued to endow the reward-predictive cue with the value inherent in the reward, motivating behavior toward cues signaling the presence of reward. Yet theoretical and empirical research has implicated prediction-error signaling in learning that extends far beyond a transfer of quantitative value to a reward-predictive cue. Here, we review the research which demonstrates the complexity of how dopaminergic prediction errors facilitate learning. After briefly discussing the literature demonstrating that phasic dopaminergic signals can act in the manner described by Sutton and Barto (1981), we consider how these signals may also influence attentional processing across multiple attentional systems in distinct brain circuits. Then, we discuss how prediction errors encode and promote the development of context-specific associations between cues and rewards. Finally, we consider recent evidence that shows dopaminergic activity contains information about causal relationships between cues and rewards that reflect information garnered from rich associative models of the world that can be adapted in the absence of direct experience. In discussing this research we hope to support the expansion of how dopaminergic prediction errors are thought to contribute to the learning process beyond the traditional concept of transferring quantitative value.

The prelimbic cortex uses higher-order cues to modulate both the acquisition and expression of conditioned fear

The prelimbic (PL) cortex allows rodents to adapt their responding under changing experimental circumstances. In line with this, the PL cortex has been implicated in strategy set shifting, attentional set shifting, the resolution of response conflict, and the modulation of attention towards predictive stimuli. One interpretation of this research is that the PL cortex is involved in using information garnered from higher-order cues in the environment to modulate how an animal responds to environmental stimuli. However, data supporting this view of PL function in the aversive domain are lacking. In the following experiments, we attempted to answer two questions. Firstly, we wanted to investigate whether the role of the PL cortex in using higher-order cues to influence responding generalizes across appetitive and aversive domains. Secondly, as much of the research has focused on a role for the PL cortex in performance, we wanted to assess whether this region is also involved in the acquisition of hierarchal associations which facilitate an ability to use higher-order cues to modulate responding. In order to answer these questions, we assessed the impact of PL inactivation during both the acquisition and expression of a contextual bi-conditional discrimination. A contextual bi-conditional discrimination involves presenting two stimuli. In one context, one stimulus is paired with shock while the other is presented without shock. In another context, these contingencies are reversed. Thus, animals have to use the present contextual cues to disambiguate the significance of the stimulus and respond appropriately. We found that PL inactivation disrupted both the encoding and expression of these context-dependent associations. This supports a role for the PL cortex in allowing higher-order cues to modulate both learning about, and responding towards, different cues. We discuss these findings in the broader context of functioning in the medial prefrontal cortex (PFC).

The prelimbic cortex directs attention toward predictive cues during fear learning

The prelimbic cortex is argued to promote conditioned fear expression, at odds with appetitive research implicating this region in attentional processing. Consistent with an attentional account, we report that the effect of prelimbic lesions on fear expression depends on the degree of competition between contextual and discrete cues. Further, when competition from contextual cues is low, we found that PL inactivation resulted in animals expressing fear toward irrelevant discrete cues; an effect selective to inactivation during the learning phase and not during retrieval. These data demonstrate that the prelimbic cortex modulates attention toward cues to preferentially direct fear responding on the basis of their predictive value.

Back to basics: Making predictions in the orbitofrontal-amygdala circuit.

Underlying many complex behaviors are simple learned associations that allow humans and animals to anticipate the consequences of their actions. The orbitofrontal cortex and basolateral amygdala are two regions which are crucial to this process. In this review, we go back to basics and discuss the literature implicating both these regions in simple paradigms requiring the development of associations between stimuli and the motivationally-significant outcomes they predict. Much of the functional research surrounding this ability has suggested that the orbitofrontal cortex and basolateral amygdala play very similar roles in making these predictions. However, electrophysiological data demonstrates critical differences in the way neurons in these regions respond to predictive cues, revealing a difference in their functional role. On the basis of these data and theories that have come before, we propose that the basolateral amygdala is integral to updating information about cue-outcome contingencies whereas the orbitofrontal cortex is critical to forming a wider network of past and present associations that are called upon by the basolateral amygdala to benefit future learning episodes. The tendency for orbitofrontal neurons to encode past and present contingencies in distinct neuronal populations may facilitate its role in the formation of complex, high-dimensional state-specific associations.

Daily Exposure to Sucrose Impairs Subsequent Learning About Food Cues: A Role for Alterations in Ghrelin Signaling and Dopamine D2 Receptors.

The prevalence of hedonic foods and associated advertising slogans has contributed to the rise of the obesity epidemic in the modern world. Research has shown that intake of these foods disrupt dopaminergic systems. It may be that a disruption of these circuits produces aberrant learning about food–cue relationships. We found that rodents given 28 days of intermittent access to sucrose exhibited a deficit in the ability to block learning about a stimulus when it is paired in compound with food and another stimulus that has already been established as predictive of the food outcome. This deficit was characterized by an approach to a cue signaling food delivery that is usually blocked by prior learning, an effect dependent on dopaminergic prediction-error signaling in the midbrain. Administering the D2 agonist quinpirole during learning restored blocking in animals with a prior history of sucrose exposure. Further, repeated central infusions of ghrelin produced a deficit in blocking in the same manner as sucrose exposure. We argue that changes in dopaminergic systems resulting from sucrose exposure are mediated by a disruption of ghrelin signaling as rodents come to anticipate delivery of the highly palatable sucrose outside of normal feeding schedules. This suggestion is supported by our finding that both sucrose and ghrelin treatments resulted in increases in amphetamine-induced locomotor responding. Thus, for the first time, we have provided evidence of a potential link between alterations in D2 receptors caused by the intake of hedonic foods and aberrant learning about cue–food relationships capable of promoting inappropriate feeding habits. In addition, we have found preliminary evidence to suggest that this is mediated by changes in ghrelin signaling, a finding that should stimulate further research into modulation of ghrelin activity to treat obesity.

The chemotherapy agent oxaliplatin impairs the renewal of fear to an extinguished conditioned stimulus in rats.

Recent evidence has shown that diverse chemotherapy agents can induce cognitive impairments and neurotoxic damage to the central nervous system. Oxaliplatin (OXP), a platinum compound, has been linked with acute and chronic peripheral neuropathies. This study explored the cognitive impacts of OXP in the rat with a fear conditioning procedure. 10 days prior to conditioning and testing, rats received an intraperitoneal injection of OXP (12 mg/kg). On the first day of conditioning, the rats were conditioned to two CSs (CS-ren and CS-ext) in one set of chambers (context A). They then received three tests on separate days. First, the rats were assessed for contextual fear conditioning in context A. Next, the CSs were presented 20 times in a new context (B) until fear conditioning had extinguished. Finally, one of the CSs (CS-ext) was tested again in the extinction context (B), and the other (CS-ren) presented in a new context (C). Results showed that OXP had no effect on the ability of rats to express fear to the conditioning context (A), or on the expression and extinction of conditioned fear to either CS when presented in a second context (B). However, the administration of OXP did impair the ability of rats to renew levels of conditioned fear to CS-ren when this CS was presented in a novel context (C) following extinction. This profile of impairment is consistent with hippocampal damage, and may also involve frontal cortical, amygdalar and thalamic regions important for context discrimination and the contextual modulation of behaviour.

The State of the Orbitofrontal Cortex

State representation is fundamental to behavior. However, identifying the true state of the world is challenging when explicit cues are ambiguous. Here, Bradfield and colleagues show that the medial OFC is critical for using associative information to discriminate ambiguous states.