Elizabeth Tricomi

Modulation of Caudate Activity by Action Contingency

Research has increasingly implicated the striatum in the processing of reward-related information in both animals and humans. However, it is unclear whether human striatal activation is driven solely by the hedonic properties of rewards or whether such activation is reliant on other factors, such as anticipation of upcoming reward or performance of an action to earn a reward. We used event-related functional magnetic resonance imaging to investigate hemodynamic responses to monetary rewards and punishments in three experiments that made use of an oddball paradigm. We presented reward and punishment displays randomly in time, following an anticipatory cue, or following a button press response. Robust and differential activation of the caudate nucleus occurred only when a perception of contingency existed between the button press response and the outcome. This finding suggests that the caudate is involved in reinforcement of action potentially leading to reward, rather than in processing reward per se.

A specific role for posterior dorsolateral striatum in human habit learning

Habits are characterized by an insensitivity to their consequences and, as such, can be distinguished from goal‐directed actions. The neural basis of the development of demonstrably outcome‐insensitive habitual actions in humans has not been previously characterized. In this experiment, we show that extensive training on a free‐operant task reduces the sensitivity of participants’ behavior to a reduction in outcome value. Analysis of functional magnetic resonance imaging data acquired during training revealed a significant increase in task‐related cue sensitivity in a right posterior putamen–globus pallidus region as training progressed. These results provide evidence for a shift from goal‐directed to habit‐based control of instrumental actions in humans, and suggest that cue‐driven activation in a specific region of dorsolateral posterior putamen may contribute to the habitual control of behavior in humans.

Neural evidence for inequality-averse social preferences

A popular hypothesis in the social sciences is that humans have social preferences to reduce inequality in outcome distributions because it has a negative impact on their experienced reward. Although there is a large body of behavioural and anthropological evidence consistent with the predictions of these theories, there is no direct neural evidence for the existence of inequality-averse preferences. Such evidence would be especially useful because some behaviours that are consistent with a dislike for unequal outcomes could also be explained by concerns for social image or reciprocity, which do not require a direct aversion towards inequality. Here we use functional MRI to test directly for the existence of inequality-averse social preferences in the human brain. Inequality was created by recruiting pairs of subjects and giving one of them a large monetary endowment. While both subjects evaluated further monetary transfers from the experimenter to themselves and to the other participant, we measured neural responses in the ventral striatum and ventromedial prefrontal cortex, two areas that have been shown to be involved in the valuation of monetary and primary rewards in both social and non-social contexts. Consistent with inequality-averse models of social preferences, we find that activity in these areas was more responsive to transfers to others than to self in the ‘high-pay’ subject, whereas the activity of the ‘low-pay’ subject showed the opposite pattern. These results provide direct evidence for the validity of this class of models, and also show that the brain’s reward circuitry is sensitive to both advantageous and disadvantageous inequality.

Performance Feedback Drives Caudate Activation in a Phonological Learning Task

Adults have difficulty discriminating nonnative phonetic contrasts, but under certain circumstances training can lead to improvement in this ability. Despite the ubiquitous use of performance feedback in training paradigms in this and many other domains, the mechanisms by which feedback affects learning are not well understood. In this event-related functional magnetic resonance imaging study, we examined how performance feedback is processed during perceptual learning. Thirteen Japanese speakers for whom the English phonemes [r] and [l] were nondistinct performed an identification task of the words road and load that has been shown to be effective in inducing learning only when performance feedback is present. Each subject performed alternating runs of training with and without feedback, followed by performance of a card-guessing task with monetary reward and punishment outcomes. We found that the caudate nucleus was more robustly activated bilaterally when performing the perceptual identification task with feedback than without feedback, and the right caudate nucleus also showed a differential response to positive and negative feedback. Moreover, using a within-subjects design, we found that the caudate nucleus also showed a similar activation pattern to monetary reward and punishment outcomes in the card-guessing task. These results demonstrate that the caudate responds to positive and negative feedback during learning in a manner analogous to its processing of extrinsic affective reinforcers and indicate that this region may be a critical moderator of the influence of feedback on learning. These findings impact our broader understanding of the mechanisms underlying nondeclarative learning and language acquisition.

Neural Correlates of Instrumental Contingency Learning: Differential Effects of Action–Reward Conjunction and Disjunction

Contingency theories of goal-directed action propose that experienced disjunctions between an action and its specific consequences, as well as conjunctions between these events, contribute to encoding the action–outcome association. Although considerable behavioral research in rats and humans has provided evidence for this proposal, relatively little is known about the neural processes that contribute to the two components of the contingency calculation. Specifically, while recent findings suggest that the influence of action–outcome conjunctions on goal-directed learning is mediated by a circuit involving ventromedial prefrontal, medial orbitofrontal cortex, and dorsomedial striatum, the neural processes that mediate the influence of experienced disjunctions between these events are unknown. Here we show differential responses to probabilities of conjunctive and disjunctive reward deliveries in the ventromedial prefrontal cortex, the dorsomedial striatum, and the inferior frontal gyrus. Importantly, activity in the inferior parietal lobule and the left middle frontal gyrus varied with a formal integration of the two reward probabilities, ΔP, as did response rates and explicit judgments of the causal efficacy of the action.

Individual differences in delay discounting under acute stress: the role of trait perceived stress.

Delay discounting refers to the reduction of the value of a future reward as the delay to that reward increases. The rate at which individuals discount future rewards varies as a function of both individual and contextual differences, and high delay discounting rates have been linked with problematic behaviors, including drug abuse and gambling. The current study investigated the effects of acute anticipatory stress on delay discounting, while considering two important factors: individual perceptions of stress and whether the stressful situation is future-focused or present-focused. Half of the participants experienced acute stress by anticipating giving a videotaped speech. This stress was either future-oriented (speech about future job) or present-oriented (speech about physical appearance). They then performed a delay discounting task, in which they chose between smaller, immediate rewards, and larger, delayed rewards. Their scores on the Perceived Stress Scale were also collected. The way in which one appraises stressful situations interacts with acute stress to influence choices; under stressful conditions, delay discounting rate was highest in individuals with low trait perceived stress and lowest for individuals with high trait perceived stress. This result might be related to individual variation in reward responsiveness under stress. Furthermore, the time orientation of the task interacted with its stressfulness to affect the individual’s propensity to choose immediate rewards. These findings add to our understanding of the intermediary factors between stress and decision-making.

Information content and reward processing in the human striatum during performance of a declarative memory task

Negative feedback can signal poor performance, but it also provides information that can help learners reach the goal of task mastery. The primary aim of this study was to test the hypothesis that the amount of information provided by negative feedback during a paired-associate learning task influences feedback-related processing in the caudate nucleus. To do this, we manipulated the number of response options: With two options, positive and negative feedback provide equal amounts of information, whereas with four options, positive feedback provides more information than does negative feedback. We found that positive and negative feedback activated the caudate similarly when there were two response options. With four options, the caudate’s response to negative feedback was reduced. A secondary goal was to investigate the link between brain-based measures of feedback-related processing and behavioral indices of learning. Analysis of the posttest measures showed that trials with positive feedback were associated with higher posttest confidence ratings. Additionally, when positive feedback was delivered, caudate activity was greater for trials with high than with low posttest confidence. This experiment demonstrated the context sensitivity of feedback processing and provided evidence that feedback processing in the striatum can contribute to the strengthening of the representations available within declarative memory.

Goals and task difficulty expectations modulate striatal responses to feedback

The striatum plays a critical role in learning from reward, and it has been implicated in learning from performance-related feedback as well. Positive and negative performance-related feedback is known to engage the striatum during learning by eliciting a response similar to the reinforcement signal for extrinsic rewards and punishments. Feedback is an important tool used to teach new skills and promote healthful lifestyle changes, so it is important to understand how motivational contexts can modulate its effectiveness at promoting learning. While it is known that striatal responses scale with subjective factors influencing the desirability of rewards, it is less clear how expectations and goals might modulate the striatal responses to cognitive feedback during learning. We used functional magnetic resonance imaging to investigate the effects of task difficulty expectations and achievement goals on feedback processing during learning. We found that individuals who scored high in normative goals, which reflect a desire to outperform other students academically, showed the strongest effects of our manipulation. High levels of normative goals were associated with greater performance gains and exaggerated striatal sensitivity to positive versus negative feedback during blocks that were expected to be more difficult. Our findings suggest that normative goals may enhance performance when difficulty expectations are high, while at the same time modulating the subjective value of feedback as processed in the striatum.

Effects of intrinsic motivation on feedback processing during learning.

Learning commonly requires feedback about the consequences of ones actions, which can drive learners to modify their behavior. Motivation may determine how sensitive an individual might be to such feedback, particularly in educational contexts where some students value academic achievement more than others. Thus, motivation for a task might influence the value placed on performance feedback and how effectively it is used to improve learning. To investigate the interplay between intrinsic motivation and feedback processing, we used functional magnetic resonance imaging (fMRI) during feedback-based learning before and after a novel manipulation based on motivational interviewing, a technique for enhancing treatment motivation in mental health settings. Because of its role in the reinforcement learning system, the striatum is situated to play a significant role in the modulation of learning based on motivation. Consistent with this idea, motivation levels during the task were associated with sensitivity to positive versus negative feedback in the striatum. Additionally, heightened motivation following a brief motivational interview was associated with increases in feedback sensitivity in the left medial temporal lobe. Our results suggest that motivation modulates neural responses to performance-related feedback, and furthermore that changes in motivation facilitate processing in areas that support learning and memory.

Value and probability coding in a feedback-based learning task utilizing food rewards

For the consequences of our actions to guide behavior, the brain must represent different types of outcome-related information. For example, an outcome can be construed as negative because an expected reward was not delivered or because an outcome of low value was delivered. Thus behavioral consequences can differ in terms of the information they provide about outcome probability and value. We investigated the role of the striatum in processing probability-based and value-based negative feedback by training participants to associate cues with food rewards and then employing a selective satiety procedure to devalue one food outcome. Using functional magnetic resonance imaging, we examined brain activity related to receipt of expected rewards, receipt of devalued outcomes, omission of expected rewards, omission of devalued outcomes, and expected omissions of an outcome. Nucleus accumbens activation was greater for rewarding outcomes than devalued outcomes, but activity in this region did not correlate with the probability of reward receipt. Activation of the right caudate and putamen, however, was largest in response to rewarding outcomes relative to expected omissions of reward. The dorsal striatum (caudate and putamen) at the time of feedback also showed a parametric increase correlating with the trialwise probability of reward receipt. Our results suggest that the ventral striatum is sensitive to the motivational relevance, or subjective value, of the outcome, while the dorsal striatum codes for a more complex signal that incorporates reward probability. Value and probability information may be integrated in the dorsal striatum, to facilitate action planning and allocation of effort.