Philippe N. Tobler

Neural Correlates of Value, Risk, and Risk Aversion Contributing to Decision Making under Risk

Decision making under risk is central to human behavior. Economic decision theory suggests that value, risk, and risk aversion influence choice behavior. Although previous studies identified neural correlates of decision parameters, the contribution of these correlates to actual choices is unknown. In two different experiments, participants chose between risky and safe options. We identified discrete blood oxygen level-dependent (BOLD) correlates of value and risk in the ventral striatum and anterior cingulate, respectively. Notably, increasing inferior frontal gyrus activity to low risk and safe options correlated with higher risk aversion. Importantly, the combination of these BOLD responses effectively decoded the behavioral choice. Striatal value and cingulate risk responses increased the probability of a risky choice, whereas inferior frontal gyrus responses showed the inverse relationship. These findings suggest that the BOLD correlates of decision factors are appropriate for an ideal observer to detect behavioral choices. More generally, these biological data contribute to the validity of the theoretical decision parameters for actual decisions under risk.

Neural mechanisms of observational learning

Individuals can learn by interacting with the environment and experiencing a difference between predicted and obtained outcomes (prediction error). However, many species also learn by observing the actions and outcomes of others. In contrast to individual learning, observational learning cannot be based on directly experienced outcome prediction errors. Accordingly, the behavioral and neural mechanisms of learning through observation remain elusive. Here we propose that human observational learning can be explained by two previously uncharacterized forms of prediction error, observational action prediction errors (the actual minus the predicted choice of others) and observational outcome prediction errors (the actual minus predicted outcome received by others). In a functional MRI experiment, we found that brain activity in the dorsolateral prefrontal cortex and the ventromedial prefrontal cortex respectively corresponded to these two distinct observational learning signals.

Functional imaging of the human dopaminergic midbrain

Invasive recording of dopamine neurons in the substantia nigra and ventral tegmental area (SN/VTA) of behaving animals suggests a role for these neurons in reward learning and novelty processing. In humans, functional magnetic resonance imaging (fMRI) is currently the only non-invasive event-related method to measure SN/VTA activity, but it is debated to what extent fMRI enables inference about dopaminergic responses within the SN/VTA. We consider the anatomical and functional parcellation of the primate SN/VTA and find that its homogeneity suggests little variation in the regional specificity of fMRI signals for reward-related dopaminergic responses. Hence, these responses seem to be well captured by the compound fMRI signal from the SN/VTA, which seems quantitatively related to dopamine release in positron emission tomography (PET). We outline how systematic investigation of the functional parcellation of the SN/VTA in animals, new developments in fMRI analysis and combined PET-fMRI studies can narrow the gap between fMRI and dopaminergic neurotransmission.

Risk-dependent reward value signal in human prefrontal cortex

When making choices under uncertainty, people usually consider both the expected value and risk of each option, and choose the one with the higher utility. Expected value increases the expected utility of an option for all individuals. Risk increases the utility of an option for risk-seeking individuals, but decreases it for risk averse individuals. In 2 separate experiments, one involving imperative (no-choice), the other choice situations, we investigated how predicted risk and expected value aggregate into a common reward signal in the human brain. Blood oxygen level dependent responses in lateral regions of the prefrontal cortex increased monotonically with increasing reward value in the absence of risk in both experiments. Risk enhanced these responses in risk-seeking participants, but reduced them in risk-averse participants. The aggregate value and risk responses in lateral prefrontal cortex contrasted with pure value signals independent of risk in the striatum. These results demonstrate an aggregate risk and value signal in the prefrontal cortex that would be compatible with basic assumptions underlying the mean-variance approach to utility.

Short-Term Temporal Discounting of Reward Value in Human Ventral Striatum

Delayed rewards lose their value for economic decisions and constitute weaker reinforcers for learning. Temporal discounting of reward value already occurs within a few seconds in animals, which allows investigations of the underlying neurophysiological mechanisms. However, it is difficult to relate these mechanisms to human discounting behavior, which is usually studied over days and months and may engage different brain processes. Our study aimed to bridge the gap by using very short delays and measuring human functional magnetic resonance responses in one of the key reward centers of the brain, the ventral striatum. We used psychometric methods to assess subjective timing and valuation of monetary rewards with delays of 4.0–13.5 s. We demonstrated hyperbolic and exponential decreases of striatal responses to reward predicting stimuli within this time range, irrespective of changes in reward rate. Lower reward magnitudes induced steeper behavioral and striatal discounting. By contrast, striatal responses following the delivery of reward reflected the uncertainty in subjective timing associated with delayed rewards rather than value discounting. These data suggest that delays of a few seconds affect the neural processing of predicted reward value in the ventral striatum and engage the temporal sensitivity of reward responses. Comparisons with electrophysiological animal data suggest that ventral striatal reward discounting may involve dopaminergic and orbitofrontal inputs.

Segregated and Integrated Coding of Reward and Punishment in the Cingulate Cortex

Reward and punishment have opposite affective value but are both processed by the cingulate cortex. However, it is unclear whether the positive and negative affective values of monetary reward and punishment are processed by separate or common subregions of the cingulate cortex. We performed a functional magnetic resonance imaging study using a free-choice task and compared cingulate activations for different levels of monetary gain and loss. Gain-specific activation (increasing activation for increasing gain, but no activation change in relation to loss) occurred mainly in the anterior part of the anterior cingulate and in the posterior cingulate cortex. Conversely, loss-specific activation (increasing activation for increasing loss, but no activation change in relation to gain) occurred between these areas, in the middle and posterior part of the anterior cingulate. Integrated coding of gain and loss (increasing activation throughout the full range, from biggest loss to biggest gain) occurred in the dorsal part of the anterior cingulate, at the border with the medial prefrontal cortex. Finally, unspecific activation increases to both gains and losses (increasing activation to increasing gains and increasing losses, possibly reflecting attention) occurred in dorsal and middle regions of the cingulate cortex. Together, these results suggest separate and common coding of monetary reward and punishment in distinct subregions of the cingulate cortex. Further meta-analysis suggested that the presently found reward- and punishment-specific areas overlapped with those processing positive and negative emotions, respectively.

Identity-specific coding of future rewards in the human orbitofrontal cortex

Significance To make adaptive choices based on reward-predicting stimuli, organisms must take into account information about both the value and the specific identity of the reward to be obtained. Using appetizing food odors and pattern-based functional magnetic resonance imaging, we show that the human orbitofrontal cortex encodes future rewards in the form of identity-specific value codes. That is, even if valued the same, different expected rewards, such as pizza and chocolate cake, are differently encoded in this region. We further show that identity-specific and -general value coding regions are functionally linked to distinct regions, providing a novel account for the neural circuitry that underlies integration of both sensory and affective information to guide reward-related behavior. Nervous systems must encode information about the identity of expected outcomes to make adaptive decisions. However, the neural mechanisms underlying identity-specific value signaling remain poorly understood. By manipulating the value and identity of appetizing food odors in a pattern-based imaging paradigm of human classical conditioning, we were able to identify dissociable predictive representations of identity-specific reward in orbitofrontal cortex (OFC) and identity-general reward in ventromedial prefrontal cortex (vmPFC). Reward-related functional coupling between OFC and olfactory (piriform) cortex and between vmPFC and amygdala revealed parallel pathways that support identity-specific and -general predictive signaling. The demonstration of identity-specific value representations in OFC highlights a role for this region in model-based behavior and reveals mechanisms by which appetitive behavior can go awry.

Disentangling neural representations of value and salience in the human brain

Significance The value and salience of predictive cues are important signals for regulating approach–avoidance behavior and attentional processing, respectively. However, the two signals often are confounded in studies of decision-making. Indeed, recent results suggest that neural signals in the primate posterior parietal cortex (PPC) which previously were thought to encode value actually reflect salience. This finding has created considerable uncertainty about previously identified value signals. Here we experimentally dissociate value and salience and use pattern-based functional MRI to demonstrate distinct encoding of both signals in the PPC, thereby reinforcing the earlier reports of value in the PPC. Moreover, we show that the orbitofrontal cortex encodes the predicted value of appetitive and aversive outcomes on a common neural scale. A large body of evidence has implicated the posterior parietal and orbitofrontal cortex in the processing of value. However, value correlates perfectly with salience when appetitive stimuli are investigated in isolation. Accordingly, considerable uncertainty has remained about the precise nature of the previously identified signals. In particular, recent evidence suggests that neurons in the primate parietal cortex signal salience instead of value. To investigate neural signatures of value and salience, here we apply multivariate (pattern-based) analyses to human functional MRI data acquired during a noninstrumental outcome-prediction task involving appetitive and aversive outcomes. Reaction time data indicated additive and independent effects of value and salience. Critically, we show that multivoxel ensemble activity in the posterior parietal cortex encodes predicted value and salience in superior and inferior compartments, respectively. These findings reinforce the earlier reports of parietal value signals and reconcile them with the recent salience report. Moreover, we find that multivoxel patterns in the orbitofrontal cortex correlate with value. Importantly, the patterns coding for the predicted value of appetitive and aversive outcomes are similar, indicating a common neural scale for appetite and aversive values in the orbitofrontal cortex. Thus orbitofrontal activity patterns satisfy a basic requirement for a neural value signal.

Restricting temptations: neural mechanisms of precommitment.

Summary Humans can resist temptations by exerting willpower, the effortful inhibition of impulses. But willpower can be disrupted by emotions and depleted over time. Luckily, humans can deploy alternative self-control strategies like precommitment, the voluntary restriction of access to temptations. Here, we examined the neural mechanisms of willpower and precommitment using fMRI. Behaviorally, precommitment facilitated choices for large delayed rewards, relative to willpower, especially in more impulsive individuals. While willpower was associated with activation in dorsolateral prefrontal cortex (DLPFC), posterior parietal cortex (PPC), and inferior frontal gyrus, precommitment engaged lateral frontopolar cortex (LFPC). During precommitment, LFPC showed increased functional connectivity with DLPFC and PPC, especially in more impulsive individuals, and the relationship between impulsivity and LFPC connectivity was mediated by value-related activation in ventromedial PFC. Our findings support a hierarchical model of self-control in which LFPC orchestrates precommitment by controlling action plans in more caudal prefrontal regions as a function of expected value.

Social discounting involves modulation of neural value signals by temporoparietal junction

Significance People often consider the well-being of others. However, they are more likely to be generous toward individuals they feel close to than to those they only meet sporadically. Using neuroimaging tools, we show that the decline in generosity across social distance is realized by the interplay of two brain structures—the ventromedial prefrontal cortex coding the relative appeal of a selfish or a generous option, and the temporoparietal junction modulating appeal signals of the generous outcome, depending on social distance between participant and beneficiary. Based on these findings, we developed a biologically plausible model explaining social discounting in particular, and prosocial behavior in general. Our study opens up new avenues to understand and tackle frictions arising in social networks. Most people are generous, but not toward everyone alike: generosity usually declines with social distance between individuals, a phenomenon called social discounting. Despite the pervasiveness of social discounting, social distance between actors has been surprisingly neglected in economic theory and neuroscientific research. We used functional magnetic resonance imaging (fMRI) to study the neural basis of this process to understand the neural underpinnings of social decision making. Participants chose between selfish and generous alternatives, yielding either a large reward for the participant alone, or smaller rewards for the participant and another individual at a particular social distance. We found that generous choices engaged the temporoparietal junction (TPJ). In particular, the TPJ activity was scaled to the social-distance–dependent conflict between selfish and generous motives during prosocial choice, consistent with ideas that the TPJ promotes generosity by facilitating overcoming egoism bias. Based on functional coupling data, we propose and provide evidence for a biologically plausible neural model according to which the TPJ supports social discounting by modulating basic neural value signals in the ventromedial prefrontal cortex to incorporate social-distance–dependent other-regarding preferences into an otherwise exclusively own-reward value representation.