P. Read Montague

A framework for studying the neurobiology of value-based decision making

Neuroeconomics is the study of the neurobiological and computational basis of value-based decision making. Its goal is to provide a biologically based account of human behaviour that can be applied in both the natural and the social sciences. This Review proposes a framework to investigate different aspects of the neurobiology of decision making. The framework allows us to bring together recent findings in the field, highlight some of the most important outstanding problems, define a common lexicon that bridges the different disciplines that inform neuroeconomics, and point the way to future applications.

Neural economics and the biological substrates of valuation

A recent flurry of neuroimaging and decision-making experiments in humans, when combined with single-unit data from orbitofrontal cortex, suggests major additions to current models of reward processing. We review these data and models and use them to develop a specific computational relationship between the value of a predictor and the future rewards or punishments that it promises. The resulting computational model, the predictor-valuation model (PVM), is shown to anticipate a class of single-unit neural responses in orbitofrontal and striatal neurons. The model also suggests how neural responses in the orbitofrontal-striatal circuit may support the conversion of disparate types of future rewards into a kind of internal currency, that is, a common scale used to compare the valuation of future behavioral acts or stimuli.

Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum

Functional MRI experiments in human subjects strongly suggest that the striatum participates in processing information about the predictability of rewarding stimuli. However, stimuli can be unpredictable in character (what stimulus arrives next), unpredictable in time (when the stimulus arrives), and unpredictable in amount (how much arrives). These variables have not been dissociated in previous imaging work in humans, thus conflating possible interpretations of the kinds of expectation errors driving the measured brain responses. Using a passive conditioning task and fMRI in human subjects, we show that positive and negative prediction errors in reward delivery time correlate with BOLD changes in human striatum, with the strongest activation lateralized to the left putamen. For the negative prediction error, the brain response was elicited by expectations only and not by stimuli presented directly; that is, we measured the brain response to nothing delivered (juice expected but not delivered) contrasted with nothing delivered (nothing expected).

Computational roles for dopamine in behavioural control

Neuromodulators such as dopamine have a central role in cognitive disorders. In the past decade, biological findings on dopamine function have been infused with concepts taken from computational theories of reinforcement learning. These more abstract approaches have now been applied to describe the biological algorithms at play in our brains when we form value judgements and make choices. The application of such quantitative models has opened up new fields, ripe for attack by young synthesizers and theoreticians.

Adult attachment predicts maternal brain and oxytocin response to infant cues.

Infant cues, such as smiling or crying facial expressions, are powerful motivators of human maternal behavior, activating dopamine-associated brain reward circuits. Oxytocin, a neurohormone of attachment, promotes maternal care in animals, although its role in human maternal behavior is unclear. We examined 30 first-time new mothers to test whether differences in attachment, based on the Adult Attachment Interview, were related to brain reward and peripheral oxytocin response to infant cues. On viewing their own infants smiling and crying faces during functional MRI scanning, mothers with secure attachment showed greater activation of brain reward regions, including the ventral striatum, and the oxytocin-associated hypothalamus/pituitary region. Peripheral oxytocin response to infant contact at 7 months was also significantly higher in secure mothers, and was positively correlated with brain activation in both regions. Insecure/dismissing mothers showed greater insular activation in response to their own infants sad faces. These results suggest that individual differences in maternal attachment may be linked with development of the dopaminergic and oxytocinergic neuroendocrine systems.

The rupture and repair of cooperation in borderline personality disorder.

To sustain or repair cooperation during a social exchange, adaptive creatures must understand social gestures and the consequences when shared expectations about fair exchange are violated by accident or intent. We recruited 55 individuals afflicted with borderline personality disorder (BPD) to play a multiround economic exchange game with healthy partners. Behaviorally, individuals with BPD showed a profound incapacity to maintain cooperation, and were impaired in their ability to repair broken cooperation on the basis of a quantitative measure of coaxing. Neurally, activity in the anterior insula, a region known to respond to norm violations across affective, interoceptive, economic, and social dimensions, strongly differentiated healthy participants from individuals with BPD. Healthy subjects showed a strong linear relation between anterior insula response and both magnitude of monetary offer received from their partner (input) and the amount of money repaid to their partner (output). In stark contrast, activity in the anterior insula of BPD participants was related only to the magnitude of repayment sent back to their partner (output), not to the magnitude of offers received (input). These neural and behavioral data suggest that norms used in perception of social gestures are pathologically perturbed or missing altogether among individuals with BPD. This game-theoretic approach to psychopathology may open doors to new ways of characterizing and studying a range of mental illnesses.

The Neural Substrates of Reward Processing in Humans: The Modern Role of fMRI

Experimental work in animals has identified numerous neural structures involved in reward processing and reward-dependent learning. Until recently, this work provided the primary basis for speculations about the neural substrates of human reward processing. The widespread use of neuroimaging technology has changed this situation dramatically over the past decade through the use of PET and fMRI. Here, the authors focus on the role played by fMRI studies, where recent work has replicated the animal results in human subjects and has extended the view of putative reward-processing neural structures. In particular, fMRI work has identified a set of reward-related brain structures including the orbitofrontal cortex, amygdala, ventral striatum, and medial prefrontal cortex. Moreover, the human experiments have probed the dependence of human reward responses on learned expectations, context, timing, and the reward dimension. Current experiments aim to assess the function of human reward-processing structures to determine how they allow us to predict, assess, and act in response to rewards. The authors review current accomplishments in the study of human reward processing and focus their discussion on explanations directed particularly at the role played by the ventral striatum. They discuss how these findings may contribute to a better understanding of deficits associated with Parkinson’s disease.

Activity in human ventral striatum locked to errors of reward prediction.

The mesolimbic dopaminergic system has long been known to be involved in the processing of rewarding stimuli, although recent evidence from animal research has suggested a more specific role of signaling errors in the prediction of rewards. We tested this hypothesis in humans, using functional magnetic resonance imaging (fMRI) and an operant conditioning paradigm for the discrete delivery of small quantities of fruit juice, along with a control experiment in which juice was substituted with a neutral visual stimulus. A local estimation of the activity in the ventral striatum showed a significant differentiation when the juice was withheld at the expected time of delivery; this finding was not replicated in the case of visual stimulation, providing evidence for time-locked processing of reward prediction errors in human ventral striatum.

Learning and selective attention

Selective attention involves the differential processing of different stimuli, and has widespread psychological and neural consequences. Although computational modeling should offer a powerful way of linking observable phenomena at different levels, most work has focused on the relatively narrow issue of constraints on processing resources. By contrast, we consider statistical and informational aspects of selective attention, divorced from resource constraints, which are evident in animal conditioning experiments involving uncertain predictions and unreliable stimuli. Neuromodulatory systems and limbic structures are known to underlie attentional effects in such tasks.

When Things Are Better or Worse than Expected: The Medial Frontal Cortex and the Allocation of Processing Resources

Access to limited-capacity neural systems of cognitive control must be restricted to the most relevant information. How the brain identifies and selects items for preferential processing is not fully understood. Anatomical models often place the selection mechanism in the medial frontal cortex (MFC), and one computational model proposes that the mesotelencephalic dopamine (DA) system, via its reward prediction properties, provides a gate through which information gains access to limited-capacity systems. There is a medial frontal event-related potential (ERP) index of attention selection, the anterior positivity (P2a), associated with DA reward system input to the MFC for the identification of task-relevant perceptual representations. The P2a has a similar spatio-temporal distribution as the medial frontal negativity (MFN), elicited to error responses or choices resulting in monetary loss. The MFN has also been linked to DA projections to the MFC but for action monitoring rather than attention selection. This study proposes that the P2a and the MFN reflect the same MFC evaluation function and use a passive reward prediction design containing neither instructed attention nor response to demonstrate that the ERP over medial frontal leads at the P2a/MFN latency is consistent with activity of midbrain DA neurons, positive to unpredicted rewards and negative when a predicted reward is withheld. This result suggests that MFC activity is regulated by DA reward system input and may function to identify items or actions that exceed or fail to meet motivational prediction.