Ming Hsu

Neural Systems Responding to Degrees of Uncertainty in Human Decision-Making

Much is known about how people make decisions under varying levels of probability (risk). Less is known about the neural basis of decision-making when probabilities are uncertain because of missing information (ambiguity). In decision theory, ambiguity about probabilities should not affect choices. Using functional brain imaging, we show that the level of ambiguity in choices correlates positively with activation in the amygdala and orbitofrontal cortex, and negatively with a striatal system. Moreover, striatal activity correlates positively with expected reward. Neurological subjects with orbitofrontal lesions were insensitive to the level of ambiguity and risk in behavioral choices. These data suggest a general neural circuit responding to degrees of uncertainty, contrary to decision theory.

The right and the good: distributive justice and neural encoding of equity and efficiency.

Distributive justice concerns how individuals and societies distribute benefits and burdens in a just or moral manner. We combined distribution choices with functional magnetic resonance imaging to investigate the central problem of distributive justice: the trade-off between equity and efficiency. We found that the putamen responds to efficiency, whereas the insula encodes inequity, and the caudate/septal subgenual region encodes a unified measure of efficiency and inequity (utility). Notably, individual differences in inequity aversion correlate with activity in inequity and utility regions. Against utilitarianism, our results support the deontological intuition that a sense of fairness is fundamental to distributive justice but, as suggested by moral sentimentalists, is rooted in emotional processing. More generally, emotional responses related to norm violations may underlie individual differences in equity considerations and adherence to ethical rules.

The Wick in the Candle of Learning Epistemic Curiosity Activates Reward Circuitry and Enhances Memory

Curiosity has been described as a desire for learning and knowledge, but its underlying mechanisms are not well understood. We scanned subjects with functional magnetic resonance imaging while they read trivia questions. The level of curiosity when reading questions was correlated with activity in caudate regions previously suggested to be involved in anticipated reward. This finding led to a behavioral study, which showed that subjects spent more scarce resources (either limited tokens or waiting time) to find out answers when they were more curious. The functional imaging also showed that curiosity increased activity in memory areas when subjects guessed incorrectly, which suggests that curiosity may enhance memory for surprising new information. This prediction about memory enhancement was confirmed in a behavioral study: Higher curiosity in an initial session was correlated with better recall of surprising answers 1 to 2 weeks later.

Explicit neural signals reflecting reward uncertainty

The acknowledged importance of uncertainty in economic decision making has stimulated the search for neural signals that could influence learning and inform decision mechanisms. Current views distinguish two forms of uncertainty, namely risk and ambiguity, depending on whether the probability distributions of outcomes are known or unknown. Behavioural neurophysiological studies on dopamine neurons revealed a risk signal, which covaried with the standard deviation or variance of the magnitude of juice rewards and occurred separately from reward value coding. Human imaging studies identified similarly distinct risk signals for monetary rewards in the striatum and orbitofrontal cortex (OFC), thus fulfilling a requirement for the mean variance approach of economic decision theory. The orbitofrontal risk signal covaried with individual risk attitudes, possibly explaining individual differences in risk perception and risky decision making. Ambiguous gambles with incomplete probabilistic information induced stronger brain signals than risky gambles in OFC and amygdala, suggesting that the brains reward system signals the partial lack of information. The brain can use the uncertainty signals to assess the uncertainty of rewards, influence learning, modulate the value of uncertain rewards and make appropriate behavioural choices between only partly known options.

Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning

Decision-making in the presence of other competitive intelligent agents is fundamental for social and economic behavior. Such decisions require agents to behave strategically, where in addition to learning about the rewards and punishments available in the environment, they also need to anticipate and respond to actions of others competing for the same rewards. However, whereas we know much about strategic learning at both theoretical and behavioral levels, we know relatively little about the underlying neural mechanisms. Here, we show using a multi-strategy competitive learning paradigm that strategic choices can be characterized by extending the reinforcement learning (RL) framework to incorporate agents’ beliefs about the actions of their opponents. Furthermore, using this characterization to generate putative internal values, we used model-based functional magnetic resonance imaging to investigate neural computations underlying strategic learning. We found that the distinct notions of prediction errors derived from our computational model are processed in a partially overlapping but distinct set of brain regions. Specifically, we found that the RL prediction error was correlated with activity in the ventral striatum. In contrast, activity in the ventral striatum, as well as the rostral anterior cingulate (rACC), was correlated with a previously uncharacterized belief-based prediction error. Furthermore, activity in rACC reflected individual differences in degree of engagement in belief learning. These results suggest a model of strategic behavior where learning arises from interaction of dissociable reinforcement and belief-based inputs.

Damage To Dorsolateral Prefrontal Cortex Affects Tradeoffs Between Honesty And Self-Interest

Substantial correlational evidence suggests that prefrontal regions are critical to honest and dishonest behavior, but causal evidence specifying the nature of this involvement remains absent. We found that lesions of the human dorsolateral prefrontal cortex (DLPFC) decreased the effect of honesty concerns on behavior in economic games that pit honesty motives against self-interest, but did not affect decisions when honesty concerns were absent. These results point to a causal role for DLPFC in honest behavior.

The Neurobiological Foundations of Valuation in Human Decision-making under Uncertainty

This chapter focuses on valuation in the context of choice under uncertainty. When valuation is a pursuit distinct from choice, values revealed through choice may be different from the valuations that emerge from the (choice-independent) computations. In many human endeavors, valuation is performed even in the absence of any immediate necessity to choose. Again, finance is a case in point, in part because financial valuation is often complex and time-consuming, while good choice opportunities are rare and short-lived. The cost of computing values provides a normative rationale for why valuation may be done in the absence of free choice. In fact, the neurobiological choice model predicts choices better than a utility-index-based model estimated from the choices themselves. This demonstrates not only that valuation is done during imperative trials, but that the resulting values are relevant for choice in free-choice trials as well. Although brain activation during imperative trials reflects valuations that are compatible with the values revealed in free-choice trials, and hence that brain activation in imperative trials can be used to predict choice in free-choice trials, the fit is not 100%. These neurobiological data suggest that there are two value signals: one revealed through activation in brain regions, not directly involved in the physical implementation of choice, and a second one revealed through activation of the neurons controlling the physical act of choice.

Dopamine Modulates Egalitarian Behavior in Humans

Egalitarian motives form a powerful force in promoting prosocial behavior and enabling large-scale cooperation in the human species [1]. At the neural level, there is substantial, albeit correlational, evidence suggesting a link between dopamine and such behavior [2, 3]. However, important questions remain about the specific role of dopamine in setting or modulating behavioral sensitivity to prosocial concerns. Here, using a combination of pharmacological tools and economic games, we provide critical evidence for a causal involvement of dopamine in human egalitarian tendencies. Specifically, using the brain penetrant catechol-O-methyl transferase (COMT) inhibitor tolcapone [4, 5], we investigated the causal relationship between dopaminergic mechanisms and two prosocial concerns at the core of a number of widely used economic games: (1) the extent to which individuals directly value the material payoffs of others, i.e., generosity, and (2) the extent to which they are averse to differences between their own payoffs and those of others, i.e., inequity. We found that dopaminergic augmentation via COMT inhibition increased egalitarian tendencies in participants who played an extended version of the dictator game [6]. Strikingly, computational modeling of choice behavior [7] revealed that tolcapone exerted selective effects on inequity aversion, and not on other computational components such as the extent to which individuals directly value the material payoffs of others. Together, these data shed light on the causal relationship between neurochemical systems and human prosocial behavior and have potential implications for our understanding of the complex array of social impairments accompanying neuropsychiatric disorders involving dopaminergic dysregulation.

Dissociable contribution of prefrontal and striatal dopaminergic genes to learning in economic games

Significance Game theory is used throughout the social and biological sciences to study behavior in social interactions. Recent research suggests an important role for the dopamine neurotransmitter system in these types of decisions. This study used a competitive game to study how people varied in their decision-making processes and related these differences in the set of genes that carry out biological functions required for dopaminergic functioning. We found that genes differentially expressed in separate brain regions influenced distinct components of people’s decision-making processes and that a surprising degree of consistency exists with what is known at the brain level about how people make decisions in social interactions. Game theory describes strategic interactions where success of players’ actions depends on those of coplayers. In humans, substantial progress has been made at the neural level in characterizing the dopaminergic and frontostriatal mechanisms mediating such behavior. Here we combined computational modeling of strategic learning with a pathway approach to characterize association of strategic behavior with variations in the dopamine pathway. Specifically, using gene-set analysis, we systematically examined contribution of different dopamine genes to variation in a multistrategy competitive game captured by (i) the degree players anticipate and respond to actions of others (belief learning) and (ii) the speed with which such adaptations take place (learning rate). We found that variation in genes that primarily regulate prefrontal dopamine clearance—catechol-O-methyl transferase (COMT) and two isoforms of monoamine oxidase—modulated degree of belief learning across individuals. In contrast, we did not find significant association for other genes in the dopamine pathway. Furthermore, variation in genes that primarily regulate striatal dopamine function—dopamine transporter and D2 receptors—was significantly associated with the learning rate. We found that this was also the case with COMT, but not for other dopaminergic genes. Together, these findings highlight dissociable roles of frontostriatal systems in strategic learning and support the notion that genetic variation, organized along specific pathways, forms an important source of variation in complex phenotypes such as strategic behavior.

Dopamine-system genes and cultural acquisition: the norm sensitivity hypothesis §

Previous research in cultural psychology shows that cultures vary in the social orientation of independence and interdependence. To date, however, little is known about how people may acquire such global patterns of cultural behavior or cultural norms. Nor is it clear what genetic mechanisms may underlie the acquisition of cultural norms. Here, we draw on recent evidence for certain genetic variability in the susceptibility to environmental influences and propose a norm sensitivity hypothesis, which holds that people acquire culture, and rules of cultural behaviors, through reinforcement-mediated social learning processes. One corollary of the hypothesis is that the degree of cultural acquisition should be influenced by polymorphic variants of genes involved in dopaminergic neural pathways, which have been widely implicated in reinforcement learning. We reviewed initial evidence for this prediction and discussed challenges and directions for future research.