Mark Dean

Testing the reward prediction error hypothesis with an axiomatic model

Neuroimaging studies typically identify neural activity correlated with the predictions of highly parameterized models, like the many reward prediction error (RPE) models used to study reinforcement learning. Identified brain areas might encode RPEs or, alternatively, only have activity correlated with RPE model predictions. Here, we use an alternate axiomatic approach rooted in economic theory to formally test the entire class of RPE models on neural data. We show that measurements of human neural activity from the striatum, medial prefrontal cortex, amygdala, and posterior cingulate cortex satisfy necessary and sufficient conditions for the entire class of RPE models. However, activity measured from the anterior insula falsifies the axiomatic model, and therefore no RPE model can account for measured activity. Further analysis suggests the anterior insula might instead encode something related to the salience of an outcome. As cognitive neuroscience matures and models proliferate, formal approaches of this kind that assess entire model classes rather than specific model exemplars may take on increased significance.

Search, choice, and revealed preference

With complete information, choice of one option over another conveys preference. Yet when search is incomplete, this is not necessarily the case. It may instead reflect unawareness that a superior alternative was available. To separate these phenomena, we consider non-standard data on the evolution of provisional choices with contemplation time. We characterize precisely when the resulting data could have been generated by a general form of sequential search. We characterize also search that terminates based on a reservation utility stopping rule. We outline an experimental design that captures provisional choices in the pre-decision period.

Trading off speed and accuracy in rapid, goal-directed movements

Many studies have shown that humans face a trade-off between the speed and accuracy with which they can make movements. In this article, we asked whether humans choose movement time to maximize expected gain by taking into account their own speed-accuracy trade-off (SAT). We studied this question within the context of a rapid pointing task in which subjects received a reward for hitting a target on a monitor. The experimental design we used had two parts. First, we estimated individual trade-offs by motivating subjects to perform the pointing task under four different time constraints. Second, we tested whether subjects selected movement time optimally in an environment where they were rewarded for both speed and accuracy; the value of the target decreased linearly over time to zero. We ran two conditions in which the subjects faced different decay rates. Overall, the performance of 13 out of 16 subjects was indistinguishable from optimal. We concluded that in planning movements, humans take into account their own SAT to maximize expected gain.

Axiomatic methods, dopamine and reward prediction error

The phasic firing rate of midbrain dopamine neurons has been shown to respond both to the receipt of rewarding stimuli, and the degree to which such stimuli are anticipated by the recipient. This has led to the hypothesis that these neurons encode reward prediction error (RPE)-the difference between how rewarding an event is, and how rewarding it was expected to be. However, the RPE model is one of a number of competing explanations for dopamine activity that have proved hard to disentangle, mainly because they are couched in terms of latent, or unobservable, variables. This article describes techniques for dealing with latent variables common in economics and decision theory, and reviews work that uses these techniques to provide simple, non-parametric tests of the RPE hypothesis, allowing clear differentiation between competing explanations.

Measuring Rationality with the Minimum Cost of Revealed Preference Violations

We introduce a new measure of how close a set of choices are to satisfying the observable implications of rational choice and apply it to a large balanced panel of household level consumption data. This new measure, the Minimum Cost Index, is the minimum cost of breaking all revealed preference cycles found in choices from budget sets. Unlike existing measures of rationality, it responds to both the number and severity of revealed preference violations.

A game theoretic approach to multimodal communication

Over the last few decades the animal communication community has become increasingly aware that much communication occurs using multiple signals in multiple modalities. The majority of this work has been empirical, with less theoretical work on the advantages conferred by such communication. In the present paper, we ask: Why should animals communicate with multiple signals in multiple modalities? To tackle this question we use game theoretic techniques, and highlight developments in the economic signaling literature that might offer insight into biological problems. We start by establishing a signaling game, and investigate signal honesty under two prevailing paradigms of honest communication – costly signaling and cheap talk. In both paradigms, without further constraint, it is simple to show that anything that can be achieved with multiple signals can be achieved with one. We go on to investigate different sets of possible constraints that may make multiple signals and multimodal signals in particular more likely to evolve. We suggest that constraints on cost functions and bandwidths, orthogonal noise across modalities, strategically distinct modes, multiple qualities, multiple signalers, and multiple audiences, all provide biologically plausible scenarios that theoretically favor multiple and multimodal signaling.

Credit Constraints and the Measurement of Time Preferences

Incentivized experiments are commonly used to estimate marginal rate of intertemporal substitution (MRS) in the lab and in the ?eld, and to make inferences about subject’s time preference. This paper considers the implications of an integrated model of behavior in which individuals are subject to ?nancial shocks and credit constraints and take those into account when making experimental choices. The model shows that measured MRS depends on the individual’s e?ective interest rate and her marginal utility of current and future consumption. Experimental responses should therefore be correlated with other variables that describe the subject’s ?nancial situation, like savings, income and consumption shocks. We test the model with a panel data set from Mali and ?nd evidence for such e?ects. We discuss how our model can be combined with repeated time preference measures to identify time preferences and other household characteristics - including credit constraints and the importance of di?erent types of ?nancial shocks.

Is it All Connected? A Testing Ground for Unified Theories of Behavioral Economics Phenomena

We estimate 11 well-studied behavioral phenomena in a group of 190 laboratory subjects (short-term discount rates, small stakes risk aversion, present bias, loss aversion, the endowment effect, aversion to ambiguity and compound lotteries, the common ratio and common consequence effects and sender/receiver behavior in trust games). We study the joint distribution of these behaviors and compare it to the predictions of existing models as a step in the development of a parsimonious, general model of economic choice. We find strong correlations between loss aversion and the endowment effect, and between probability weighting (as measured by the common ratio and common consequence effects) and risk aversion, in line with Cumulative Prospect Theory (CPT). We also find risk aversion to be related to ambiguity aversion, compound lottery aversion and discounting, consistent with the curvature of the utility function being an important determinant of all three behaviors. However, we do not find evidence that probability weighting in the risk domain is related to ambiguity and compound lottery aversion or present bias (as implied by recent extensions to CPT). Behavior in the trust game is unrelated to attitudes to risk or uncertainty. We find little relation between intelligence or personality measures and economic behavior, although we do find overconfidence to be negatively related to many behaviors (particularly ambiguity aversion) and women to be much more loss averse than men.

Limited attention and status quo bias

We introduce and axiomatically characterize a model of status quo bias in which the status quo affects choices by both imposing psychological constraints and focusing attention. The resulting Limited Attention Status Quo Bias model can explain both the findings that status quo bias is more prevalent in larger choice sets and that the introduction of a status quo can change choices between non-status quo alternatives. Existing models of status quo bias are inconsistent with the former finding while models of decision avoidance are inconsistent with the latter. We show that the interaction of the two effects has important economic implications, and report the results of laboratory experiments which show that both attention and psychological constraints are necessary to explain the impact of status quo on choice.

How can neuroscience inform economics

Neuroeconomics is now a well-established discipline at the intersection of neuroscience, psychology and economics, yet its influence on mainstream economics has been smaller than on the other two fields. This is in part because, unlike neuroscientists and psychologists, most economists are not interested in the process of decision making per se . We argue that neuroscience is most likely to influence economics in the short run by providing new insights into the relationships between variables that economists already study. In recent years the field has made many such contributions, using models from cognitive neuroscience to better explain choice behavior. Here we review the work that we think has great promise to contribute to economics in the near future.