Ido Erev

Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term*

We use simple learning models to track the behavior observed in experiments concerning three extensive form games with similar perfect equilibria. In only two of the games does observed behavior approach the perfect equilibrium as players gain experience. We examine a family of learning models which possess some of the robust properties of learning noted in the psychology literature. The intermediate term predictions of these models track well the observed behavior in all three games, even though the models considered differ in their very long term predictions. We argue that for predicting observed behavior the intermediate term predictions of dynamic learning models may be even more important than their asymptotic properties. Journal of Economic Literature Classification Numbers: C7, C92.

Decisions from Experience and the Effect of Rare Events in Risky Choice

When people have access to information sources such as newspaper weather forecasts, drug-package inserts, and mutual-fund brochures, all of which provide convenient descriptions of risky prospects, they can make decisions from description. When people must decide whether to back up their computers hard drive, cross a busy street, or go out on a date, however, they typically do not have any summary description of the possible outcomes or their likelihoods. For such decisions, people can call only on their own encounters with such prospects, making decisions from experience. Decisions from experience and decisions from description can lead to dramatically different choice behavior. In the case of decisions from description, people make choices as if they overweight the probability of rare events, as described by prospect theory. We found that in the case of decisions from experience, in contrast, people make choices as if they underweight the probability of rare events, and we explored the impact of two possible causes of this underweighting—reliance on relatively small samples of information and overweighting of recently sampled information. We conclude with a call for two different theories of risky choice.

The description-experience gap in risky choice.

According to a common conception in behavioral decision research, two cognitive processes-overestimation and overweighting-operate to increase the impact of rare events on peoples choices. Supportive findings stem primarily from investigations in which people learn about options via descriptions thereof. Recently, a number of researchers have begun to investigate risky choice in settings in which people learn about options by experiential sampling over time. This article reviews work across three experiential paradigms. Converging findings show that when people make decisions based on experience, rare events tend to have less impact than they deserve according to their objective probabilities. Striking similarities in human and animal experience-based choices, ways of modeling these choices, and their implications for risk and precautionary behavior are discussed.

On Adaptation, Maximization, and Reinforcement Learning Among Cognitive Strategies

Analysis of binary choice behavior in iterated tasks with immediate feedback reveals robust deviations from maximization that can be described as indications of 3 effects: (a) a payoff variability effect, in which high payoff variability seems to move choice behavior toward random choice; (b) underweighting of rare events, in which alternatives that yield the best payoffs most of the time are attractive even when they are associated with a lower expected return; and (c) loss aversion, in which alternatives that minimize the probability of losses can be more attractive than those that maximize expected payoffs. The results are closer to probability matching than to maximization. Best approximation is provided with a model of reinforcement learning among cognitive strategies (RELACS). This model captures the 3 deviations, the learning curves, and the effect of information on uncertainty avoidance. It outperforms other models in fitting the data and in predicting behavior in other experiments.

Verbal versus Numerical Probabilities: Efficiency, Biases, and the Preference Paradox

Abstract Experts (sportswriters and broadcasters) were asked to assess the probabilities of upcoming basketball game events. Based on these predictions, decision makers (students) had to rate the attractiveness of gambles. Half of the students could win real stakes based on the quality of their decisions and the outcomes of the events, while the other half were paid a flat rate. The gambles were also constructed so as to elicit the conjunction fallacy and wishful thinking biases. While most conveyors of information used verbal terms when expressing their opinions spontaneously, most decision makers preferred to receive numerical probabilities. However, no difference between the efficiency of the verbal and the numerical assessments was found. The occurrence of judgmental biases was unrelated to communication mode, but the conjunction fallacy was marginally related to the monetary payoff condition of the students. Two possible explanations for the communication mode preference inconsistency were examined, one of which seems to be supported by the results. A theoretical framework is suggested that accounts for the present data and former results.

Comonotonic Independence: The Critical Test between Classical and Rank-Dependent Utility Theories

This article compares classical expected utility (EU) with the more general rank-dependent utility (RDU) models. The difference between the independence condition for preferences of EU and its comonotonic generalization in RDU provides the exact demarcation between EU and rank-dependent models. Other axiomatic differences are not essential. An experimental design is described that tests this difference between independence and comonotonic independence in its most basic form and is robust against violations of other assumptions that may confound the results, in particular the reduction principle and transitivity. It is well known that in the classical counterexamples to EU, comonotonic independence performs better than full-force independence. For our more general choice pairs, however, we find that comonotonic independence does not perform better. This is contrary to our prior expectation and suggests that rank-dependent models, in full generality, do not provide a descriptive improvement over EU. For rank-dependent models to have a future, submodels and choice situations need to be identified for which rank-dependence does contribute descriptively.

Evaluating and Combining Subjective Probability Estimates

This paper concerns the evaluation and combination of subjective probability estimates for categorical events. We argue that the appropriate criterion for evaluating individual and combined estimates depends on the type of uncertainty the decision maker seeks to represent, which in turn depends on his or her model of the event space. Decision makers require accurate estimates in the presence of aleatory uncertainty about exchangeable events, diagnostic estimates given epistemic uncertainty about unique events, and some combination of the two when the events are not necessarily unique, but the best equivalence class definition for exchangeable events is not apparent. Following a brief reveiw of the mathematical and empirical literature on combining judgments, we present an approach to the topic that derives from (1) a weak cognitive model of the individual that assumes subjective estimates are a function of underlying judgment perturbed by random error and (2) a classification of judgment contexts in terms of the underlying information structure. In support of our developments, we present new analyses of two sets of subjective probability estimates, one of exchangeable and the other of unique events. As predicted, mean estimates were more accurate than the individual values in the first case and more diagnostic in the second. #1997 by John Wiley & Sons, Ltd.

The Role of Personal Experience in Contributing to Different Patterns of Response to Rare Terrorist Attacks

An examination of the behavioral effect of repeated terrorist attacks reveals that local residents (of the attacked area) appear to be much less sensitive to this risk than international tourists. Furthermore, the limited sensitivity on the part of local residents seems to diminish with time, even when the attacks continue. An experimental study shows a similar pattern in a laboratory experiment that focuses on a basic decision task: when making a single decision based on a description of the problem, people tend to be more risk averse. Personal experience with the problem reduces this sensitivity. These results highlight an interesting relationship between basic decision-making research and the study of the response to traumatic events.

Provision of Step-Level Public Goods The Sequential Contribution Mechanism

Groups of five players participated in a social dilemma game in which each player receives a monetary endowment and then chooses whether to contribute it to a monetary public good. The good is provided to all group members if at least three contributions are made; it is not provided, otherwise. Experiment 1 showed that the simultaneous protocol of play—where decisions are made privately and anonymously—is significantly less effective in solving the dilemma than the sequential protocol—where decisions are made sequentially with complete information about previous decisions in the sequence. Experiment 2 replicated this finding and, in addition, showed that the sequential protocol with only information about previous noncooperative choices is significantly more effective in solving the dilemma than the sequential protocol with only information about previous cooperative choices.

THE EFFECT OF ADDING A CONSTANT TO ALL PAYOFFS : EXPERIMENTAL INVESTIGATION, AND IMPLICATIONS FOR REINFORCEMENT LEARNING MODELS

This paper examines the effect on learning in simple decision tasks of the addition of a constant to all payoffs. Experiment 1 reveals that this effect, initially observed in a probability learning task, is not limited to single person decision making under uncertainty. Experiment 2 shows that the effect is not linear. Two additional experiments show that the non-linearity cannot be explained by whether zero is in the payoff range. The implications of these results for reinforcement learning models are evaluated and two models that capture the main results are proposed. ©1999 Elsevier Science B.V. All rights reserved.