Davide Marchiori

Constructing shared interpretations in a team of intelligent agents: the effects of communication intensity and structure

In this paper we explore a model of a team of intelligent agents constructing a shared interpretation of the state of their environment. Each agent is modeled as a constraint satisfaction network of the Hopfield (1982) type.

Neural network models of learning and categorization in multigame experiments.

Previous research has shown that regret-driven neural networks predict behavior in repeated completely mixed games remarkably well, substantially equating the performance of the most accurate established models of learning. This result prompts the question of what is the added value of modeling learning through neural networks. We submit that this modeling approach allows for models that are able to distinguish among and respond differently to different payoff structures. Moreover, the process of categorization of a game is implicitly carried out by these models, thus without the need of any external explicit theory of similarity between games. To validate our claims, we designed and ran two multigame experiments in which subjects faced, in random sequence, different instances of two completely mixed 2 × 2 games. Then, we tested on our experimental data two regret-driven neural network models, and compared their performance with that of other established models of learning and Nash equilibrium.

On loss aversion, level-1 reasoning, and betting

Previous research suggests that human reaction to risky opportunities reflects two contradicting biases: “loss aversion”, and “limited level of reasoning” that leads to overconfidence. Rejection of attractive gambles is explained by loss aversion, while counterproductive risk seeking is attributed to limited level of reasoning. The current research highlights a shortcoming of this popular (but often implicit) “contradicting biases” assertion. Studies of “negative-sum betting games” reveal high rate of counterproductive betting even when limited level of reasoning and loss aversion imply no betting. The results reflect two reasons for the high betting rate: initial tendency to participate and slow learning. Under certain conditions, the observed betting rate was higher than the rate predicted under random choice even after 250 trials with immediate feedback. These results can be captured with a model that assumes a tendency to select strategies that have led to good outcomes in a small set of similar past experiences, and allows for an initial framing effect.

Toward a general theoretical framework for judgment and decision-making

Over the past 30 years, behavioral and experimental economists and psychologists have made great strides in identifying phenomena that cannot be explained by the classical model of rational choice—anomalies in the discounting of future wealth, present bias, loss aversion, the endowment effect, and aversion to ambiguity, for example. In response to these findings, there has been an enormous amount of research by behavioral scientists aimed at modeling and understanding the nature of these biases1. However, these models, typically assuming situation-specific psychological processes, have shed limited light on the conditions for and boundaries of the different biases, substantially neglecting their relative importance and joint effect. Much less attention has been paid to the investigation of the links between different biases. As a consequence of this approach, it is not always clear which model should be used to predict behavior in a new setting, and maybe a more general theory is needed. We believe that the field of neuroeconomics, which has experienced a rapid growth over the past decade, can play an important role in bridging these gaps, contributing to the building of a general theoretical framework for judgment and decision-making behaviors.

Pack Light on the Move: Exploitation and Exploration in a Dynamic Environment

This paper revisits a recent study by Posen and Levinthal (2012) on the exploration/exploitation tradeoff for a multi-armed bandit problem, where the reward probabilities undergo random shocks. We show that their analysis suffers two shortcomings: it assumes that learning is based on stale evidence, and it overlooks the steady state. We let the learning rule endogenously discard stale evidence, and we perform the long run analyses. The comparative study demonstrates that some of their conclusions must be qualified.

Physiological plausibility and boundary conditions of theories of risk sensitivity.

The contribution by Yechiam and Telpaz (Y&T) published in Frontiers in Cognitive Science places it in a corpus of literature which bridges at least three different disciplines, i.e., psychology, economics, and neuroscience. The goal of this line of research is to explore the neurological and physiological underpinnings of one of the central topics in judgment and decision-making (JDM) research – choice behavior in decisions from experience. Y&T successfully contributes to this goal by demonstrating a novel effect that losses increase experimental participants’ arousal as measured by pupil dilatation, which in turn positively correlates with a risk aversion behavior. They hypothesize that participants’ attention is increased in decision problems involving losses, which trigger an innate prudent behavior in situations entailing danger and/or hazard. Interestingly, Y&T find that the nature of attention is not selective, i.e., when losses are present, participants are shown to devote more attention to the task as a whole rather than to the single negative outcomes, in contrast to Prospect Theorys loss aversion. YT Tom et al., 2007). These studies suggest that behavioral loss aversion in decisions from description reflects an asymmetric response to gain and losses in the neural system encoding for reward values (the ventromedial prefrontal cortex, orbitofrontal cortex, and ventral striatum). What makes Y&Ts contribution particularly noteworthy is their mediating attentional hypothesis, which links physiological mechanisms to the psychological processes involved in experience-based decisions. One of the possible future developments from YT Holt and Laury, 2005). Specifically, it has been observed that participants’ degree of risk aversion increases significantly as actual positive payoffs are scaled up, and that this effect is negligible when payoffs are hypothetical. These findings provide an opportunity to widen the scope of the attentional hypothesis. Specifically, payoffs corresponding to large cash amounts might have the analogous effects of losses of increasing arousal and of triggering a higher level of risk aversion; whereas hypothetical payoffs might result in a substantial inhibition of attention. Therefore, the motivation implied by real stakes can be interpreted as one of the possible boundary conditions (see below) for Y&Ts attentional hypothesis, giving rise to a question of the relative weight of attention and motivation in shaping risk attitudes. Y&Ts report can also be contextualized within the wide literature on individual differences in reasoning, judgment and decision making (e.g., Stanovich and West, 2000) and their implications to the rationality debate. The prototypical finding in that literature is the correlation between cognitive ability and normative responding, with a strong emphasis on normative evaluation of rationality. This so-called “normativist” approach has recently been subject to criticism (Elqayam and Evans, 2011) as unhelpful in developing a psychological theory of human rationality. It is therefore noteworthy that Y&T take their individual differences work in a completely different direction, with what seems to be a purely ‘descriptivist’ approach, with no normativist connotations. As one reviewer of this manuscript put it, any behavior in this setting could be justified as ‘rational’. The behavioral patterns described vary qualitatively rather than quantitatively. This is typical of descriptivist approaches to cognitive variability higher mental processing (Evans and Elqayam, 2011). Given the dearth of such focus in higher mental processing, this is a welcome development. Lastly, a potentially significant issue here is the implications to risk aversion as originally portrayed in prospect theory (Kahneman and Tversky, 1979). One could argue that Y&T contribute to defining boundary conditions for Prospect Theory, by proposing an alternative explanation for specific settings in which Prospect Theory is not supported by empirical evidence1. Indeed, as a unified theory of risk aversion is not yet at hand, knowing the range of application of each of the existing theories is crucial. One reason that YT the algorithmic level, which has to do with processes (e.g., the calculators software); and the implementational level, which explores the physical underpinnings of the system – its hardware/wetware characterization (e.g., the calculators chip). Viewed in these terms, we see prospect theory as portraying behavior mainly on the computational (i.e., functional) level of analysis; or, as some authors put it – an “axiomatic” system (see Wakker, 2010). In contrast, YT Tom et al., 2007), and studies that combine several levels of analysis, as Y&T have done, are even rarer. This makes Y&Ts contribution of particular interest to scholars of human thinking and decision making.

Hierarchical decision-making produces persistent differences in learning performance

Human organizations are commonly characterized by a hierarchical chain of command that facilitates division of labor and integration of effort. Higher-level employees set the strategic frame that constrains lower-level employees who carry out the detailed operations serving to implement the strategy. Typically, strategy and operational decisions are carried out by different individuals that act over different timescales and rely on different kinds of information. We hypothesize that when such decision processes are hierarchically distributed among different individuals, they produce highly heterogeneous and strongly path-dependent joint learning dynamics. To investigate this, we design laboratory experiments of human dyads facing repeated joint tasks, in which one individual is assigned the role of carrying out strategy decisions and the other operational ones. The experimental behavior generates a puzzling bimodal performance distribution–some pairs learn, some fail to learn after a few periods. We also develop a computational model that mirrors the experimental settings and predicts the heterogeneity of performance by human dyads. Comparison of experimental and simulation data suggests that self-reinforcing dynamics arising from initial choices are sufficient to explain the performance heterogeneity observed experimentally.

Noisy Retrievers and the Fourfold Reaction to Rare Events

Previous research documents two pairs of inconsistent reactions to rare events: 1) Studies of probability judgment reveal conservatism which implies overestimation of rare events, and overconfidence which implies underestimation of rare events. 2) Studies of choice behavior reveal overweighting of rare events in one-shot tasks, and the opposite bias in decisions from experience. The current analysis and experimental results demonstrate that the coexistence and relative importance of the four biases can be captured with simple models that share the assumption that judgments and decisions are made based on the information conveyed by small and noisy samples of past experiences.

Physiological plausibility and boundary conditions of theories of risk sensitivity: Commentary on Yechiam & Telpaz.

The contribution by Yechiam and Telpaz (Y&T) published in Frontiers in Cognitive Science places it in a corpus of literature which bridges at least three different disciplines, i.e., psychology, economics, and neuroscience. The goal of this line of research is to explore the neurological and physiological underpinnings of one of the central topics in judgment and decision-making (JDM) research – choice behavior in decisions from experience. Y&T successfully contributes to this goal by demonstrating a novel effect that losses increase experimental participants’ arousal as measured by pupil dilatation, which in turn positively correlates with a risk aversion behavior. They hypothesize that participants’ attention is increased in decision problems involving losses, which trigger an innate prudent behavior in situations entailing danger and/or hazard. Interestingly, Y&T find that the nature of attention is not selective, i.e., when losses are present, participants are shown to devote more attention to the task as a whole rather than to the single negative outcomes, in contrast to Prospect Theorys loss aversion. YT Tom et al., 2007). These studies suggest that behavioral loss aversion in decisions from description reflects an asymmetric response to gain and losses in the neural system encoding for reward values (the ventromedial prefrontal cortex, orbitofrontal cortex, and ventral striatum). What makes Y&Ts contribution particularly noteworthy is their mediating attentional hypothesis, which links physiological mechanisms to the psychological processes involved in experience-based decisions. One of the possible future developments from YT Holt and Laury, 2005). Specifically, it has been observed that participants’ degree of risk aversion increases significantly as actual positive payoffs are scaled up, and that this effect is negligible when payoffs are hypothetical. These findings provide an opportunity to widen the scope of the attentional hypothesis. Specifically, payoffs corresponding to large cash amounts might have the analogous effects of losses of increasing arousal and of triggering a higher level of risk aversion; whereas hypothetical payoffs might result in a substantial inhibition of attention. Therefore, the motivation implied by real stakes can be interpreted as one of the possible boundary conditions (see below) for Y&Ts attentional hypothesis, giving rise to a question of the relative weight of attention and motivation in shaping risk attitudes. Y&Ts report can also be contextualized within the wide literature on individual differences in reasoning, judgment and decision making (e.g., Stanovich and West, 2000) and their implications to the rationality debate. The prototypical finding in that literature is the correlation between cognitive ability and normative responding, with a strong emphasis on normative evaluation of rationality. This so-called “normativist” approach has recently been subject to criticism (Elqayam and Evans, 2011) as unhelpful in developing a psychological theory of human rationality. It is therefore noteworthy that Y&T take their individual differences work in a completely different direction, with what seems to be a purely ‘descriptivist’ approach, with no normativist connotations. As one reviewer of this manuscript put it, any behavior in this setting could be justified as ‘rational’. The behavioral patterns described vary qualitatively rather than quantitatively. This is typical of descriptivist approaches to cognitive variability higher mental processing (Evans and Elqayam, 2011). Given the dearth of such focus in higher mental processing, this is a welcome development. Lastly, a potentially significant issue here is the implications to risk aversion as originally portrayed in prospect theory (Kahneman and Tversky, 1979). One could argue that Y&T contribute to defining boundary conditions for Prospect Theory, by proposing an alternative explanation for specific settings in which Prospect Theory is not supported by empirical evidence1. Indeed, as a unified theory of risk aversion is not yet at hand, knowing the range of application of each of the existing theories is crucial. One reason that YT the algorithmic level, which has to do with processes (e.g., the calculators software); and the implementational level, which explores the physical underpinnings of the system – its hardware/wetware characterization (e.g., the calculators chip). Viewed in these terms, we see prospect theory as portraying behavior mainly on the computational (i.e., functional) level of analysis; or, as some authors put it – an “axiomatic” system (see Wakker, 2010). In contrast, YT Tom et al., 2007), and studies that combine several levels of analysis, as Y&T have done, are even rarer. This makes Y&Ts contribution of particular interest to scholars of human thinking and decision making.

Toward a Physiologically Grounded Theory of Risk Sensitivity: Commentary on Yechiam & Telpaz

The contribution by Yechiam and Telpaz (Y&T) published in Frontiers in Cognitive Science places it in a corpus of literature which bridges at least three different disciplines, i.e., psychology, economics, and neuroscience. The goal of this line of research is to explore the neurological and physiological underpinnings of one of the central topics in judgment and decision-making (JDM) research – choice behavior in decisions from experience. Y&T successfully contributes to this goal by demonstrating a novel effect that losses increase experimental participants’ arousal as measured by pupil dilatation, which in turn positively correlates with a risk aversion behavior. They hypothesize that participants’ attention is increased in decision problems involving losses, which trigger an innate prudent behavior in situations entailing danger and/or hazard. Interestingly, Y&T find that the nature of attention is not selective, i.e., when losses are present, participants are shown to devote more attention to the task as a whole rather than to the single negative outcomes, in contrast to Prospect Theorys loss aversion. YT Tom et al., 2007). These studies suggest that behavioral loss aversion in decisions from description reflects an asymmetric response to gain and losses in the neural system encoding for reward values (the ventromedial prefrontal cortex, orbitofrontal cortex, and ventral striatum). What makes Y&Ts contribution particularly noteworthy is their mediating attentional hypothesis, which links physiological mechanisms to the psychological processes involved in experience-based decisions. One of the possible future developments from YT Holt and Laury, 2005). Specifically, it has been observed that participants’ degree of risk aversion increases significantly as actual positive payoffs are scaled up, and that this effect is negligible when payoffs are hypothetical. These findings provide an opportunity to widen the scope of the attentional hypothesis. Specifically, payoffs corresponding to large cash amounts might have the analogous effects of losses of increasing arousal and of triggering a higher level of risk aversion; whereas hypothetical payoffs might result in a substantial inhibition of attention. Therefore, the motivation implied by real stakes can be interpreted as one of the possible boundary conditions (see below) for Y&Ts attentional hypothesis, giving rise to a question of the relative weight of attention and motivation in shaping risk attitudes. Y&Ts report can also be contextualized within the wide literature on individual differences in reasoning, judgment and decision making (e.g., Stanovich and West, 2000) and their implications to the rationality debate. The prototypical finding in that literature is the correlation between cognitive ability and normative responding, with a strong emphasis on normative evaluation of rationality. This so-called “normativist” approach has recently been subject to criticism (Elqayam and Evans, 2011) as unhelpful in developing a psychological theory of human rationality. It is therefore noteworthy that Y&T take their individual differences work in a completely different direction, with what seems to be a purely ‘descriptivist’ approach, with no normativist connotations. As one reviewer of this manuscript put it, any behavior in this setting could be justified as ‘rational’. The behavioral patterns described vary qualitatively rather than quantitatively. This is typical of descriptivist approaches to cognitive variability higher mental processing (Evans and Elqayam, 2011). Given the dearth of such focus in higher mental processing, this is a welcome development. Lastly, a potentially significant issue here is the implications to risk aversion as originally portrayed in prospect theory (Kahneman and Tversky, 1979). One could argue that Y&T contribute to defining boundary conditions for Prospect Theory, by proposing an alternative explanation for specific settings in which Prospect Theory is not supported by empirical evidence1. Indeed, as a unified theory of risk aversion is not yet at hand, knowing the range of application of each of the existing theories is crucial. One reason that YT the algorithmic level, which has to do with processes (e.g., the calculators software); and the implementational level, which explores the physical underpinnings of the system – its hardware/wetware characterization (e.g., the calculators chip). Viewed in these terms, we see prospect theory as portraying behavior mainly on the computational (i.e., functional) level of analysis; or, as some authors put it – an “axiomatic” system (see Wakker, 2010). In contrast, YT Tom et al., 2007), and studies that combine several levels of analysis, as Y&T have done, are even rarer. This makes Y&Ts contribution of particular interest to scholars of human thinking and decision making.