Damon Tomlin

Towards Human–Robot Teams: Model-Based Analysis of Human Decision Making in Two-Alternative Choice Tasks With Social Feedback

With a principled methodology for systematic design of human-robot decision-making teams as a motivating goal, we seek an analytic, model-based description of the influence of team and network design parameters on decision-making performance. Given that there are few reliably predictive models of human decision making, we consider the relatively well-understood two-alternative choice tasks from cognitive psychology, where individuals make sequential decisions with limited information, and we study a stochastic decision-making model, which has been successfully fitted to human behavioral and neural data for a range of such tasks. We use an extension of the model, fitted to experimental data from groups of humans performing the same task simultaneously and receiving feedback on the choices of others in the group. First, we show how the task and model can be regarded as a Markov process. Then, we derive analytically the steady-state probability distributions for decisions and performance as a function of model and design parameters such as the strength and path of the social feedback. Finally, we discuss application to human-robot team and network design and next steps with a multirobot testbed.

A simple decision task in a social context: Experiments, a model, and preliminary analyses of behavioral data

To investigate the influence of input from fellow group members in a constrained decision-making context, we consider a game in which subjects freely select one of two options (A or B), and are informed of the reward resulting from that choice following each trial. Rewards are computed based on the fraction x of past A choices by two functions fA(x), fB(x) (unknown to the subject) which intersect at a matching point x¿ that does not generally represent globally optimal behavior. Playing individually, subjects typically remain close to the matching point, although some discover the optimum. We investigate the effects of additional feedback regarding the choices and reward scores of other players. We generalize a drift-diffusion model, commonly used to model individual decision making, to incorporate feedback from other players, study the resulting coupled stochastic differential equations, and compare the distributions of choices that they predict with those produced by a pool of subjects playing in groups of five without feedback and with feedback on other players¿ choices.

The evolution and devolution of cognitive control: The costs of deliberation in a competitive world

Dual-system theories of human cognition, under which fast automatic processes can complement or compete with slower deliberative processes, have not typically been incorporated into larger scale population models used in evolutionary biology, macroeconomics, or sociology. However, doing so may reveal important phenomena at the population level. Here, we introduce a novel model of the evolution of dual-system agents using a resource-consumption paradigm. By simulating agents with the capacity for both automatic and controlled processing, we illustrate how controlled processing may not always be selected over rigid, but rapid, automatic processing. Furthermore, even when controlled processing is advantageous, frequency-dependent effects may exist whereby the spread of control within the population undermines this advantage. As a result, the level of controlled processing in the population can oscillate persistently, or even go extinct in the long run. Our model illustrates how dual-system psychology can be incorporated into population-level evolutionary models, and how such a framework can be used to examine the dynamics of interaction between automatic and controlled processing that transpire over an evolutionary time scale.

Deterministic Modeling and Evaluation of Decision-Making Dynamics in Sequential Two-Alternative Forced Choice Tasks

The focus of the work in this paper is a systems-theoretic construction, analysis, and evaluation of a deterministic model of human decision making relative to experimental data. In sequential two-alternative forced choice decision tasks, a human subject is presented with two choices at every time step, is given finite time to select one of the choices, and receives a reward after a choice is made (presented as a number on a computer screen). The goal for the human is to obtain the maximal reward while not knowing the underlying reward assignment process. In this work, we present a parameterized deterministic model for human decision making in this context and analyze optimality and stability using a finite state machine approach. This model is then evaluated relative to experimental data from human subjects performing each of six tasks.

Cyclical Population Dynamics of Automatic Versus Controlled Processing: An Evolutionary Pendulum

Psychologists, neuroscientists, and economists often conceptualize decisions as arising from processes that lie along a continuum from automatic (i.e., “hardwired” or overlearned, but relatively inflexible) to controlled (less efficient and effortful, but more flexible). Control is central to human cognition, and plays a key role in our ability to modify the world to suit our needs. Given its advantages, reliance on controlled processing may seem predestined to increase within the population over time. Here, we examine whether this is so by introducing an evolutionary game theoretic model of agents that vary in their use of automatic versus controlled processes, and in which cognitive processing modifies the environment in which the agents interact. We find that, under a wide range of parameters and model assumptions, cycles emerge in which the prevalence of each type of processing in the population oscillates between 2 extremes. Rather than inexorably increasing, the emergence of control often creates conditions that lead to its own demise by allowing automaticity to also flourish, thereby undermining the progress made by the initial emergence of controlled processing. We speculate that this observation may have relevance for understanding similar cycles across human history, and may lend insight into some of the circumstances and challenges currently faced by our species.

Modeling and evaluation of decision-making dynamics in sequential two-alternative forced choice tasks

The focus of the work in this paper is the evaluation of a model of human decision making relative to experimental data. In sequential two-alternative forced choice decision tasks, a human subject is presented with two alternative choices at every time step, and the human gets a reward (presented as a number on the screen) after a choice is made. The goal for the human is to obtain the maximal reward. This type of experiment is carried out by cognitive scientists and psychologists in order to study human behaviors in decision making. In this work, we present a simple idealized model for human decision-making with several extensions, and we discuss evaluation of the model using experimental data from human subjects.

The integration of social influence and reward: Computational approaches and neural evidence

Decades of research have established that decision-making is dramatically impacted by both the rewards an individual receives and the behavior of others. How do these distinct influences exert their influence on an individual’s actions, and can the resulting behavior be effectively captured in a computational model? To address this question, we employed a novel spatial foraging game in which groups of three participants sought to find the most rewarding location in an unfamiliar two-dimensional space. As the game transitioned from one block to the next, the availability of information regarding other group members was varied systematically, revealing the relative impacts of feedback from the environment and information from other group members on individual decision-making. Both reward-based and socially-based sources of information exerted a significant influence on behavior, and a computational model incorporating these effects was able to recapitulate several key trends in the behavioral data. In addition, our findings suggest how these sources were processed and combined during decision-making. Analysis of reaction time, location of gaze, and functional magnetic resonance imaging (fMRI) data indicated that these distinct sources of information were integrated simultaneously for each decision, rather than exerting their influence in a separate, all-or-none fashion across separate subsets of trials. These findings add to our understanding of how the separate influences of reward from the environment and information derived from other social agents are combined to produce decisions.

Rational Constraints and the Evolution of Fairness in the Ultimatum Game.

Behavior in the Ultimatum Game has been well-studied experimentally, and provides a marked contrast between the theoretical model of a self-interested economic agent and that of an actual human concerned with social norms such as fairness. How did such norms evolve, when punishing unfair behavior can be costly to the punishing agent? The work described here simulated a series of Ultimatum Games, in which populations of agents earned resources based on their preferences for proposing and accepting (or rejecting) offers of various sizes. Two different systems governing the acceptance or rejection of offers were implemented. Under one system, the probability that an agent accepted an offer of a given size was independent of the probabilities of accepting the other possible offers. Under the other system, a simple, ordinal constraint was placed on the acceptance probabilities such that a given offer was at least as likely to be accepted as a smaller offer. For simulations under either system, agents’ preferences and their corresponding behavior evolved over multiple generations. Populations without the ordinal constraint came to emulate maximizing economic agents, while populations with the constraint came to resemble the behavior of human players.