Varun Dutt

Instance-Based Learning: Integrating Sampling and Repeated Decisions from Experience.

In decisions from experience, there are 2 experimental paradigms: sampling and repeated-choice. In the sampling paradigm, participants sample between 2 options as many times as they want (i.e., the stopping point is variable), observe the outcome with no real consequences each time, and finally select 1 of the 2 options that cause them to earn or lose money. In the repeated-choice paradigm, participants select 1 of the 2 options for a fixed number of times and receive immediate outcome feedback that affects their earnings. These 2 experimental paradigms have been studied independently, and different cognitive processes have often been assumed to take place in each, as represented in widely diverse computational models. We demonstrate that behavior in these 2 paradigms relies upon common cognitive processes proposed by the instance-based learning theory (IBLT; Gonzalez, Lerch, & Lebiere, 2003) and that the stopping point is the only difference between the 2 paradigms. A single cognitive model based on IBLT (with an added stopping point rule in the sampling paradigm) captures human choices and predicts the sequence of choice selections across both paradigms. We integrate the paradigms through quantitative model comparison, where IBLT outperforms the best models created for each paradigm separately. We discuss the implications for the psychology of decision making.

Cyber Situation Awareness: Modeling Detection of Cyber Attacks With Instance-Based Learning Theory

Objective: To determine the effects of an adversary’s behavior on the defender’s accurate and timely detection of network threats. Background: Cyber attacks cause major work disruption. It is important to understand how a defender’s behavior (experience and tolerance to threats), as well as adversarial behavior (attack strategy), might impact the detection of threats. In this article, we use cognitive modeling to make predictions regarding these factors. Method: Different model types representing a defender, based on Instance-Based Learning Theory (IBLT), faced different adversarial behaviors. A defender’s model was defined by experience of threats: threat-prone (90% threats and 10% nonthreats) and nonthreat-prone (10% threats and 90% nonthreats); and different tolerance levels to threats: risk-averse (model declares a cyber attack after perceiving one threat out of eight total) and risk-seeking (model declares a cyber attack after perceiving seven threats out of eight total). Adversarial behavior is simulated by considering different attack strategies: patient (threats occur late) and impatient (threats occur early). Results: For an impatient strategy, risk-averse models with threat-prone experiences show improved detection compared with risk-seeking models with nonthreat-prone experiences; however, the same is not true for a patient strategy. Conclusions: Based upon model predictions, a defender’s prior threat experiences and his or her tolerance to threats are likely to predict detection accuracy; but considering the nature of adversarial behavior is also important. Application: Decision-support tools that consider the role of a defender’s experience and tolerance to threats along with the nature of adversarial behavior are likely to improve a defender’s overall threat detection.

A Loser Can Be a Winner: Comparison of Two Instance-based Learning Models in a Market Entry Competition

This paper presents a case of parsimony and generalization in model comparisons. We submitted two versions of the same cognitive model to the Market Entry Competition (MEC), which involved four-person and two-alternative (enter or stay out) games. Our model was designed according to the Instance-Based Learning Theory (IBLT). The two versions of the model assumed the same cognitive principles of decision making and learning in the MEC. The only difference between the two models was the assumption of homogeneity among the four participants: one model assumed homogeneous participants (IBL-same) while the other model assumed heterogeneous participants (IBL-different). The IBL-same model involved three free parameters in total while the IBL-different involved 12 free parameters, i.e., three free parameters for each of the four participants. The IBL-different model outperformed the IBL-same model in the competition, but after exposing the models to a more challenging generalization test (the Technion Prediction Tournament), the IBL-same model outperformed the IBL-different model. Thus, a loser can be a winner depending on the generalization conditions used to compare models. We describe the models and the process by which we reach these conclusions.

A generic dynamic control task for behavioral research and education

Recent research in behavioral sciences presents strong evidence of poor human understanding for dynamic systems. Computer-based dynamic control tasks have an important potential for helping behavioral scientists advance research that investigates reasons for poor understanding and for helping students understand how dynamic systems work. In this paper, we introduce a simulation called Dynamic Stocks and Flows (DSF) that portrays the basic building blocks of dynamic systems: an accumulation; an inflow and outflow determined by an environment; and an inflow and outflow determined by a decision maker. In DSF, decision makers control the accumulation to a goal level by making repeated inflow and outflow decisions. We provide details of an experiment conducted with DSF that highlight some problems people face in controlling a dynamic system with different kinds of environmental inflow and outflow functions. DSF is flexible enough to represent dynamic systems with continuous or discrete accumulations, and with real-time or event-driven decision-making. We suggest that these and other features in DSF make it a good research and educational tool.

Dissociation of S-R compatibility and Simon effects with mixed tasks and mappings.

Binary-choice reactions are typically faster when the stimulus location corresponds with that of the response than when it does not. This advantage of spatial correspondence is known as the stimulus-response compatibility (SRC) effect when the mapping of stimulus location, as the relevant stimulus dimension, is varied to be compatible or incompatible with response location. It is called the Simon effect when stimulus location is task-irrelevant. The SRC effect is eliminated when compatible and incompatible spatial mappings are mixed within a trial block, and the Simon effect is eliminated when the Simon task is mixed with the SRC task with incompatible spatial mapping. Eliminations of both types have been attributed to suppression of an automatic response-activation route. We tested predictions of this suppression hypothesis for conditions in which the SRC and Simon tasks were intermixed and the spatial mappings on the SRC trials could be compatible or incompatible. In Experiment 1, the two tasks were equally likely, as were compatible and incompatible spatial mappings on SRC trials; in Experiment 2, the SRC or Simon task was more frequent; and, in Experiment 3, the compatible or incompatible location mapping for the SRC task was more frequent. The SRC effect was absent overall in all experiments, whereas the Simon effect was robust to the manipulations and showed the characteristic decrease across the reaction time (RT) distribution. This dissociation of effects implies that the automatic response-activation route is not suppressed in mixed conditions and suggests that mixing influences the SRC and Simon effects by different means.

The Role of Inertia in Modeling Decisions from Experience with Instance-Based Learning

One form of inertia is the tendency to repeat the last decision irrespective of the obtained outcomes while making decisions from experience (DFE). A number of computational models based upon the Instance-Based Learning Theory, a theory of DFE, have included different inertia implementations and have shown to simultaneously account for both risk-taking and alternations between alternatives. The role that inertia plays in these models, however, is unclear as the same model without inertia is also able to account for observed risk-taking quite well. This paper demonstrates the predictive benefits of incorporating one particular implementation of inertia in an existing IBL model. We use two large datasets, estimation and competition, from the Technion Prediction Tournament involving a repeated binary-choice task to show that incorporating an inertia mechanism in an IBL model enables it to account for the observed average risk-taking and alternations. Including inertia, however, does not help the model to account for the trends in risk-taking and alternations over trials compared to the IBL model without the inertia mechanism. We generalize the two IBL models, with and without inertia, to the competition set by using the parameters determined in the estimation set. The generalization process demonstrates both the advantages and disadvantages of including inertia in an IBL model.

A Cognitive Model of Dynamic Cooperation With Varied Interdependency Information

We analyze the dynamics of repeated interaction of two players in the Prisoners Dilemma (PD) under various levels of interdependency information and propose an instance-based learning cognitive model (IBL-PD) to explain how cooperation emerges over time. Six hypotheses are tested regarding how a player accounts for an opponents outcomes: the selfish hypothesis suggests ignoring information about the opponent and utilizing only the players own outcomes; the extreme fairness hypothesis weighs the players own and the opponents outcomes equally; the moderate fairness hypothesis weighs the opponents outcomes less than the players own outcomes to various extents; the linear increasing hypothesis increasingly weighs the opponents outcomes at a constant rate with repeated interactions; the hyperbolic discounting hypothesis increasingly and nonlinearly weighs the opponents outcomes over time; and the dynamic expectations hypothesis dynamically adjusts the weight a player gives to the opponents outcomes, according to the gap between the expected and the actual outcomes in each interaction. When players lack explicit feedback about their opponents choices and outcomes, results are consistent with the selfish hypothesis; however, when this information is made explicit, the best predictions result from the dynamic expectations hypothesis.

Making Instance-based Learning Theory usable and understandable: The Instance-based Learning Tool

This paper focuses on the creation and presentation of a user-friendly experience for developing computational models of human behavior. Although computational models of human behavior have enjoyed a rich history in cognitive psychology, they have lacked widespread impact, partly due to the technical knowledge and programming required in addition to the complexities of the modeling process. We describe a modeling tool called IBLTool that is a computational implementation of the Instance-based Learning Theory (IBLT). IBLT is a theory that represents how decisions are made from experience in dynamic tasks. The IBLTool makes IBLT usable and understandable to a wider community of cognitive and behavioral scientists. The tool uses graphical user interfaces that take a modeler step-by-step through several IBLT processes and help the modeler derive predictions of human behavior in a particular task. A task would connect and interact with the IBLTool and store the decision-making data while the tool collects statistical data from the execution of a model for the task. We explain the functioning of the IBLTool and demonstrate a concrete example of the design and execution of a model for the Iowa Gambling task. The example is intended to provide a concrete demonstration of the capabilities of the IBLTool.

Instance-based Learning Models of Training

The IBL theory (IBLT) was developed to demonstrate the cognitive processes and mechanisms involved in dynamic decision making. IBLT has been implemented in several ACT-R cognitive models that showcase a variety of training effects and generate predictions resulting from several training manipulations. Particular implementations of IBLT-based models of training may depend on the task representation, the cues that are important for that task, and the particular decision actions involved, but the cognitive processes followed are generic. Here, we summarize different cognitive models for a diverse set of tasks relying on the IBLT. We also present an initial implementation of an IBL modeling tool that will help researchers develop their own IBL models without a need to program in ACT-R.

Modeling a robotics operator manager in a tactical battlefield

In a tactical battlefield, ground- and air- based robotic assets can be useful, but there is currently no one role to coordinate the use of multiple robots in the U.S. Army. The current work outlines and models the duties of a robotics operator manager (ROM), which could serve this role, using a theory of dynamic decision-making, instance-based learning (IBL). A model of the ROM is created that is based upon the IBL theory and that is intended to be a model of ROMs cognition. The IBL model first consults a Tactical Ground Reporting Network (TiGRNet) to recognize potential threat locations in a simulated battlefield. Then, the model decides whether a potential threat location warrants more investigation and if so weighs a number of factors to determine the type of robotic surveillance required (i.e., ground- or air- based). These factors include: the presence of certain attributes in a threat location, the utility of investigating a threat determining priority of investigation, the altitude of the terrain where the threat is located, whether there is time pressure, and the number of threat locations to investigate. The execution of the IBL model of the ROM generates predictions of the decisions made by a ROM under different factors. Future work in this area will focus on collecting human data to validate and adjust the predictions made by the IBL model of the ROM.