Elizabeth Whitaker

An Ensemble Architecture for Learning Complex Problem-Solving Techniques from Demonstration

We present a novel ensemble architecture for learning problem-solving techniques from a very small number of expert solutions and demonstrate its effectiveness in a complex real-world domain. The key feature of our “Generalized Integrated Learning Architecture” (GILA) is a set of heterogeneous independent learning and reasoning (ILR) components, coordinated by a central meta-reasoning executive (MRE). The ILRs are weakly coupled in the sense that all coordination during learning and performance happens through the MRE. Each ILR learns independently from a small number of expert demonstrations of a complex task. During performance, each ILR proposes partial solutions to subproblems posed by the MRE, which are then selected from and pieced together by the MRE to produce a complete solution. The heterogeneity of the learner-reasoners allows both learning and problem solving to be more effective because their abilities and biases are complementary and synergistic. We describe the application of this novel learning and problem solving architecture to the domain of airspace management, where multiple requests for the use of airspaces need to be deconflicted, reconciled, and managed automatically. Formal evaluations show that our system performs as well as or better than humans after learning from the same training data. Furthermore, GILA outperforms any individual ILR run in isolation, thus demonstrating the power of the ensemble architecture for learning and problem solving.

Heuristica: Designing a serious game for improving decision making

This paper describes the design process and development of a 3D immersive serious game, Heuristica. The objective of this video game is to train players to improve their decision making by mitigating cognitive biases in an engaging and effective way. Heuristica is the result of three development and empirical evaluation cycles over 18 months. Several game features have been tested, and only those that improved learning while maintaining engagement have been retained in the latest version of the video game. These include reward, real time feedback, and game customization. Our development and playtesting process is summarized, and the implications for designing training are described.

The effect of camera perspective and session duration on training decision making in a serious video game

In this paper, we examine the effects of three video game variables: camera perspective (1st person versus 3rd person), session duration, and repeated play on training participants to mitigate three cognitive biases. We developed a 70 minute, 3D immersive video game for use as an experimentation test bed. One-hundred and sixty three participants either watched an instructional decision video or played one of the four versions of the game. Each participants learning was assessed by comparing his or her post-test scores and pre-test scores for knowledge of the biases and ability to mitigate them. Results indicated that repeated game play across two sessions produced the largest improvement in learning, and was more effective than the instructional decision video and single session game for mitigating biases. Surprisingly, session duration did not improve learning, and results were mixed for the third person perspective improved learning. Overall, the video game did improve participants ability to learn and to mitigate three cognitive biases. Implications for training using video game are discussed.

The effectiveness of intelligent tutoring on training in a video game

In this paper we evaluate the effectiveness of intelligent tutoring approaches on mastery and learning in a serious 3D immersive game called Heuristica. Heuristica teaches students to recognize and mitigate cognitive biases using a set of scenarios on a space station to perform tasks such as diagnosing and repairing problems or observing and evaluating game characters performing tasks. The student is evaluated on interactions in the 3D environment and on answers to questions provided by text or audio. We tested two types of tailoring: a) Student Model guided gameplay based on performance and b) Student Model guided gameplay with added worked-out examples (WOEs) whenever the student displays specific misconceptions or bugs in reasoning. We expected that customizing a players game experience based on his or her pre-test knowledge scores and in-game behavior would tailor the learning experience and improve the effectiveness of the training. Ninety-four participants played one of three versions of the game, and the experiment evaluated and compared the efficacy of the game using either a fixed-order version of the game (no tailoring) or either of the two tailoring approaches. Differences in the mastery scores captured during gameplay provided additional insight into these results. Implications for intelligent tutoring use in games are discussed.

Evaluating Counter-IED Strategies

Improvised explosive devices (IEDs) are one of the largest threats facing coalition forces in current military conflicts. The United States and other nations are greatly invested in mitigating these deadly devices. Past results have shown that completely technological counter-IED (cIED) efforts will be insufficient and, therefore, attention is focusing on augmenting the technological methods with neutralizing factors that contribute to human involvement in the IED perpetration process.To do so successfully requires an understanding of the behavioral aspects and influences of human involvement.This has led to an interest in socio-technical and systems-based models of terrorist activity. By integrating behavioral aspects of adversarial activities with computational methods, a greater understanding of these activities can be attained; simultaneously, potentially effective intervention points can be ascertained.This is often accomplished by modeling individuals, organizations, and societies via the creation of micro-, meso-, and macro-scale models to analyze and experiment with the impact of potential influences on population behavior. In addition to providing insight, model flexibility and model dynamics are required to assessmultiple interpretations of situations as they play out over time. Static models often cannot achieve this since disparate motivations and ideological factors evolve as a function of time. This article focuses on modeling the IED perpetration process, where knowledge was provided by subject matter experts (SMEs) from the United States and the United Kingdom, to ascertain behavioral aspects of cIED efforts. Defense & Security AnalysisVol. 27, No. 2, pp. 135–147, June 2011

Case-Based Reasoning for Knowledge Discovery

The Georgia Tech Research Institutes goal in the Novel Intelligence in Massive Data (NIMD) Program, is to investigate certain aspects of an intelligence analysts preferences and analytic strategies used in the process of discovering new knowledge. We are analyzing search strategies used by analysts in an attempt to understand their current task model. Based on this understanding, we will design and prototype a software tool that applies case-based reasoning in combination with other advanced reasoning techniques to help analysts perform knowledge discovery. The main objective of our work is the development, validation and incremental improvement of a set of knowledge discovery automation aids that significantly reduces the manual searches done by intelligence analysts and increases the quality and quantity of derived intelligence. We believe that these technical improvements will depend on our explicit understanding of the cognitive issues implied by the use of advanced reasoning techniques, integrated into a next generation NIMD prototype.

Two Complementary Network Modeling and Simulation Approaches to Aid in Understanding Advanced Cyber Threats

This paper describes two complementary approaches to modeling and simulation (M&S) of sophisticated malware attacks for their use in understanding and preparing for potential threats. Modern malware operates at multiple scales, and successfully defending against these attacks requires the ability to understand the effects of decisions across this range. We present two types of M&S frameworks that differ in fidelity and scalability. The first is a low fidelity, scalable approach for representing and studying the spread of malware in a large network at a macro scale. The network is both modelled and simulated in ns-3, a discrete event simulation tool typically used for protocol exploration and traffic monitoring that supports the simulation of tens of thousands of nodes. The second type of simulation is a higher-fidelity, micro scale approach that includes nodes that closely emulate the behavior of actual computer systems and may include real hardware and software. Ns-3 allows outside networks to interact in real-time with ns-3. This enables the combination of the network simulation environment with real and virtual machines to allow detailed observation of the ways in which a hypothetical advanced persistent threat would play out in a small subnetwork. The interface between the ns-3 simulation, attack framework (e.g. Metasploit), and the real and virtual nodes is managed by a controller that also supplies configuration, business logic and results logging. We present use cases for both simulation types, showing how each approach can be used in the analysis of malware.

Intelligent Agent Representations of Malware: Analysis to Prepare for Future Cyber Threats

There have been several recent examples of cyber-attacks that contain multiple components and have more advanced approaches than those that cyber-defense teams have become accustomed to. Some of these attacks have characteristics of intelligence and can be modelled as a set of collaborating software components such as those used in intelligent agents. In this paper, we discuss a set of parameters useful for analyzing and characterizing potential advanced cyber threats and for helping cybersecurity experts prepare to defend against them. A set of intelligent agents can be designed to collaborate in order to solve a complex problem, each agent having its own set of knowledge and expertise and being able to respond to requests from other agents for help in solving the problem. An intelligent agent can contain or have access to knowledge about context (e.g. patterns of network traffic) or problem-solving and can use any of the artificial intelligence reasoning techniques that are available to larger, more comprehensive software modules. Some agents are mobile, that is they can move across a network to operate on multiple network nodes. These intelligent agent paradigms can represent advanced threats. For example, intelligent agents as individual intelligent software entities, as a collaborating set, or as a swarm with emergent intelligence could be used to model threats which manifest cyber tactics, techniques and procedures (TTPs). This paper includes an analysis of the design parameters of intelligent agent architectures and the implications of these parameter choices for agent behaviors that can be used for analyzing and testing systems for the purpose of learning to secure them against sophisticated cyber-attacks. In order to motivate and support this analysis we provide a scenario use case which envisions the use of advanced intelligent agent teams for analysis of possible threats and for cybersecurity testing.

Agent-Based Simulation

Agent-based modeling and simulation (ABMS) is an approach for exploring the behaviors and interactions of individuals or organizations in particular situations or environments. Individuals can be any entity that behaves somewhat autono-mously and interacts with other agents, e.g. humans, animals, bacteria, blood cells or molecules. Organizations can be any collection of entities whose behavior can be characterized as the behavior of a group. Examples might be sports teams, project teams, political organizations, terrorist organizations, legislatures, military organizations, or towns. An ABMS may be used to model a system and answer questions about that system, or predict the ways that the system will respond to external influences. The system being modeled may be an existing system, which is being analyzed to understand the behavior in response to specific changes in the environment, or a new system being designed or built. This chapter will give an overview of ABMS, discuss agent characteristics and frameworks, and use an example to describe how to create an ABMS.

A systems-level understanding of adversarial behaviour

Modelling behaviour related to the perpetration of improvised explosive devices is extremely complex. Behavioural aspects range from those who create a plan to those who gather supplies for developing the devices to those who passively look the other way. Developing computational approaches to understanding such behaviour necessitates either a decomposition of behavioural activity into smaller, manageable behaviours or generalising larger, group behaviour where gross trends are observed. This may suffice for particular applications; however, additional consideration can be given to developing more comprehensive approaches. Specifically, for those seeking to understand terrorism, a number of social, cultural and behavioural perspectives are being developed by experts worldwide. These perspectives may complement each other or they may be in conflict, but they equally contribute to a broader understanding. Our research is developing computational methods to analyse and experiment with differing views and perspectives of potential influences on adversarial behaviour at this system-level.