Ethan Trewhitt

An Ensemble Architecture for Learning Complex Problem-Solving Techniques from Demonstration

We present a novel ensemble architecture for learning problem-solving techniques from a very small number of expert solutions and demonstrate its effectiveness in a complex real-world domain. The key feature of our “Generalized Integrated Learning Architecture” (GILA) is a set of heterogeneous independent learning and reasoning (ILR) components, coordinated by a central meta-reasoning executive (MRE). The ILRs are weakly coupled in the sense that all coordination during learning and performance happens through the MRE. Each ILR learns independently from a small number of expert demonstrations of a complex task. During performance, each ILR proposes partial solutions to subproblems posed by the MRE, which are then selected from and pieced together by the MRE to produce a complete solution. The heterogeneity of the learner-reasoners allows both learning and problem solving to be more effective because their abilities and biases are complementary and synergistic. We describe the application of this novel learning and problem solving architecture to the domain of airspace management, where multiple requests for the use of airspaces need to be deconflicted, reconciled, and managed automatically. Formal evaluations show that our system performs as well as or better than humans after learning from the same training data. Furthermore, GILA outperforms any individual ILR run in isolation, thus demonstrating the power of the ensemble architecture for learning and problem solving.

The effectiveness of intelligent tutoring on training in a video game

In this paper we evaluate the effectiveness of intelligent tutoring approaches on mastery and learning in a serious 3D immersive game called Heuristica. Heuristica teaches students to recognize and mitigate cognitive biases using a set of scenarios on a space station to perform tasks such as diagnosing and repairing problems or observing and evaluating game characters performing tasks. The student is evaluated on interactions in the 3D environment and on answers to questions provided by text or audio. We tested two types of tailoring: a) Student Model guided gameplay based on performance and b) Student Model guided gameplay with added worked-out examples (WOEs) whenever the student displays specific misconceptions or bugs in reasoning. We expected that customizing a players game experience based on his or her pre-test knowledge scores and in-game behavior would tailor the learning experience and improve the effectiveness of the training. Ninety-four participants played one of three versions of the game, and the experiment evaluated and compared the efficacy of the game using either a fixed-order version of the game (no tailoring) or either of the two tailoring approaches. Differences in the mastery scores captured during gameplay provided additional insight into these results. Implications for intelligent tutoring use in games are discussed.

Evaluating Counter-IED Strategies

Improvised explosive devices (IEDs) are one of the largest threats facing coalition forces in current military conflicts. The United States and other nations are greatly invested in mitigating these deadly devices. Past results have shown that completely technological counter-IED (cIED) efforts will be insufficient and, therefore, attention is focusing on augmenting the technological methods with neutralizing factors that contribute to human involvement in the IED perpetration process.To do so successfully requires an understanding of the behavioral aspects and influences of human involvement.This has led to an interest in socio-technical and systems-based models of terrorist activity. By integrating behavioral aspects of adversarial activities with computational methods, a greater understanding of these activities can be attained; simultaneously, potentially effective intervention points can be ascertained.This is often accomplished by modeling individuals, organizations, and societies via the creation of micro-, meso-, and macro-scale models to analyze and experiment with the impact of potential influences on population behavior. In addition to providing insight, model flexibility and model dynamics are required to assessmultiple interpretations of situations as they play out over time. Static models often cannot achieve this since disparate motivations and ideological factors evolve as a function of time. This article focuses on modeling the IED perpetration process, where knowledge was provided by subject matter experts (SMEs) from the United States and the United Kingdom, to ascertain behavioral aspects of cIED efforts. Defense & Security AnalysisVol. 27, No. 2, pp. 135–147, June 2011

Two Complementary Network Modeling and Simulation Approaches to Aid in Understanding Advanced Cyber Threats

This paper describes two complementary approaches to modeling and simulation (M&S) of sophisticated malware attacks for their use in understanding and preparing for potential threats. Modern malware operates at multiple scales, and successfully defending against these attacks requires the ability to understand the effects of decisions across this range. We present two types of M&S frameworks that differ in fidelity and scalability. The first is a low fidelity, scalable approach for representing and studying the spread of malware in a large network at a macro scale. The network is both modelled and simulated in ns-3, a discrete event simulation tool typically used for protocol exploration and traffic monitoring that supports the simulation of tens of thousands of nodes. The second type of simulation is a higher-fidelity, micro scale approach that includes nodes that closely emulate the behavior of actual computer systems and may include real hardware and software. Ns-3 allows outside networks to interact in real-time with ns-3. This enables the combination of the network simulation environment with real and virtual machines to allow detailed observation of the ways in which a hypothetical advanced persistent threat would play out in a small subnetwork. The interface between the ns-3 simulation, attack framework (e.g. Metasploit), and the real and virtual nodes is managed by a controller that also supplies configuration, business logic and results logging. We present use cases for both simulation types, showing how each approach can be used in the analysis of malware.

A systems-level understanding of adversarial behaviour

Modelling behaviour related to the perpetration of improvised explosive devices is extremely complex. Behavioural aspects range from those who create a plan to those who gather supplies for developing the devices to those who passively look the other way. Developing computational approaches to understanding such behaviour necessitates either a decomposition of behavioural activity into smaller, manageable behaviours or generalising larger, group behaviour where gross trends are observed. This may suffice for particular applications; however, additional consideration can be given to developing more comprehensive approaches. Specifically, for those seeking to understand terrorism, a number of social, cultural and behavioural perspectives are being developed by experts worldwide. These perspectives may complement each other or they may be in conflict, but they equally contribute to a broader understanding. Our research is developing computational methods to analyse and experiment with differing views and perspectives of potential influences on adversarial behaviour at this system-level.

Model docking using knowledge-level analysis

This paper presents an initial approach for exploring the docking of social models at the knowledge level. We have prototyped a simple blackboard environment allowing for model docking experimentation. There are research challenges in identifying which models are appropriate to dock and the concepts that they should exchange to build a richer multi-scale view of the world. Our early approach includes docking of societal system dynamics models with individual and organizational behaviors represented in agent-based models. Case-based models allow exploration of historical knowledge by other models. Our research presents initial efforts to attain opportunistic, asynchronous interactions among multi-scale models through investigation and experimentation of knowledge-level model docking. A docked system can supply a multi-scale modeling capability to support a users what-if analysis through combinations of case-based modeling, system dynamics approaches and agent-based models working together. An example is provided for the domain of terrorist recruiting.