Matthew Molineaux

Learning to win: case-based plan selection in a real-time strategy game

While several researchers have applied case-based reasoning techniques to games, only Ponsen and Spronck (2004) have addressed the challenging problem of learning to win real-time games. Focusing on Wargus, they report good results for a genetic algorithm that searches in plan space, and for a weighting algorithm (dynamic scripting) that biases subplan retrieval. However, both approaches assume a static opponent, and were not designed to transfer their learned knowledge to opponents with substantially different strategies. We introduce a plan retrieval algorithm that, by using three key sources of domain knowledge, removes the assumption of a static opponent. Our experiments show that its implementation in the Case-based Tactician (CaT) significantly outperforms the best among a set of genetically evolved plans when tested against random Wargus opponents. CaT communicates with Wargus through TIELT, a testbed for integrating and evaluating decision systems with simulators. This is the first application of TIELT. We describe this application, our lessons learned, and our motivations for future work.

GOAL‐DRIVEN AUTONOMY FOR RESPONDING TO UNEXPECTED EVENTS IN STRATEGY SIMULATIONS

To operate autonomously in complex environments, an agent must monitor its environment and determine how to respond to new situations. To be considered intelligent, an agent should select actions in pursuit of its goals, and adapt accordingly when its goals need revision. However, most agents assume that their goals are given to them; they cannot recognize when their goals should change. Thus, they have difficulty coping with the complex environments of strategy simulations that are continuous, partially observable, dynamic, and open with respect to new objects. To increase intelligent agent autonomy, we are investigating a conceptual model for goal reasoning called Goal‐Driven Autonomy (GDA), which allows agents to generate and reason about their goals in response to environment changes. Our hypothesis is that GDA enables an agent to respond more effectively to unexpected events in complex environments. We instantiate the GDA model in ARTUE (Autonomous Response to Unexpected Events), a domain‐independent autonomous agent. We evaluate ARTUE on scenarios from two complex strategy simulations, and report on its comparative benefits and limitations. By employing goal reasoning, ARTUE outperforms an off‐line planner and a discrepancy‐based replanner on scenarios requiring reasoning about unobserved objects and facts and on scenarios presenting opportunities outside the scope of its current mission.

Case-Based Reasoning in Transfer Learning

Positive transfer learning (TL) occurs when, after gaining experience from learning how to solve a (source) task, the same learner can exploit this experience to improve performance and/or learning on a different (target) task. TL methods are typically complex, and case-based reasoning can support them in multiple ways. We introduce a method for recognizing intent in a source task, and then applying that knowledge to improve the performance of a case-based reinforcement learner in a target task. We report on its ability to significantly outperform baseline approaches for a control task in a simulated game of American football. We also compare our approach to an alternative approach where source and target task learning occur concurrently, and discuss the tradeoffs between them.

The Case for Case-Based Transfer Learning

Case-based reasoning (CBR) is a problem-solving process in which a new problem is solved by retrieving a similar situation and reusing its solution. Transfer learning occurs when, after gaining experience from learning how to solve source problems, the same learner exploits this experience to improve performance and/or learning on target problems. In transfer learning, the differences between the source and target problems characterize the transfer distance. CBR can support transfer learning methods in multiple ways. We illustrate how CBR and transfer learning interact and characterize three approaches for using CBR in transfer learning: (1) as a transfer learning method, (2) for problem learning, and (3) to transfer knowledge between sets of problems. We describe examples of these approaches from our own and related work and discuss applicable transfer distances for each. We close with conclusions and directions for future research applying CBR to transfer learning.

Minimally disruptive schedule repair for MCM missions

Mine countermeasures (MCM) missions entail planning and operations in very dynamic and uncertain operating environments, which pose considerable risk to personnel and equipment. Frequent schedule repairs are needed that consider the latest operating conditions to keep mission on target. Presently no decision support tools are available for the challenging task of MCM mission rescheduling. To address this capability gap, we have developed the CARPE system to assist operation planners. CARPE constantly monitors the operational environment for changes and recommends alternative repaired schedules in response. It includes a novel schedule repair algorithm called Case-Based Local Schedule Repair (CLOSR) that automatically repairs broken schedules while satisfying the requirement of minimal operational disruption. It uses a case-based approach to represent repair strategies and apply them to new situations. Evaluation of CLOSR on simulated MCM operations demonstrates the effectiveness of case-based strategy. Schedule repairs are generated rapidly, ensure the elimination of all mines, and achieve required levels of clearance.

Semantic Classification of Utterances in a Language-Driven Game

Artificial agents that interact with humans may find that understanding those humans’ plans and goals can improve their interactions. Ideally, humans would explicitly provide information about their plans, goals, and motivations to the agent. However, if the human is unable or unwilling to provide this information then the agent will need to infer it from observed behavior. We describe a goal reasoning agent architecture that allows an agent to classify natural language utterances, hypothesize about human’s actions, and recognize their plans and goals. In this paper we focus on one module of our architecture, the Natural Language Classifier, and demonstrate its use in a multiplayer tabletop social deception game, One Night Ultimate Werewolf. Our evaluation indicates that our system can obtain reasonable performance even when the utterances are unstructured, deceptive, or ambiguous.