Héctor Muñoz-Avila

Conversational Case-Based Reasoning

Conversational case-based reasoning (CCBR) was the first widespread commercially successful form of case-based reasoning. Historically, commercial CCBR tools conducted constrained human-user dialogues and targeted customer support tasks. Due to their simple implementation of CBR technology, these tools were almost ignored by the research community (until recently), even though their use introduced many interesting applied research issues. We detail our progress on addressing three of these issues: simplifying case authoring, dialogue inferencing, and interactive planning. We describe evaluations of our approaches on these issues in the context of NaCoDAE and HICAP, our CCBR tools. In summary, we highlight important CCBR problems, evaluate approaches for solving them, and suggest alternatives to be considered for future research.

Applications of SHOP and SHOP2

We design the simple hierarchical ordered planner (SHOP) and its successor, SHOP2, with two goals in mind: to investigate research issues in automated planning and to provide some simple, practical planning tools. SHOP and SHOP2 are based on a planning formalism called hierarchical task network planning. SHOP and SHOP2 use a search-control strategy called ordered task decomposition, which breaks tasks into subtasks and generates the plans actions in the same order that the plan executor executes them. So, throughout the planning process, the planner can tell what the state of the world at each step of the plan.

Recognizing the Enemy: Combining Reinforcement Learning with Strategy Selection Using Case-Based Reasoning

This paper presents CBRetaliate , an agent that combines Case-Based Reasoning (CBR) and Reinforcement Learning (RL) algorithms. Unlike most previous work where RL is used to improve accuracy in the action selection process, CBRetaliate uses CBR to allow RL to respond more quickly to changing conditions. CBRetaliate combines two key features: it uses a time window to compute similarity and stores and reuses complete Q-tables for continuous problem solving. We demonstrate CBRetaliate on a team-based first-person shooter game, where our combined CBR+RL approach adapts quicker to changing tactics by an opponent than standalone RL.

Case-based reasoning integrations

This article presents an overview and survey of current work in case-based reasoning (CBR) integrations. There has been a recent upsurge in the integration of CBR with other reasoning modalities and computing paradigms, especially rule-based reasoning (RBR) and constraint-satisfaction problem (CSP) solving. CBR integrations with model-based reasoning (MBR), genetic algorithms, and information retrieval are also discussed. This article characterizes the types of multimodal reasoning integrations where CBR can play a role, identifies the types of roles that CBR components can fulfill, and provides examples of integrated CBR systems. Past progress, current trends, and issues for future research are discussed.

Case-based planning

We briefly examine case-based planning starting with the seminal work of Hammond. Derivational analogy represents an important shift of technical emphasis that helped mature the techniques. The choice of abstraction level is equally important. We conclude by discussing theoretical underpinnings and by providing some pointers to current directions.

Using Guidelines to Constrain Interactive Case-Based HTN Planning

This paper describes HICAP, a general-purpose, interactive case-based plan authoring architecture that can be applied to decision support tasks to yield a hierarchical course of action. It integrates a hierarchical task editor with a conversational case-based planner. HICAP maintains both a task hierarchy representing guidelines that constrain the final plan and the hierarchical social organization responsible for these tasks. It also supports bookkeeping, which is crucial for real-world large-scale planning tasks. By selecting tasks corresponding to the hierarchys leaf nodes, users can activate the conversational case-based planner to interactively refine guideline tasks into a concrete plan. Thus, HICAP can be used to generate context sensitive plans and should be useful for assisting with planning complex tasks such as noncombatant evacuation operations. We describe an experiment with a highly detailed military simulator to investigate this claim. The results show that plans generated by HICAP were superior to those generated by alternative approaches.

IMPACTing SHOP: Putting an AI Planner Into a Multi-Agent Environment

In this paper we describe a formalism for integrating the SHOP HTN planning system with the IMPACT multi-agent environment. We define the A-SHOP algorithm, an agentized adaptation of the SHOP planning algorithm that takes advantage of IMPACTs capabilities for interacting with external agents, performing mixed symbolic/numeric computations, and making queries to distributed, heterogeneous information sources (such as arbitrary legacy and/or specialized data structures or external databases). We show that A-SHOP is both sound and complete if certain conditions are met.

Knowledge acquisition for adaptive game AI

Game artificial intelligence (AI) controls the decision-making process of computer-controlled opponents in computer games. Adaptive game AI (i.e., game AI that can automatically adapt the behaviour of the computer players to changes in the environment) can increase the entertainment value of computer games. Successful adaptive game AI is invariably based on the games domain knowledge. We show that an offline evolutionary algorithm can learn important domain knowledge in the form of game tactics (i.e., a sequence of game actions) for dynamic scripting, an offline algorithm inspired by reinforcement learning approaches that we use to create adaptive game AI. We compare the performance of dynamic scripting under three conditions for defeating non-adaptive opponents in a real-time strategy game. In the first condition, we manually encode its tactics. In the second condition, we manually translate the tactics learned by the evolutionary algorithm, and use them for dynamic scripting. In the third condition, this translation is automated. We found that dynamic scripting performs best under the third condition, and both of the latter conditions outperform manual tactic encoding. We discuss the implications of these results, and the performance of dynamic scripting for adaptive game AI from the perspective of machine learning research and commercial game development.

Learning approximate preconditions for methods in hierarchical plans

A significant challenge in developing planning systems for practical applications is the difficulty of acquiring the domain knowledge needed by such systems. One method for acquiring this knowledge is to learn it from plan traces, but this method typically requires a huge number of plan traces to converge. In this paper, we show that the problem with slow convergence can be circumvented by having the learner generate solution plans even before the planning domain is completely learned. Our empirical results show that these improvements reduce the size of the training set that is needed to find correct answers to a large percentage of planning problems in the test set.

LEARNING PRECONDITIONS FOR PLANNING FROM PLAN TRACES AND HTN STRUCTURE

A great challenge in developing planning systems for practical applications is the difficulty of acquiring the domain information needed to guide such systems. This paper describes a way to learn some of that knowledge. More specifically, the following points are discussed. (1) We introduce a theoretical basis for formally defining algorithms that learn preconditions for Hierarchical Task Network (HTN) methods. (2) We describe Candidate Elimination Method Learner (CaMeL), a supervised, eager, and incremental learning process for preconditions of HTN methods. We state and prove theorems about CaMeLs soundness, completeness, and convergence properties. (3) We present empirical results about CaMeLs convergence under various conditions. Among other things, CaMeL converges the fastest on the preconditions of the HTN methods that are needed the most often. Thus CaMeLs output can be useful even before it has fully converged.