David B. Leake

Retrieval, reuse, revision and retention in case-based reasoning

Case-based reasoning (CBR) is an approach to problem solving that emphasizes the role of prior experience during future problem solving (i.e., new problems are solved by reusing and if necessary adapting the solutions to similar problems that were solved in the past). It has enjoyed considerable success in a wide variety of problem solving tasks and domains. Following a brief overview of the traditional problem-solving cycle in CBR, we examine the cognitive science foundations of CBR and its relationship to analogical reasoning. We then review a representative selection of CBR research in the past few decades on aspects of retrieval, reuse, revision and retention.

Creativity and learning in a case-based explainer

Abstract Explanation-based learning (EBL) is a very powerful method for category formation. Since EBL algorithms depend on having good explanations, it is crucial to have effective ways to build explanations, especially in complex real-world situations where complete causal information is not available. When people encounter new situations, they often explain them by remembering old explanations, and adapting them to fit. We believe that this case-based approach to explanation holds promise for use in AI systems, both for routine explanation and to creatively explain situations quite unlike what the system has encountered before. Building new explanations from old ones relies on having explanations available in memory. We describe explanation patterns (XPs), knowledge structures that package the reasoning underlying explanations. Using the SWALE system as a base, we discuss the retrieval and modification process, and the criteria used when deciding which explanation to accept. We also discuss issues in learning XPs: what generalization strategies are appropriate for real-world explanations, and which indexing strategies are appropriate for XPs. SWALEs explanations allow it to understand nonstandard stories, and the XPs it learns increase its efficiency in dealing with similar anomalies in the future.

Categorizing Case-Base Maintenance: Dimensions and Directions

Experience with the growing number of large-scale CBR systems has led to increasing recognition of the importance of case-base maintenance. Multiple researchers have addressed pieces of the case-base maintenance problem, considering such issues as maintaining consistency and controlling case-base growth. However, despite the existence of these cases of case-base maintenance, there is no general framework of dimensions for describing case-base maintenance systems. Such a framework would be useful both to understand the state of the art in case-base maintenance and to suggest new avenues of exploration by identifying points along the dimensions that have not yet been studied. This paper presents a first attempt at identifying the dimensions of case-base maintenance. It shows that characterizations along such dimensions can suggest avenues for future case-base maintenance research and presents initial steps exploring one of those avenues: identifying patterns of problems that require generalized revisions and addressing them with lazy updating.

Learning to Improve Case Adaption by Introspective Reasoning and CBR

In current CBR systems, case adaptation is usually performed by rule-based methods that use task-specific rules hand-coded by the system developer. The ability to define those rules depends on knowledge of the task and domain that may not be available a priori, presenting a serious impediment to endowing CBR systems with the needed adaptation knowledge. This paper describes ongoing research on a method to address this problem by acquiring adaptation knowledge from experience. The method uses reasoning from scratch, based on introspective reasoning about the requirements for successful adaptation, to build up a library of adaptation cases that are stored for future reuse. We describe the tenets of the approach and the types of knowledge it requires. We sketch initial computer implementation, lessons learned, and open questions for further study.

Maintaining Case-Based Reasoners: Dimensions and Directions

Experience with the growing number of large‐scale and long‐term case‐based reasoning (CBR) applications has led to increasing recognition of the importance of maintaining existing CBR systems. Recent research has focused on case‐base maintenance (CBM), addressing such issues as maintaining consistency, preserving competence, and controlling case‐base growth. A set of dimensions for case‐base maintenance, proposed by Leake and Wilson, provides a framework for understanding and expanding CBM research. However, it also has been recognized that other knowledge containers can be equally important maintenance targets. Multiple researchers have addressed pieces of this more general maintenance problem, considering such issues as how to refine similarity criteria and adaptation knowledge. As with case‐base maintenance, a framework of dimensions for characterizing more general maintenance activity, within and across knowledge containers, is desirable to unify and understand the state of the art, as well as to suggest new avenues of exploration by identifying points along the dimensions that have not yet been studied. This article presents such a framework by (1) refining and updating the earlier framework of dimensions for case‐base maintenance, (2) applying the refined dimensions to the entire range of knowledge containers, and (3) extending the theory to include coordinated cross‐container maintenance. The result is a framework for understanding the general problem of case‐based reasoner maintenance (CBRM). Taking the new framework as a starting point, the article explores key issues for future CBRM research.

Remembering Why to Remember: Performance-Guided Case-Base Maintenance

An important focus of recent CBR research is on how to develop strategies for achieving compact, competent case-bases, as a way to improve the performance of CBR systems. However, compactness and competence are not always good predictors of performance, especially when problem distributions are non-uniform. Consequently, this paper argues for developing methods that tie case-base maintenance more directly to performance concerns. The paper begins by examining the relationship between competence and performance, discussing the goals and constraints that should guide addition and deletion of cases. It next illustrates the importance of augmenting competence-based criteria with quantitative performance-based considerations, and proposes a strategy for closely reflecting adaptation performance effects when compressing a case-base. It then presents empirical studies examining the performance tradeoffs of current methods and the benefits of applying fine-grained performance-based criteria to case-base compression, showing that performance-based methods may be especially important for task domains with non-uniform problem distributions.

When Experience Is Wrong: Examining CBR for Changing Tasks and Environments

Case-based problem-solving systems reason and learn from experiences, building up case libraries of problems and solutions to guide future reasoning. The expected benefits of this learning process depend on two types of regularity: (1) problem-solution regularity, the relationship between problem-to-problem and solution-to-solution similarity measures that assures that solutions to similar prior problems are a useful starting point for solving similar current problems, and (2) problem-distribution regularity, the relationship between old and new problems that assures that the case library will contain cases similar to the new problems it encounters. Unfortunately, these types of regularity are not assured. Even in contexts for which initial regularity is sufficient, problems may arise if a systems users, tasks, or external environment change over time. This paper defines criteria for assessing the two types of regularity, discusses how the definitions may be used to assess the need for case-base maintenance, and suggests maintenance approaches for responding to those needs. In particular, it discusses the role of analysis of performance over time in responding to environmental changes.

When Two Case Bases Are Better than One: Exploiting Multiple Case Bases

Much current CBR research focuses on how to compact, refine, and augment the contents of individual case bases, in order to distill needed information into a single concise and authoritative source. However, as deployed case-based reasoning systems become increasingly prevalent, opportunities will arise for supplementing local case bases on demand, by drawing on the case bases of other CBR systems addressing related tasks. Taking full advantage of these case bases will require multi-case-base reasoning: Reasoning not only about how to apply cases, but also about when and how to draw on particular case bases. This paper begins by considering tradeoffs of attempting to merge individual case bases into a single source, versus retaining them individually, and argues that retaining multiple case bases can benefit both performance and maintenance. However, achieving the benefits requires methods for case dispatching--deciding when to retrieve from external case bases, and which case bases to select--and for cross-case-base adaptation to revise suggested solutions from one context to apply in another. The paper presents initial experiments illustrating how these procedures may affect the benefits of using multiple case bases, and closes by delineating key research issues for multi-case-base reasoning.

WordSieve: A Method for Real-Time Context Extraction

In order to be useful, intelligent information retrieval agents must provide their users with context-relevant information. This paper presents WordSieve, an algorithm for automatically extracting information about the context in which documents are consulted during web browsing. Using information extracted from the stream of documents consulted by the user, WordSieve automatically builds context profiles which differentiate sets of documents that users tend to access in groups. These profiles are used in a research-aiding system to index documents consulted in the current context and pro-actively suggest them to users in similar future contexts. In initial experiments on the capability to match documents to the task contexts in which they were consulted, WordSieve indexing outperformed indexing based on Term Frequency/Inverse Document Frequency, a common document indexing approach for intelligent agents in information retrieval.

A Case-Based Framework for Interactive Capture and Reuse of Design Knowledge

Aerospace design is a complex task requiring access to large amounts of specialized information. Consequently, intelligent systems that support and amplify the abilities of human designers by capturing and presenting relevant information can profoundly affect the speed and reliability of design generation. This article describes research on supporting aerospace design by integrating a case-based design support framework with interactive tools for capturing expert design knowledge through “concept mapping.” In the integrated system, interactive concept mapping tools provide crucial functions for generating and examining design cases and navigating their hierarchical structure, while CBR techniques facilitate retrieval and aid interactive adaptation of designs. Our goal is both to provide a useful design aid and to develop general interactive techniques to facilitate case acquisition and adaptation. Experiments illuminate the performance of the systems context-sensitive retrieval during interactive case adaptation and the conditions under which it provides the most benefit.