Vahid Jalali

Extending Case Adaptation with Automatically-Generated Ensembles of Adaptation Rules

Case-based regression often relies on simple case adaptation methods. This paper investigates new approaches to enriching the adaptation capabilities of case-based regression systems, based on the use of ensembles of adaptation rules generated from the case base. The paper explores both local and global methods for generating adaptation rules from the case base, and presents methods for ranking the generated rules and combining the resulting ensemble of adaptation rules to generate new solutions. It tests these methods in five standard domains, evaluating their performance compared to four baseline methods, standard k-NN, linear regression, locally weighted linear regression, and an ensemble of k-NN predictors with different feature subsets. The results demonstrate that the proposed method generally outperforms the baselines and that the accuracy of adaptation based on locally-generated rules is highly competitive with that of global rule-generation methods with much greater computational cost.

A Context-Aware Approach to Selecting Adaptations for Case-Based Reasoning

Case-based reasoning solves new problems by retrieving cases of similar previously-solved problems and adapting their solutions to fit new circumstances. The case adaptation step is often done by applying context-independent adaptation rules. A substantial body of research has studied generating these rules automatically from comparisons of prior pairs of cases. This paper presents a method for increasing the context-awareness of case adaptation using these rules, by exploiting contextual information about the prior problems from which the rules were generated to predict their applicability to the context of the new problem, in order to select the most relevant rules. The paper tests the approach for the task of case-based prediction of numerical values case-based regression. It evaluates performance on standard machine learning data sets to assess the methods performance benefits, and also tests it on synthetic domains to study how performance is affected by different problem space characteristics. The results show the proposed method for context-awareness brings significant gains in solution accuracy.

On Deriving Adaptation Rule Confidence from the Rule Generation Process

Previous case-based reasoning research makes a compelling case for the importance of CBR systems determining the system’s confidence in its conclusions, and has developed useful analyses of how characteristics of individual cases and the case base as a whole influence confidence. This paper argues that in systems which perform case adaptation, an important additional indicator for solution confidence is confidence in the adaptations performed. Assessing confidence of adaptation rules may be particularly important when knowledge-light methods are applied to generate adaptations automatically from the case base, giving the opportunity to improve performance by astute rule selection. The paper proposes a new method for calculating rule confidence for automatically-generated adaptation rules for regression tasks, when the rules are generated by the common “difference heuristic” method of comparing pairs of cases in a case base, and a method for confidence-influenced selection of cases to adapt. The method is evaluated in four domains, showing performance gains over baseline methods and case based regression without using confidence knowledge.

On Retention of Adaptation Rules

The difficulty of acquiring case adaptation knowledge is a classic problem for case-based reasoning (CBR). One method for addressing this problem is to use the cases in the case base as data from which to learn adaptation rules. For numeric prediction tasks, adaptation rules have been successfully learned from the case base by using the case difference heuristic, which generates rules based on comparisons of pairs of cases. However, because the case difference heuristic could potentially generate a rule for each pair of cases in the case base, controlling growth of adaptation rules is potentially an even more acute problem than controlling case base growth. This raises the question of how to select adaptation rules to retain. The ability to generate adaptation rules from cases also raises questions about the relative benefit of learning cases, learning the adaptation rules generated from them, or learning both. This paper proposes and evaluates a new adaptation rule retention approach and presents a case study assessing the relative benefits of learning cases versus learning adaptation rules derived from the cases, at different points in the growth of the case base.

Enhancing case-based regression with automatically-generated ensembles of adaptations

Instance-based methods have been successfully applied to numerical prediction (regression) tasks in many domains. Such methods often rely on a simple combination function to generate a prediction from past instances. Case-based reasoning for regression adds a richer case adaptation step to adjust prior solutions to fit new problems. This article presents a new approach to case adaptation for case-based regression systems, based on applying an ensemble of case adaptation rules generated automatically from pairs of cases in the case base, using the case difference heuristic. It evaluates the method’s performance, considering in particular the effects of using local versus global case information to generate adaptation rules from the case base. Experimental results support that the proposed method generally outperforms baselines and that the accuracy of adaptation based on locally-generated rules is highly competitive with that of global rule-generation methods considering many more cases.

CBR Meets Big Data: A Case Study of Large-Scale Adaptation Rule Generation

Adaptation knowledge generation is a difficult problem for CBR. In previous work we developed ensembles of adaptation for regression (EAR), a family of methods for generating and applying ensembles of adaptation rules for case-based regression. EAR has been shown to provide good performance, but at the cost of high computational complexity. When efficiency problems result from case base growth, a common CBR approach is to focus on case base maintenance, to compress the case base. This paper presents a case study of an alternative approach, harnessing big data methods, specifically MapReduce and locality sensitive hashing (LSH), to make the EAR approach feasible for large case bases without compression. Experimental results show that the new method, BEAR, substantially increases accuracy compared to a baseline big data k-NN method using LSH. BEAR’s accuracy is comparable to that of traditional k-NN without using LSH, while its processing time remains reasonable for a case base of millions of cases. We suggest that increased use of big data methods in CBR has the potential for a departure from compression-based case-base maintenance methods, with their concomitant solution quality penalty, to enable the benefits of full case bases at much larger scales.

Ensemble of Adaptations for Classification: Learning Adaptation Rules for Categorical Features

Acquiring knowledge for case adaptation is a classic challenge for case-based reasoning (CBR). To provide CBR systems with adaptation knowledge, machine learning methods have been developed for automatically generating adaptation rules. An influential approach uses the case difference heuristic (CDH) to generate rules by comparing pairs of cases in the case base. The CDH method has been studied for case-based prediction of numeric values (regression) from inputs with primarily numeric features, and has proven effective in that context. However, previous work has not attempted to apply the CDH method to classification tasks, to generate rules for adapting categorical solutions. This paper introduces an approach to applying the CDH to cases with categorical features and target values, based on the generalized case value difference heuristic (GCVDH). It also proposes a classification method using ensembles of GCVDH-generated rules, ensemble of adaptations for classification (EAC), an extension to our previous work on ensembles of adaptations for regression (EAR). It reports on an evaluation comparing the accuracy of EAC to three baseline methods on four standard domains, as well as comparing EAC to an ablation relying on single adaptation rules, and assesses the effect of training/test size on accuracy. Results are encouraging for the effectiveness of the GCVDH approach and for the value of applying ensembles of learned adaptation rules for classification.

A Study of RDB-based RDF data management techniques

RDF has gained great interest in both academia and industry as an important language to describe graph data. Several approaches have been proposed for storing and querying RDF data efficiently; each works best under certain circumstances, e.g. certain types of data and/or queries. However, there was lack of a thorough understanding of exactly what these circumstances are, as different data-sets and query sets are used in the empirical evaluations in the literature to highlight their proposed techniques. In this work, we capture the characteristics of data and queries that are critical to the RDF storage and query evaluation efficiency and provide a thorough analysis of the existing storage, indexing and query evaluation techniques based on these characteristics. We believe that our study not only can be used in evaluating both existing and emerging RDF data management techniques, but also lays the foundations for designing RDF benchmarks for more in-depth performance analysis of RDF data management systems.

Learning and Applying Case Adaptation Rules for Classification: An Ensemble Approach

The ability of case-based reasoning systems to solve novel problems depends on their capability to adapt past solutions to new circumstances. However, acquiring the knowledge required for case adaptation is a classic challenge for CBR. This motivates the use of machine learning to generate adaptation knowledge. Much adaptation learning research has studied the case difference heuristic (CDH) approach, which generates adaptation rules from pairs of cases in the case base by ascribing observed differences in case solutions to the differences in the problems they solve, to generate rules for adapting similar problem differences. Extensive research has successfully applied the CDH approach to adaptation rule learning for case-based regression (numerical prediction) tasks. However, classification tasks have been outside of its scope. The work presented in this paper addresses that gap by extending CDH-based learning of adaptation rules to apply to cases with categorical features and solutions. It presents the generalized case value heuristic to assess case and solution differences and applies it in an ensemble-based casebased classification method, ensembles of adaptations for classification (EAC), built on the authors’ previous work on ensembles of adaptations for regression (EAR). Experimental results support the effectiveness of EAC.

Harnessing Hundreds of Millions of Cases: Case-Based Prediction at Industrial Scale

Building predictive models is central to many big data applications. However, model building is computationally costly at scale. An appealing alternative is bypassing model building by applying case-based prediction to reason directly from data. However, to our knowledge case-based prediction still has not been applied at true industrial scale. In previous work we introduced a knowledge-light/data intensive approach to case-based prediction, using ensembles of automatically-generated adaptations. We developed foundational scaleup methods, using Locality Sensitive Hashing (LSH) for fast approximate nearest neighbor retrieval of both cases and adaptation rules, and tested them for millions of cases. This paper presents research on extending these methods to address the practical challenges raised by case bases of hundreds of millions of cases for a real world industrial e-commerce application. Handling this application required addressing how to keep LSH practical for skewed data; the resulting efficiency gains in turn enabled applying an adaptation generation strategy that previously was computationally infeasible. Experimental results show that our CBR approach achieves accuracy comparable to or better than state of the art machine learning methods commonly applied, while avoiding their model-building cost. This supports the opportunity to harness CBR for industrial scale prediction.