Stewart Massie

Feature Selection and Generalisation for Retrieval of Textual Cases

Textual CBR systems solve problems by reusing experiences that are in textual form. Knowledge-rich comparison of textual cases remains an important challenge for these systems. However mapping text data into a structured case representation requires a significant knowledge engineering effort. In this paper we look at automated acquisition of the case indexing vocabulary as a two step process involving feature selection followed by feature generalisation. Boosted decision stumps are employed as a means to select features that are predictive and relatively orthogonal. Association rule induction is employed to capture feature co-occurrence patterns. Generalised features are constructed by applying these rules. Essentially, rules preserve implicit semantic relationships between features and applying them has the desired effect of bringing together cases that would have otherwise been overlooked during case retrieval. Experiments with four textual data sets show significant improvement in retrieval accuracy whenever generalised features are used. The results further suggest that boosted decision stumps with generalised features to be a promising combination.

When Similar Problems Don't Have Similar Solutions

The performance of a Case-Based Reasoning system relies on the integrity of its case base but in real life applications the available data used to construct the case base invariably contains erroneous, noisy cases. Automated removal of these noisy cases can improve system accuracy. In addition, error rates for nearest neighbour classifiers can often be reduced by removing cases to give smoother decision boundaries between classes. In this paper we argue that the optimallevel of boundary smoothing is domain dependent and, therefore, our approach to error reduction reacts to the characteristics of the domain to set an appropriate level of smoothing. We present a novel, yet transparent algorithm, Threshold Error Reduction, which identifies and removes noisy and boundary cases with the aid of a local complexity measure. Evaluation results confirm it to be superior to benchmark algorithms.

From Anomaly Reports to Cases

Creating case representations in unsupervised textual case-based reasoning applications is a challenging task because class knowledge is not available to aid selection of discriminatory features or to evaluate alternative system design configurations. Representation is considered as part of the development of a tool, called CAM , which supports an anomaly report processing task for the European Space Agency. Novel feature selection/extraction techniques are created which consider word co-occurrence patterns to calculate similarity between words. These are used together with existing techniques to create 5 different case representations. A new evaluation technique is introduced to compare these representations empirically, without the need for expensive, domain expert analysis. Alignment between the problem and solution space is measured at a local level and profiles of these local alignments used to evaluate the competenceof the system design.

Unsupervised feature selection for text data

Feature selection for unsupervised tasks is particularly challenging, especially when dealing with text data. The increase in online documents and email communication creates a need for tools that can operate without the supervision of the user. In this paper we look at novel feature selection techniques that address this need. A distributional similarity measure from information theory is applied to measure feature utility. This utility informs the search for both representative and diverse features in two complementary ways: Cluster divides the entire feature space, before then selecting one feature to represent each cluster; and Greedy increments the feature subset size by a greedily selected feature. In particular we found that Greedys local search is suited to learning smaller feature subset sizes while Cluster is able to improve the global quality of larger feature sets. Experiments with four email data sets show significant improvement in retrieval accuracy with nearest neighbour based search methods compared to an existing frequency-based method. Importantly both Greedy and Cluster make significant progress towards the upper bound performance set by a standard supervised feature selection method.

Complexity profiling for informed case-base editing

The contents of the case knowledge container is critical to the performance of case-based classification systems. However the knowledge engineer is given little support in the selection of suitable techniques to maintain and monitor the case-base. In this paper we present a novel technique that provides an insight into the structure of a case-base by means of a complexity profile that can assist maintenance decision-making and provide a benchmark to assess future changes to the case-base. We also introduce a complexity-guided redundancy reduction algorithm which uses a local complexity measure to actively retain cases close to boundaries. The algorithm offers control over the balance between maintaining competence and reducing case-base size. The ability of the algorithm to maintain accuracy in a compacted case-base is demonstrated on seven public domain classification datasets.

Learning pseudo-tags to augment sparse tagging in hybrid music recommender systems

Online recommender systems are an important tool that people use to find new music. To generate recommendations, many systems rely on tag representations of music. Such systems, however, suffer from tag sparsity, whereby tracks lack a strong tag representation. Current state-of-the-art techniques that reduce this sparsity problem create hybrid systems using multiple representations, for example both content and tags. In this paper we present a novel hybrid representation that augments sparse tag representations without introducing content directly. Our hybrid representation integrates pseudo-tags learned from content into the tag representation of a track, and a dynamic weighting scheme limits the number of pseudo-tags that are allowed to contribute. Experiments demonstrate that this method allows tags to remain dominant when they provide a strong representation, and pseudo-tags to take over when tags are sparse. We show that our approach significantly improves recommendation quality not only for queries with a sparse tag representation but also those that are well-tagged. Our hybrid approach has potential to be extended to other music representations that are used for recommendation but suffer from data sparsity, such as user profiles.

Index driven selective sampling for CBR

In real environments it is often difficult to obtain a collection of cases for the case base that would cover all the problem solving situations. Although it is often somewhat easier to generate potential problem cases that cover the domain tasks, acquiring the solutions for the problems captured by the cases may demand valuable time of a busy expert. This paper investigates how a Case-Based Reasoning system can be empowered to actively select a small number of useful cases from a pool of problem cases, for which the expert can then provide the solution. Past cases that are complete, containing both the problem and solution, together with partial cases containing just the problem, are clustered by exploiting a decision tree index built over the complete cases. We introduce a Cluster Utility Score ClUS and Case Utility Score CaUS, which then guide case selection from these clusters. Experimental results for six public domain datasets show that selective sampling techniques employing ClUS and CaUS are able to select cases that significantly improve the accuracy of the case base. There is further evidence to show that the influence of complete and partial cases utilised by these scores needs also to consider the number of partitions created by the case base index.

Lexicon Generation for Emotion Detection from Text

General-purpose emotion lexicons (GPELs) that associate words with emotion categories remain a valuable resource for emotion detection. However, the static and formal nature of their vocabularies make them an inadequate resource for detecting emotions in domains that are inherently dynamic in nature. This calls for lexicons that are not only adaptive to the lexical variations in a domain but which also provide finer-grained quantitative estimates to accurately capture word-emotion associations. In this article, the authors demonstrate how to harness labeled emotion text (such as blogs and news headlines) and weakly labeled emotion text (such as tweets) to learn a word-emotion association lexicon by jointly modeling emotionality and neutrality of words using a generative unigram mixture model (UMM). Empirical evaluation confirms that UMM generated emotion language models (topics) have significantly lower perplexity compared to those from state-of-the-art generative models like supervised Latent Dirichlet Allocation (sLDA). Further emotion detection tasks involving word-emotion classification and document-emotion ranking confirm that the UMM lexicon significantly out performs GPELs and also state-of-the-art domain specific lexicons.

Generating a Word-Emotion Lexicon from #Emotional Tweets

Research in emotion analysis of text suggest that emotion lexicon based features are superior to corpus based n-gram features. However the static nature of the general purpose emotion lexicons make them less suited to social media analysis, where the need to adopt to changes in vocabulary usage and context is crucial. In this paper we propose a set of methods to extract a word-emotion lexicon automatically from an emotion labelled corpus of tweets. Our results confirm that the features derived from these lexicons outperform the standard Bag-of-words features when applied to an emotion classification task. Furthermore, a comparative analysis with both manually crafted lexicons and a state-of-the-art lexicon generated using Point-Wise Mutual Information, show that the lexicons generated from the proposed methods lead to significantly better classification performance.

Using a Markov network as a surrogate fitness function in a genetic algorithm

Surrogate models of fitness have been presented as a way of reducing the number of fitness evaluations required by an evolutionary algorithm. This is of particular interest with expensive fitness functions where the cost of building the model is outweighed by the saving of using fewer function evaluations. In this paper we show how a Markov network model can be used as a surrogate fitness function in a genetic algorithm. We demonstrate this applied to a number of well-known benchmark functions and although the results are good in terms of function evaluations the model-building overhead requires a substantially more expensive fitness function to be worthwhile. We move on to describe a fitness function for feature selection in Case-Based Reasoning, which is considerably more expensive than the other benchmark functions we used. We show that for this problem using the surrogate offers a significant decrease in total run time compared to a GA using the true fitness function.