Robert Lothian

Sprinkling: supervised latent semantic indexing

Latent Semantic Indexing (LSI) is an established dimensionality reduction technique for Information Retrieval applications. However, LSI generated dimensions are not optimal in a classification setting, since LSI fails to exploit class labels of training documents. We propose an approach that uses class information to influence LSI dimensions whereby class labels of training documents are endoded as new terms, which are appended to the documents. When LSI is carried out on the augmented term-document matrix, terms pertaining to the same class are pulled closer to each other. Evaluation over experimental data reveals significant improvement in classification accuracy over LSI. The results also compare favourably with naive Support Vector Machines.

Unsupervised feature selection for text data

Feature selection for unsupervised tasks is particularly challenging, especially when dealing with text data. The increase in online documents and email communication creates a need for tools that can operate without the supervision of the user. In this paper we look at novel feature selection techniques that address this need. A distributional similarity measure from information theory is applied to measure feature utility. This utility informs the search for both representative and diverse features in two complementary ways: Cluster divides the entire feature space, before then selecting one feature to represent each cluster; and Greedy increments the feature subset size by a greedily selected feature. In particular we found that Greedys local search is suited to learning smaller feature subset sizes while Cluster is able to improve the global quality of larger feature sets. Experiments with four email data sets show significant improvement in retrieval accuracy with nearest neighbour based search methods compared to an existing frequency-based method. Importantly both Greedy and Cluster make significant progress towards the upper bound performance set by a standard supervised feature selection method.

Tracking Drifting Concepts by Time Window Optimisation

This paper addresses the task of learning concept descriptions from streams of data. As new data are obtained the concept description has to be updated regularly to include the new data. In this case we can face the problem that the concept changes over time. Hence the old data become irrelevant to the current concept and have to be removed from the training dataset. This problem is known in the area of machine learning as concept drift. We develop a mechanism that tracks changing concepts using an adaptive time window. The method uses a significance test to detect concept drift and then optimizes the size of the time window, aiming to maximise the classification accuracy on recent data. The method presented is general in nature and can be used with any learning algorithm. The method is tested with three standard learning algorithms (kNN, ID3 and NBC). Three datasets have been used in these experiments. The experimental results provide evidence that the suggested forgetting mechanism is able significantly to improve predictive accuracy on changing concepts.

Acquiring Word Similarities with Higher Order Association Mining

We present a novel approach to mine word similarity in Textual Case Based Reasoning. We exploit indirect associations of words, in addition to direct ones for estimating their similarity. If word Aco-occurs with word B, we say Aand Bshare a first order association between them. If Aco-occurs with Bin some documents, and Bwith Cin some others, then Aand Care said to share a second order co-occurrence via B. Higher orders of co-occurrence may similarly be defined. In this paper we present algorithms for mining higher order co-occurrences. A weighted linear model is used to combine the contribution of these higher orders into a word similarity model. Our experimental results demonstrate significant improvements compared to similarity models based on first order co-occurrences alone. Our approach also outperforms state-of-the-art techniques like SVM and LSI in classification tasks of varying complexity.

Machine learning for improved pathological staging of prostate cancer: A performance comparison on a range of classifiers

OBJECTIVES Prediction of prostate cancer pathological stage is an essential step in a patients pathway. It determines the treatment that will be applied further. In current practice, urologists use the pathological stage predictions provided in Partin tables to support their decisions. However, Partin tables are based on logistic regression (LR) and built from US data. Our objective is to investigate a range of both predictive methods and of predictive variables for pathological stage prediction and assess them with respect to their predictive quality based on U.K. data. METHODS AND MATERIAL The latest version of Partin tables was applied to a large scale British dataset in order to measure their performances by mean of concordance index (c-index). The data was collected by the British Association of Urological Surgeons (BAUS) and gathered records from over 1700 patients treated with prostatectomy in 57 centers across UK. The original methodology was replicated using the BAUS dataset and evaluated using concordance index. In addition, a selection of classifiers, including, among others, LR, artificial neural networks and Bayesian networks (BNs) was applied to the same data and compared with each other using the area under the ROC curve (AUC). Subsets of the data were created in order to observe how classifiers perform with the inclusion of extra variables. Finally a local dataset prepared by the Aberdeen Royal Infirmary was used to study the effect on predictive performance of using different variables. RESULTS Partin tables have low predictive quality (c-index=0.602) when applied on UK data for comparison on patients with organ confined and extra prostatic extension conditions, patients at the two most frequently observed pathological stages. The use of replicate lookup tables built from British data shows an improvement in the classification, but the overall predictive quality remains low (c-index=0.610). Comparing a range of classifiers shows that BNs generally outperform other methods. Using the four variables from Partin tables, naive Bayes is the best classifier for the prediction of each class label (AUC=0.662 for OC). When two additional variables are added, the results of LR (0.675), artificial neural networks (0.656) and BN methods (0.679) are overall improved. BNs show higher AUCs than the other methods when the number of variables raises CONCLUSION The predictive quality of Partin tables can be described as low to moderate on U.K. data. This means that following the predictions generated by Partin tables, many patients would received an inappropriate treatment, generally associated with a deterioration of their quality of life. In addition to demographic differences between U.K. and the original U.S. population, the methodology and in particular LR present limitations. BN represents a promising alternative to LR from which prostate cancer staging can benefit. Heuristic search for structure learning and the inclusion of more variables are elements that further improve BN models quality.

Selecting Bi-Tags for Sentiment Analysis of Text

Sentiment Analysis aims to determine the overall sentiment orientation of a given input text. One motivation for research in this area is the need for consumer related industries to extract public opinion from online portals such as blogs, discussion boards, and reviews. Estimating sentiment orientation in text involves extraction of sentiment rich phrases and the aggregation of their sentiment orientation. Identifying sentiment rich phrases is typically achieved by using manually selected part-of-speech (PoS) patterns. In this paper we present an algorithm for automated discovery of PoS patterns from sentiment rich background data. Here PoS patterns are selected by applying standard feature selection heuristics: Information Gain (IG), Chi-Squared (CHI) score, and Document Frequency (DF). Experimental results from two real-world datasets suggests that classification accuracy is significantly better with DF selected patterns than with IG or the CHI score. Importantly, we also found DF selected patterns to result in comparative classifier accuracy to that of manually selected patterns.

Case Retrieval Reuse Net (CR2N): An Architecture for Reuse of Textual Solutions

This paper proposes textual reuse as the identification of reusable textual constructs in a retrieved solution text. This is done by annotating a solution text so that reusable sections are identifiable from those that need revision. We present a novel and generic architecture, Case Retrieval Reuse Net (CR2N), that can be used to generate these annotations to denote text content as reusable or not. Obtaining evidence for and against reuse is crucial for annotation accuracy, therefore a comparative evaluation of different evidence gathering techniques is presented. Evaluation on two domains of weather forecast revision and health & safety incident reporting shows significantly better accuracy over a retrieve-only system and a comparable reuse technique. This also provides useful insight into the text revision stage.

Fast case retrieval nets for textual data

Case Retrieval Networks (CRNs) facilitate flexible and efficient retrieval in Case-Based Reasoning (CBR) systems. While CRNs scale up well to handle large numbers of cases in the case-base, the retrieval efficiency is still critically determined by the number of feature values (referred to as Information Entities) and by the nature of similarity relations defined over the feature space. In textual domains it is typical to perform retrieval over large vocabularies with many similarity interconnections between words. This can have adverse effects on retrieval efficiency for CRNs. This paper proposes an extension to CRN, called the Fast Case Retrieval Network (FCRN) that eliminates redundant computations at run time. Using artificial and real-world datasets, it is demonstrated that FCRNs can achieve significant retrieval speedups over CRNs, while maintaining retrieval effectiveness.

A propositional approach to textual case indexing

Problem solving with experiences that are recorded in text form requires a mapping from text to structured cases, so that case comparison can provide informed feedback for reasoning. One of the challenges is to acquire an indexing vocabulary to describe cases. We explore the use of machine learning and statistical techniques to automate aspects of this acquisition task. A propositional semantic indexing tool, Psi, which forms its indexing vocabulary from new features extracted as logical combinations of existing keywords, is presented. We propose that such logical combinations correspond more closely to natural concepts and are more transparent than linear combinations. Experiments show Psi-derived case representations to have superior retrieval performance to the original keyword-based representations. Psi also has comparable performance to Latent Semantic Indexing, a popular dimensionality reduction technique for text, which unlike Psi generates linear combinations of the original features.

Contextual Sentiment Analysis in Social Media Using High-Coverage Lexicon

Automatically generated sentiment lexicons offer sentiment information for a large number of terms and often at a more granular level than manually generated ones. While such rich information has the potential of enhancing sentiment analysis, it also presents the challenge of finding the best possible strategy to utilising the information. In SentiWordNet, negation terms and lexical valence shifters (i.e. intensifier and diminisher terms) are associated with sentiment scores. Therefore, such terms could either be treated as sentiment-bearing using the scores offered by the lexicon, or as sentiment modifiers that influence the scores assigned to adjacent terms. In this paper, we investigate the suitability of both these approaches applied to sentiment classification. Further, we explore the role of non-lexical modifiers common to social media and introduce a sentiment score aggregation strategy named SmartSA. Evaluation on three social media datasets show that the strategy is effective and outperform the baseline of using aggregate-and-average approach.