Tom Howley

The Genetic Kernel Support Vector Machine: Description and Evaluation

The Support Vector Machine (SVM) has emerged in recent years as a popular approach to the classification of data. One problem that faces the user of an SVM is how to choose a kernel and the specific parameters for that kernel. Applications of an SVM therefore require a search for the optimum settings for a particular problem. This paper proposes a classification technique, which we call the Genetic Kernel SVM (GK SVM), that uses Genetic Programming to evolve a kernel for a SVM classifier. Results of initial experiments with the proposed technique are presented. These results are compared with those of a standard SVM classifier using the Polynomial, RBF and Sigmoid kernel with various parameter settings

The effect of principal component analysis on machine learning accuracy with high-dimensional spectral data

This paper presents the results of an investigation into the use of machine learning methods for the identification of narcotics from Raman spectra. The classification of spectral data and other high-dimensional data, such as images, gene-expression data and spectral data, poses an interesting challenge to machine learning, as the presence of high numbers of redundant or highly correlated attributes can seriously degrade classification accuracy. This paper investigates the use of principal component analysis (PCA) to reduce high-dimensional spectral data and to improve the predictive performance of some well-known machine learning methods. Experiments are carried out on a high-dimensional spectral dataset. These experiments employ the NIPALS (Non-Linear Iterative Partial Least Squares) PCA method, a method that has been used in the field of chemometrics for spectral classification, and is a more efficient alternative than the widely used eigenvector decomposition approach. The experiments show that the use of this PCA method can improve the performance of machine learning in the classification of high-dimensional data.

Transfer of Experience Between Reinforcement Learning Environments with Progressive Difficulty

This paper describes an extension to reinforcement learning (RL), in which a standard RL algorithm is augmented with a mechanism for transferring experience gained in one problem to new but related problems. In this approach, named Progressive RL, an agent acquires experience of operating in a simple environment through experimentation, and then engages in a period of introspection, during which it rationalises the experience gained and formulates symbolic knowledge describing how to behave in that simple environment. When subsequently experimenting in a more complex but related environment, it is guided by this knowledge until it gains direct experience. A test domain with 15 maze environments, arranged in order of difficulty, is described. A range of experiments in this domain are presented, that demonstrate the benefit of Progressive RL relative to a basic RL approach in which each puzzle is solved from scratch. The experiments also analyse the knowledge formed during introspection, illustrate how domain knowledge may be incorporated, and show that Progressive Reinforcement Learning may be used to solve complex puzzles more quickly.

Classification of a target analyte in solid mixtures using principal component analysis, support vector machines and Raman spectroscopy

The quantitative analysis of illicit materials using Raman spectroscopy is of widespread interest for law enforcement and healthcare applications. One of the difficulties faced when analysing illicit mixtures is the fact that the narcotic can be mixed with many different cutting agents. This obviously complicates the development of quantitative analytical methods. In this work we demonstrate some preliminary efforts to try and account for the wide variety of potential cutting agents, by discrimination between the target substance and a wide range of excipients. Near-infrared Raman spectroscopy (785 nm excitation) was employed to analyse 217 samples, a number of them consisting of a target analyte (acetaminophen) mixed with excipients of different concentrations by weight. The excipients used were sugars (maltose, glucose, lactose, sorbitol), inorganic materials (talcum powder, sodium bicarbonate, magnesium sulphate), and food products (caffeine, flour). The spectral data collected was subjected to a number of pre-treatment statistical methods including first derivative and normalisation transformations, to make the data more suitable for analysis. Various methods were then used to discriminate the target analytes, these included Principal Component Analysis (PCA), Principal Component Regression (PCR) and Support Vector Machines.

An evolutionary approach to automatic kernel construction

Kernel-based learning presents a unified approach to machine learning problems such as classification and regression. The selection of a kernel and associated parameters is a critical step in the application of any kernel-based method to a problem. This paper presents a data-driven evolutionary approach for constructing kernels, named KTree. An application of KTree to the Support Vector Machine (SVM) classifier is described. Experiments on a synthetic dataset are used to determine the best evolutionary strategy, e.g. what fitness function to use for kernel evaluation. The performance of an SVM based on KTree is compared with that of standard kernel SVMs on a synthetic dataset and on a number of real-world datasets. KTree is shown to outperform or match the best performance of all the standard kernels tested.

Qualitative analysis using Raman spectroscopy and chemometrics: a comprehensive model system for narcotics analysis.

The rapid, on-site identification of illicit narcotics, such as cocaine, is hindered by the diverse nature of the samples, which can contain a large variety of materials in a wide concentration range. This sample variance has a very strong influence on the analytical methodologies that can be utilized and in general prevents the widespread use of quantitative analysis of illicit narcotics on a routine basis. Raman spectroscopy, coupled with chemometric methods, can be used for in situ qualitative and quantitative analysis of illicit narcotics; however, careful consideration must be given to dealing with the extensive variety of sample types. To assess the efficacy of combining Raman spectroscopy and chemometrics for the identification of a target analyte under real-world conditions, a large-scale model sample system (633 samples) using a target (acetaminophen) mixed with a wide variety of excipients was created. Materials that exhibit problematic factors such as fluorescence, variable Raman scattering intensities, and extensive peak overlap were included to challenge the efficacy of chemometric data preprocessing and classification methods. In contrast to spectral matching analyte identification approaches, we have taken a chemometric classification model-based approach to account for the wide variances in spectral data. The first derivative of the Raman spectra from the fingerprint region (750–1900 cm−1) yielded the best classifications. Using a robust segmented cross-validation method, correct classification rates of better than ∼90% could be attained with regression-based classification, compared to ∼35% for SIMCA. This study demonstrates that even with very high degrees of sample variance, as evidenced by dramatic changes in Raman spectra, it is possible to obtain reasonably reliable identification using a combination of Raman spectroscopy and chemometrics. The model sample set can now be used to validate more advanced chemometric or machine learning algorithms being developed for the identification of analytes such as illicit narcotics.

A Machine Learning Application for Classification of Chemical Spectra

This paper presents a software package that allows chemists to analyze spectroscopy data using innovative machine learning (ML) techniques. The package, designed for use in conjunction with lab-based spectroscopic instruments, includes features to encourage its adoption by analytical chemists, such as having an intuitive graphical user interface with a step-by-step ‘wizard’ for building new ML models, supporting standard file types and data preprocessing, and incorporating well-known standard chemometric analysis techniques as well as new ML techniques for analysis of spectra, so that users can compare their performance. The ML techniques that were developed for this application have been designed based on considerations of the defining characteristics of this problem domain, and combine high accuracy with visualization, so that users are provided with some insight into the basis for classification decisions.

The Effect of Principal Component Analysis on Machine Learning Accuracy with High Dimensional Spectral Data.

This paper presents the results of an investigation into the use of machine learning methods for the identification of narcotics from Raman spectra. The classification of spectral data and other high dimensional data, such as images, gene-expression data and spectral data, poses an interesting challenge to machine learning, as the presence of high numbers of redundant or highly correlated attributes can seriously degrade classification accuracy. This paper investigates the use of Principal Component Analysis (PCA) to reduce high dimensional spectral data and to improve the predictive performance of some well known machine learning methods. Experiments are carried out on a high dimensional spectral dataset. These experiments employ the NIPALS (Non-Linear Iterative Partial Least Squares) PCA method, a method that has been used in the field of chemometrics for spectral classification, and is a more efficient alternative than the widely used eigenvector decomposition approach. The experiments show that the use of this PCA method can improve the performance of machine learning in the classification of high dimensionsal data.