Stephan K. Chalup

Support vector clustering through proximity graph modelling

Support vector machines (SVMs) have been widely adopted for classification, regression and novelty detection. Recent studies (A. Ben-Hur et al., 2001) proposed to employ them for cluster analysis too. The basis of this support vector clustering (SVC) is density estimation through SVM training. SVC is a boundary-based clustering method, where the support information is used to construct cluster boundaries. Despite its ability to deal with outliers, to handle high dimensional data and arbitrary boundaries in data space, there are two problems in the process of cluster labelling. The first problem is its low efficiency when the number of free support vectors increases. The other problem is that it sometimes produces false negatives. We propose a robust cluster assignment method that harvests clustering results efficiently. Our method uses proximity graphs to model the proximity structure of the data. We experimentally analyze and illustrate the performance of this new approach.

Incremental training of first order recurrent neural networks to predict a context-sensitive language

In recent years it has been shown that first order recurrent neural networks trained by gradient-descent can learn not only regular but also simple context-free and context-sensitive languages. However, the success rate was generally low and severe instability issues were encountered. The present study examines the hypothesis that a combination of evolutionary hill climbing with incremental learning and a well-balanced training set enables first order recurrent networks to reliably learn context-free and mildly context-sensitive languages. In particular, we trained the networks to predict symbols in string sequences of the context-sensitive language [a(n)b(n)c(n); n > or = 1. Comparative experiments with and without incremental learning indicated that incremental learning can accelerate and facilitate training. Furthermore, incrementally trained networks generally resulted in monotonic trajectories in hidden unit activation space, while the trajectories of non-incrementally trained networks were oscillating. The non-incrementally trained networks were more likely to generalise.

Incremental learning in biological and machine learning systems.

Incremental learning concepts are reviewed in machine learning and neurobiology. They are identified in evolution, neurodevelopment and learning. A timeline of qualitative axon, neuron and synapse development summarizes the review on neurodevelopment. A discussion of experimental results on data incremental learning with recurrent artificial neural networks reveals that incremental learning often seems to be more efficient or powerful than standard learning but can produce unexpected side effects. A characterization of incremental learning is proposed which takes the elaborated biological and machine learning concepts into account.

A Liver Segmentation Algorithm Based on Wavelets and Machine Learning

This paper introduces an automatic liver parenchyma segmentation algorithm that can delineate liver in abdominal CT images. The proposed approach consists of three main steps. Firstly, a texture analysis is applied onto input abdominal CT images to extract pixel level features. Here, two main categories of features, namely Wavelet coefficients and Haralick texture descriptors are investigated. Secondly, support vector machines (SVM) are implemented to classify the data into pixel-wised liver or non-liver. Finally, specially combined morphological operations are designed as a post processor to remove noise and to delineate the liver. Our unique contributions to liver segmentation are twofold: one is that it has been proved through experiments that wavelet features present better classification than Haralick texture descriptors when SVMs are used; the other is that the combination of morphological operations with a pixel-wised SVM classifier can delineate volumetric liver accurately. The algorithm can be used in an advanced computer-aided liver disease diagnosis and surgical planning systems. Examples of applying the algorithm on real CT data are presented with performance validation based on the automatically segmented results and that of manually segmented ones.

Kernel Methods in Finance

Kernel methods (Cristianini and Shawe-Taylor 2000; Herbrich 2002; Scholkopf and Smola 2002; Shawe-Taylor and Cristianini 2004) can be regarded as machine learning techniques which are “kernelised” versions of other fundamental machine learning methods. The latter include traditional methods for linear dimensionality reduction such as principal component analysis (PCA) (Jolliffe 1986), methods for linear regression and methods for linear classification such as linear support vector machines (Cristianini and Shawe-Taylor 2000; Boser et al. 1992; Vapnik 2006b). For all these methods corresponding “kernel versions” have been developed which can turn them into non-linear methods. Kernel methods are very powerful, precise tools that open the door to a large variety of complex non-linear tasks which previously were beyond the horizon of feasibility, or could not appropriately be analysed with traditional machine learning techniques. However, with kernelisation come a number of new tasks and challenges that need to be addressed and considered. For example, for each application of a kernel method a suitable kernel and associated kernel parameters have to be selected. Also, high-dimensional nonlinear data can be extremely complex and can feature counter-intuitive pitfalls (Verleysen and Francois 2005).

Evaluation of Colour Models for Computer Vision Using Cluster Validation Techniques

Computer vision systems frequently employ colour segmentation as a step of feature extraction. This is particularly crucial in an environment where important features are colour-coded, such as robot soccer. This paper describes a method for determining an appropriate colour model by measuring the compactness and separation of clusters produced by the k-means algorithm. RGB, HSV, YC b C r and CIE L*a*b* colour models are assessed for a selection of artificial and real images, utilising an implementation of the Dunn’s-based cluster validation index. The effectiveness of the method is assessed by qualitatively comparing the relative correctness of the segmentation to the results of the cluster validation. Results demonstrate a significant variation in segmentation quality among colour spaces, and that YC b C r is the best choice for the DARwIn-OP platform tested.

Regime Type and International Conflict: Towards a General Model

The authors take a new look at the relationship between regime type and deadly militarized conflict among pairs of states (dyads) in the international system. With the goal of describing the general functional form, they evaluate three perspectives: democratic peace, regime similarity and regime rationality. They employ both standard logistic regression (logit) and a recently developed machine learning technique, a support vector machine (SVM). Logit is dependent on assumptions that limit flexibility and make it difficult to discern the appropriate functional form. SVM estimation, on the other hand, is highly flexible and appears capable of discovering a relationship that is contingent on other variables in the model. SVM results indicate that regime similarity and joint democracy are important in most dyadic interactions. However, for the special but important case of the most dangerous dyads, regime rationality plays a role and the democratic peace effect is dominant. The results suggest that models of international conflict excluding distinct indicators for political similarity, joint democracy and joint autocracy may be misspecified. SVMs are an especially useful complement to conventional statistical methods.

Variations of the two-spiral task

The two-spiral task is a well-known benchmark for binary classification. The data consist of points on two intertwined spirals which cannot be linearly separated. This article reviews how this task and some of its variations have significantly inspired the development of several important methods in the history of artificial neural networks. The two-spiral task became popular for several different reasons: (1) it was regarded as extremely challenging; (2) it belonged to a suite of standard benchmark tasks; and (3) it had visual appeal and was convenient to use in pilot studies. The article also presents an example which demonstrates how small variations of the two-spiral task such as relative rotations of the two spirals can lead to qualitatively different generalisation results.

Foreign Exchange Trading with Support Vector Machines

This paper analyzes and examines the general ability of Support Vector Machine (SVM) models to correctly predict and trade daily EUR exchange rate directions. Seven models with varying kernel functions are considered. Each SVM model is benchmarked against traditional forecasting techniques in order to ascertain its potential value as out-of-sample forecasting and quantitative trading tool. It is found that hyperbolic SVMs perform well in terms of forecasting accuracy and trading results via a simulated strategy. This supports the idea that SVMs are promising learning systems for coping with nonlinear classification tasks in the field of financial time series applications.

GDTW-P-SVMs: Variable-length time series analysis using support vector machines

We describe a new technique for sequential data analysis, called GDTW-P-SVMs. It is a maximum margin method for the construction of classifiers with variable-length input series. It employs potential support vector machines (P-SVMs) and Gaussian Dynamic Time Warping (GDTW) to waive the fixed-length restriction of feature vectors in training and test data. As a result, GDTW-P-SVMs enjoy the P-SVM methods properties such as the ability to: (i) handle data and kernel matrices that are neither positive definite nor square and (ii) minimise a scale-invariant capacity measure. The new technique elaborates on the P-SVM kernel functions, by utilising the well-known dynamic time warping algorithm to provide an elastic distance measure for the kernel functions. Benchmarks for classification are performed with several real-world data sets from the UCR time series classification/clustering page, the GeoLife trajectory data set, and the UCI Machine Learning Repository. The data sets include data with both variable and fixed-length input series. The results show that the new method performs significantly better than the benchmarked standard classification methods.