Thiemo Gruber

Online Signature Verification With Support Vector Machines Based on LCSS Kernel Functions

In this paper, a new technique for online signature verification or identification is proposed. The technique integrates a longest common subsequences (LCSS) detection algorithm which measures the similarity of signature time series into a kernel function for support vector machines (SVM). LCSS offers the possibility to consider the local variability of signals such as the time series of pen-tip coordinates on a graphic tablet, forces on a pen, or inclination angles of a pen measured during a signing process. Consequently, the similarity of two signature time series can be determined in a more reliable way than with other measures. A proprietary database with signatures of 153 test persons and the SVC 2004 benchmark database are used to show the properties of the new SVM-LCSS. We investigate its parameterization and compare it to SVM with other kernel functions such as dynamic time warping (DTW). Our experiments show that SVM with the LCSS kernel authenticate persons very reliably and with a performance which is significantly better than that of the best comparing technique, SVM with DTW kernel.

Online Segmentation of Time Series Based on Polynomial Least-Squares Approximations

The paper presents SwiftSeg, a novel technique for online time series segmentation and piecewise polynomial representation. The segmentation approach is based on a least-squares approximation of time series in sliding and/or growing time windows utilizing a basis of orthogonal polynomials. This allows the definition of fast update steps for the approximating polynomial, where the computational effort depends only on the degree of the approximating polynomial and not on the length of the time window. The coefficients of the orthogonal expansion of the approximating polynomial-obtained by means of the update steps-can be interpreted as optimal (in the least-squares sense) estimators for average, slope, curvature, change of curvature, etc., of the signal in the time window considered. These coefficients, as well as the approximation error, may be used in a very intuitive way to define segmentation criteria. The properties of SwiftSeg are evaluated by means of some artificial and real benchmark time series. It is compared to three different offline and online techniques to assess its accuracy and runtime. It is shown that SwiftSeg-which is suitable for many data streaming applications-offers high accuracy at very low computational costs.

On-line motif detection in time series with SwiftMotif

This article presents SwiftMotif, a novel technique for on-line motif detection in time series. With this technique, frequently occurring temporal patterns or anomalies can be discovered, for instance. The motif detection is based on a fusion of methods from two worlds: probabilistic modeling and similarity measurement techniques are combined with extremely fast polynomial least-squares approximation techniques. A time series is segmented with a data stream segmentation method, the segments are modeled by means of normal distributions with time-dependent means and constant variances, and these models are compared using a divergence measure for probability densities. Then, using suitable clustering algorithms based on these similarity measures, motifs may be defined. The fast time series segmentation and modeling techniques then allow for an on-line detection of previously defined motifs in new time series with very low run-times. SwiftMotif is suitable for real-time applications, accounts for the uncertainty associated with the occurrence of certain motifs, e.g., due to noise, and considers local variability (i.e., uniform scaling) in the time domain. This article focuses on the mathematical foundations and the demonstration of properties of SwiftMotif-in particular accuracy and run-time-using some artificial and real benchmark time series.

Temporal data mining using shape space representations of time series

Subspace representations that preserve essential information of high-dimensional data may be advantageous for many reasons such as improved interpretability, overfitting avoidance, acceleration of machine learning techniques. In this article, we describe a new subspace representation of time series which we call polynomial shape space representation. This representation consists of optimal (in a least-squares sense) estimators of trend aspects of a time series such as average, slope, curve, change of curve, etc. The shape space representation of time series allows for a definition of a novel similarity measure for time series which we call shape space distance measure. Depending on the application, time series segmentation techniques can be applied to obtain a piecewise shape space representation of the time series in subsequent segments. In this article, we investigate the properties of the polynomial shape space representation and the shape space distance measure by means of some benchmark time series and discuss possible application scenarios in the field of temporal data mining.

SwiftRule: Mining Comprehensible Classification Rules for Time Series Analysis

In this article, we provide a new technique for temporal data mining which is based on classification rules that can easily be understood by human domain experts. Basically, time series are decomposed into short segments, and short-term trends of the time series within the segments (e.g., average, slope, and curvature) are described by means of polynomial models. Then, the classifiers assess short sequences of trends in subsequent segments with their rule premises. The conclusions gradually assign an input to a class. As the classifier is a generative model of the processes from which the time series are assumed to originate, anomalies can be detected, too. Segmentation and piecewise polynomial modeling are done extremely fast in only one pass over the time series. Thus, the approach is applicable to problems with harsh timing constraints. We lay the theoretical foundations for this classifier, including a new distance measure for time series and a new technique to construct a dynamic classifier from a static one, and demonstrate its properties by means of various benchmark time series, for example, Lorenz attractor time series, energy consumption in a building, or ECG data.

All for one or one for all? Combining heterogeneous features for activity spotting

Choosing the right feature for motion based activity spotting is not a trivial task. Often, features derived by intuition or that proved to work well in previous work are used. While feature selection algorithms allow automatic decision, definition of features remains a manual task. We conduct a comparative study of features with very different origin. To this end, we propose a new type of features based on polynomial approximation of signals. The new feature type is compared to features used routinely for motion based activity recognition as well as to recently proposed body-model based features. Experiments were performed on three different, large datasets allowing a thorough, in-depth analysis. They not only show the respective strengths of the different feature types but also their complementarity resulting in improved performance through combination. It shows that each feature type with its individual and complementary strengths and weaknesses can improve results by combination.

Comparing Evolvable Hardware to Conventional Classifiers for Electromyographic Prosthetic Hand Control

Evolvable hardware has shown to be a promising approach for prosthetic hand controllers as it features self-adaptation, fast training, and a compact system-on-chip implementation. Besides these intriguing features, the classification performance is paramount to success for any classifier. However, evolvable hardware classifiers have not yet been sufficiently compared to state-of-the-art conventional classifiers. In this paper, we compare two evolvable hardware approaches for signal classification to three conventional classification techniques: k-nearest-neighbor, decision trees, and support vector machines. We provide all classifiers with features extracted from electromyographic signals taken from forearm muscle contractions, and try to recognize eight different hand movements. Experimental results demonstrate that evolvable hardware approaches are indeed able to compete with state-of-the-art classifiers. Specifically, one of our evolvable hardware approaches delivers a generalization performance similar to that of support vector machines.

Online signature verification with new time series kernels for support vector machines

In this paper, two new methods for online signature verification are proposed. The methods adopt the idea of the longest common subsequences (LCSS) algorithm to a kernel function for Support Vector Machines (SVM). The two kernels LCSS-global and LCSS-local offer the possibility to classify time series of different lengths with SVM. The similarity of two time series is determined very accurately since outliers are ignored. Consequently, LCSS-global and LCSS-local are more robust than algorithms based on dynamic time alignment such as Dynamic Time Warping (DTW). The new methods are compared to other kernel-based methods (DTW-kernel, Fisher-kernel, Gauss-kernel). Our experiments show that SVM with LCSS-local and LCSS-global authenticate persons very reliably.

Classification of Electromyographic Signals: Comparing Evolvable Hardware to Conventional Classifiers

Evolvable hardware (EHW) has shown itself to be a promising approach for prosthetic hand controllers. Besides competitive classification performance, EHW classifiers offer self-adaptation, fast training, and a compact implementation. However, EHW classifiers have not yet been sufficiently compared to state-of-the-art conventional classifiers. In this paper, we compare two EHW approaches to four conventional classification techniques: k-nearest-neighbor, decision trees, artificial neural networks, and support vector machines. We provide all classifiers with features extracted from electromyographic signals taken from forearm muscle contractions, and let the algorithms recognize eight to eleven different kinds of hand movements. We investigate classification accuracy on a fixed data set and stability of classification error rates when new data is introduced. For this purpose, we have recorded a short-term data set from three individuals over three consecutive days and a long-term data set from a single individual over three weeks. Experimental results demonstrate that EHW approaches are indeed able to compete with state-of-the-art classifiers in terms of classification performance.

Blazing Fast Time Series Segmentation Based on Update Techniques for Polynomial Approximations

Segmentation is an important step in processing and analyzing time series. In this article, we present an approach to speed up some standard time series segmentation techniques. Often, time series segmentation is based on piecewise polynomial approximations of the time series (including piecewise constant or linear approximations as special cases). Basically, a least-squares fit with a polynomial has a computational complexity that depends on the number of observations, i.e., the length of the time series. To improve the computational complexity of segmentation techniques we exploit the fact that approximations have to be repeated in sliding (moving) or growing time windows. Therefore, we suggest to use update techniques for the approximations that determine the approximating polynomial in a sliding or growing time window from an already existing one with a computational complexity that is independent of the number of observations, i.e., the length of the window. For that purpose bases of orthogonal polynomials must be used instead of standard bases such as monomials. We take two standard techniques for segmentation - the on-line algorithm SWAB (Sliding Window And Bottom-up) and the off-line technique OptSeg (Optimal Segmentation) - and show that the run-times can be reduced substantially for a given polynomial degree. If run-time constraints are given, e.g., in real-time applications, it would also be possible to adapt the degree of the approximating polynomials. Higher polynomial degrees typically result in lower modeling errors or longer segments. The various properties of the new realizations of segmentation techniques are outlined by means of some benchmark time series. The experimental results show that, depending on the chosen parameterization, OptSeg can be accelerated by some orders of magnitude, SWAB by a factor of up to about ten.