Michael Vogt

Iterative Single Data Algorithm for Training Kernel Machines from Huge Data Sets: Theory and Performance

The chapter introduces the latest developments and results of Itera- tive Single Data Algorithm (ISDA) for solving large-scale support vector machines (SVMs) problems. First, the equality of a Kernel AdaTron (KA) method (originating from a gradient ascent learning approach) and the Sequential Minimal Optimiza- tion (SMO) learning algorithm (based on an analytic quadratic programming step for a model without bias term b) in designing SVMs with positive definite kernels is shown for both the nonlinear classification and the nonlinear regression tasks. The chapter also introduces the classic Gauss-Seidel procedure and its derivative known as the successive over-relaxation algorithm as viable (and usually faster) training al- gorithms. The convergence theorem for these related iterative algorithms is proven. The second part of the chapter presents the effects and the methods of incorporating explicit bias term b into the ISDA. The algorithms shown here implement the single training data based iteration routine (a.k.a. per-pattern learning). This makes the proposed ISDAs remarkably quick. The final solution in a dual domain is not an approximate one, but it is the optimal set of dual variables which would have been obtained by using any of existing and proven QP problem solvers if they only could deal with huge data sets.

On-Line Adaptation of Grid-Based Look-up Tables Using a Fast Linear Regression Technique

Advanced control systems require accurate process models, while processes are often both nonlinear and time variant. After introducing the identification of nonlinear processes with grid-based look-up tables, a new learning algorithm for on-line adaptation of look-up tables is proposed. Using a linear regression approach, this new adaptation algorithm considerably reduces the convergence time in relation to conventional gradientbased adaptation algorithms. An application example and experimental results are shown for the learning feedforward control of the ignition angle of a spark ignition engine. @DOI: 10.1115/1.1849241#

Active-Set Methods for Support Vector Machines

This chapter describes an active-set algorithm for the solution of quadratic programming problems in the context of Support Vector Machines (SVMs). Most of the common SVM optimizers implement working-set algorithms like the SMO method because of their ability to handle large data sets. Although they show generally good results, they may perform weakly in some situations, e.g., if the problem is ill-posed or if high precision is needed. In these cases, active-set techniques (which are robust general-purpose QP solvers) are a reasonable alternative. Algorithms are derived for classification and regression problems for both fixed and variable bias term. The approximation of the solution is considered as well as the comparison with other optimization methods.

An Active-set algorithm for support vector machines in nonlinear system identification

Abstract This contribution describes an active-set algorithm for the optimization of regression support vector machines (SVMs). Its intended use is mainly system identification. Currently, SVMs are computed solving a QP problem by working-set algorithms like the SMO method. Although showing good results in general, they may perform weakly in some situations, particularly when solving regression problems. In these cases, active-set techniques (which are robust general-purpose QP solvers) have been shown to be a reasonable alternative. The paper considers how to adapt them to SVM regession with fixed or variable bias term and applies them to the identification of a condensing boiler.

IDENTIFICATION OF A HYDRAULIC SERVO-AXIS USING SUPPORT VECTOR MACHINES

Abstract In this paper, different models of the pressure buildup inside a hydraulic servo-axis are compared. These models are obtained using RBF networks, local linear models and support vector machines (SVMs), with a particular focus on the latter. For SVMs, a reduction method is derived, which allows to reduce the number of support vectors without losing the generalization abilities of the SVM. Experimental results obtained at a hydraulic servo-axis and a comparison of the different modelling techniques conclude this paper.

Identification of a high efficiency boiler by support vector machines without bias term

Abstract This paper considers the application of support vector machines for the identification of the nonlinear dynamic behavior of a high efficiency boiler. A new algorithm for the computation of support vector machines without bias term is proposed. Whereas the advantages of this concept are known in classification, it has been hardly made use of for regression. The main intention is to provide a simulation tool for the development engineer.

Zeitdiskrete Filteralgorithmen zur Erzeugung zeitlicher Ableitungen (Discrete-Time Filter Algorithms for the Computation of Time-Derivatives)

Für zahlreiche Anwendungen in der Automatisierungstechnik werden die zeitlichen Ableitungen gemessener Signale benötigt. Da diese häufig nicht oder nur mit hohem Aufwand messtechnisch erfasst werden können, ist eine approximative Bestimmung erforderlich. Im Rahmen dieses Beitrags sollen zeitdiskrete Filteralgorithmen vorgestellt werden, die insbesondere bei der Identifikation zeitkontinuierlicher Prozessmodelle zum Einsatz kommen, aber auch für andere Anwendungen geeignet sind. Dazu werden zwei grundsätzliche Methoden betrachtet: Die Zustandsvariablenfilterung und die Methode der Modulationsfunktionen. Beide Verfahren werden durch neue Entwurfsalgorithmen ergänzt und detailliert miteinander verglichen.

An overview of deep learning techniques

Abstract Deep learning is the paradigm that profoundly changed the artificial intelligence landscape within only a few years. Although accompanied by a variety of algorithmic achievements, this technology is disruptive mainly from the application perspective: It considerably pushes the border of tasks that can be automated, changes the way products are developed, and is available to virtually everyone. Subject of deep learning are artificial neural networks with a large number of layers. Compared to earlier approaches with ideally a single layer, this allows using massive computational resources to train black-box models directly on raw data with a minimum of engineering work. Most successful applications are found in visual image understanding, but also in audio and text modeling.

Heart Rate Classification Using Support Vector Machines

This contribution describes a classification technique that improves the heart rate estimation during hemodialysis treatments. After the heart rate is estimated from the pressure signal of the dialysis machine, a classifier decides if it is correctly identified and rejects it if necessary. As the classifier employs a support vector machine, special interest is put on the automatic selection of its user parameters. In this context, a comparison between different optimization techniques is presented, including a gradient projection method as latest development.