J. De Brabanter

Weighted least squares support vector machines : Robustness and sparse approximation

Abstract Least squares support vector machines (LS-SVM) is an SVM version which involves equality instead of inequality constraints and works with a least squares cost function. In this way, the solution follows from a linear Karush–Kuhn–Tucker system instead of a quadratic programming problem. However, sparseness is lost in the LS-SVM case and the estimation of the support values is only optimal in the case of a Gaussian distribution of the error variables. In this paper, we discuss a method which can overcome these two drawbacks. We show how to obtain robust estimates for regression by applying a weighted version of LS-SVM. We also discuss a sparse approximation procedure for weighted and unweighted LS-SVM. It is basically a pruning method which is able to do pruning based upon the physical meaning of the sorted support values, while pruning procedures for classical multilayer perceptrons require the computation of a Hessian matrix or its inverse. The methods of this paper are illustrated for RBF kernels and demonstrate how to obtain robust estimates with selection of an appropriate number of hidden units, in the case of outliers or non-Gaussian error distributions with heavy tails.

Optimized fixed-size kernel models for large data sets

A modified active subset selection method based on quadratic Renyi entropy and a fast cross-validation for fixed-size least squares support vector machines is proposed for classification and regression with optimized tuning process. The kernel bandwidth of the entropy based selection criterion is optimally determined according to the solve-the-equation plug-in method. Also a fast cross-validation method based on a simple updating scheme is developed. The combination of these two techniques is suitable for handling large scale data sets on standard personal computers. Finally, the performance on test data and computational time of this fixed-size method are compared to those for standard support vector machines and @n-support vector machines resulting in sparser models with lower computational cost and comparable accuracy.

New models to predict depth of infiltration in endometrial carcinoma based on transvaginal sonography

Preoperative knowledge of the depth of myometrial infiltration is important in patients with endometrial carcinoma. This study aimed at assessing the value of histopathological parameters obtained from an endometrial biopsy (Pipelle® de Cornier; results available preoperatively) and ultrasound measurements obtained after transvaginal sonography with color Doppler imaging in the preoperative prediction of the depth of myometrial invasion, as determined by the final histopathological examination of the hysterectomy specimen (the gold standard).

Fixed-size LS-SVM applied to the Wiener-Hammerstein Benchmark

Abstract This paper reports on the application of Fixed-Size Least Squares Support Vector Machines (FS-LSSVM) for the identification of the SYSID 2009 Wiener-Hammerstein benchmark data set. The FS-LSSVM is a modification of the standard Support Vector Machine and Least Squares Support Vector Machine (LS-SVM) designed to handle very large data sets. This approach is taken to estimate a nonlinear black-box (NARX) model from given input/output measurements. We indicate how to tune this approach to the specific case study. We obtain a best root mean squared error of 4.7×10 −3 on simulation of the predefined test set.

Using artificial neural networks to predict malignancy of ovarian tumors

This paper discusses the application of artificial neural networks (ANNs) to preoperative discrimination between benign and malignant ovarian tumors. With the input variables selected by logistic regression analysis, two types of feed-forward neural networks were built: multi-layer perceptrons (MLPs) and generalized regression networks (GRNNs). We assess the performance of the models using the Receiver Operating Characteristic (ROC) curve, particularly the area under the ROC curves (AUC), and statistically compare the cross-validated estimate of the AUC of different models.

Least squares support vector machine regression for discriminant analysis

Support vector machine (SVM) classifiers aim at constructing a large margin classifier in the feature space, while a nonlinear decision boundary is obtained in the input space by mapping the inputs in a nonlinear way to a possibly infinite dimensional feature space. Mercers condition is applied to avoid an explicit expression for the nonlinear mapping and the solution follows from a finite dimensional quadratic programming problem. Recently, other classifier formulations related to a regularized form of Fisher discriminant analysis have been proposed in the feature space for which practical expressions are obtained in a second step by applying the Mercer condition. In this paper, we relate these techniques to least squares SVM, for which the solution follows from a linear Karush-Kuhn-Tucker system in the dual space. Based on the link with empirical linear discriminant analysis one can adjust the bias term in order to take prior information on the class distributions into account and to analyze unbalanced training sets.

Identification of a Pilot Scale Distillation Column: A Kernel Based Approach

Abstract This paper describes the identification of a binary distillation column with Least-Squares Support Vector Machines (LS-SVM). It is our intention to investigate whether a kernel based model, particularly an LS-SVM, can be used for the simulation of the top and bottom temperature of a binary distillation column. Furthermore, we compare the latter model with standard linear models by means of mean-squared error (MSE). It will be demonstrated that this nonlinear model class achieves higher performances in MSE than linear models in the presence of nonlinear distortions. When the system is close to linear, the performance of the LS-SVM is only slightly better than the linear models.

Absence of correlation between risk factors for endometrial cancer and the presence of tamoxifen-associated endometrial polyps in postmenopausal patients with breast cancer

In order to investigate the presence of established risk factors for endometrial carcinoma in postmenopausal patients with breast cancer and with tamoxifen-associated endometrial polyps we compared a group of 25 patients with tamoxifen-associated endometrial polyps with 25 tamoxifen-treated patients without endometrial polyps. No significant differences were found between both groups of patients in age, parity, time after breast cancer and after menopause, duration and daily and total cumulative dose of tamoxifen intake, body mass index and serum levels of luteinising hormone (LH), follicle-stimulating hormone (FSH), oestradiol (E2), progesterone, sex hormone-binding globulin (SHBG), tamoxifen and CA125. So far there is no evidence that these polyps are premalignant lesions.

Implementation and experimental validation of classic MPC on Programmable Logic Controllers

Over the last years, a number of publications were written about Model Predictive Control (MPC) on industrial Programmable Logic Controllers (PLC). They focussed on explicit MPC strategies to provide a fast solution. When sufficient time is available to solve a classic MPC problem, an online solution to the corresponding Quadratic Problem (QP) can be provided. This paper investigates the use of an online quadratic programming solver to exploit MPC on a PLC. This will be illustrated with the classic Hildreth QP algorithm and qpOASES, a recently developed online active set strategy. These algorithms will be investigated on a MISO system.

Kernel Regression with Correlated Errors

Abstract It is a well-known problem that obtaining a correct bandwidth in nonparametric regression is difficult in the presence of correlated errors. There exist a wide variety of methods coping with this problem, but they all critically depend on a tuning procedure which requires accurate information about the correlation structure. Since the errors cannot be observed, the latter is a hard goal to achieve. In this paper, we show the breakdown of several data-driven parameter selection procedures. We also develop a bandwidth selection procedure based on bimodal kernels which successfully removes the error correlation without requiring any prior knowledge about its structure. Some extensions are made to use such a criterion in least squares support vector machines for regression.