Annabella Astorino

Nonsmooth Optimization Techniques for Semisupervised Classification

We apply nonsmooth optimization techniques to classification problems, with particular reference to the transductive support vector machine (TSVM) approach, where the considered decision function is nonconvex and nondifferentiable, hence difficult to minimize. We present some numerical results obtained by running the proposed method on some standard test problems drawn from the binary classification literature.

Scaling Up Support Vector Machines Using Nearest Neighbor Condensation

In this brief, we describe the FCNN-SVM classifier, which combines the support vector machine (SVM) approach and the fast nearest neighbor condensation classification rule (FCNN) in order to make SVMs practical on large collections of data. As a main contribution, it is experimentally shown that, on very large and multidimensional data sets, the FCNN-SVM is one or two orders of magnitude faster than SVM, and that the number of support vectors (SVs) is more than halved with respect to SVM. Thus, a drastic reduction of both training and testing time is achieved by using the FCNN-SVM. This result is obtained at the expense of a little loss of accuracy. The FCNN-SVM is proposed as a viable alternative to the standard SVM in applications where a fast response time is a fundamental requirement.

Ellipsoidal separation for classification problems

We state the problem of the optimal separation via an ellipsoid in ℝ n of a discrete set of points from another discrete set of points. Our formulation requires the minimization of a convex nonsmooth (piecewise affine) function under the constraint that the matrix of the decision variables is positive definite. We describe a heuristic algorithm of the local search type embedding some ideas coming from nonsmooth optimization. Finally, we present the numerical results obtained by running our method on some standard test problems drawn from the binary classification literature.

Non-smoothness in classification problems

We review the role played by non-smooth optimization techniques in many recent applications in classification area. Starting from the classical concept of linear separability in binary classification, we recall the more general concepts of polyhedral, ellipsoidal and max–min separability. Finally we focus our attention on the support vector machine (SVM) approach and on the more recent transductive SVM technique.

Piecewise-quadratic Approximations in Convex Numerical Optimization

We present a bundle method for convex nondifferentiable minimization where the model is a piecewise-quadratic convex approximation of the objective function. Unlike standard bundle approaches, the model only needs to support the objective function from below at a properly chosen (small) subset of points, as opposed to everywhere. We provide the convergence analysis for the algorithm, with a general form of master problem which combines features of trust region stabilization and proximal stabilization, taking care of all the important practical aspects such as proper handling of the proximity parameters and the bundle of information. Numerical results are also reported.

A fixed-center spherical separation algorithm with kernel transformations for classification problems

We consider a special case of the optimal separation, via a sphere, of two discrete point sets in a finite dimensional Euclidean space. In fact we assume that the center of the sphere is fixed. In this case the problem reduces to the minimization of a convex and nonsmooth function of just one variable, which can be solved by means of an “ad hoc” method in O(p log p) time, where p is the dataset size. The approach is suitable for use in connection with kernel transformations of the type adopted in the support vector machine (SVM) approach. Despite of its simplicity the method has provided interesting results on several standard test problems drawn from the binary classification literature.

Support Vector Machine Polyhedral Separability in Semisupervised Learning

We introduce separation margin maximization, a characteristic of the Support Vector Machine technique, into the approach to binary classification based on polyhedral separability and we adopt a semisupervised classification framework.In particular, our model aims at separating two finite and disjoint sets of points by means of a polyhedral surface in the semisupervised case, that is, by exploiting information coming from both labeled and unlabeled samples. Our formulation requires the minimization of a nonconvex nondifferentiable error function. Numerical results are presented on several data sets drawn from the literature.

A Nonmonotone Proximal Bundle Method with (Potentially) Continuous Step Decisions

We present a convex nondifferentiable minimization algorithm of proximal bundle type that does not rely on measuring descent of the objective function to declare the so-called serious steps; rather, a merit function is defined which is decreased at each iteration, leading to a (potentially) continuous choice of the stepsize between zero (the null step) and one (the serious step). By avoiding the discrete choice the convergence analysis is simplified, and we can more easily obtain efficiency estimates for the method. Some choices for the step selection actually reproduce the dichotomic behavior of standard proximal bundle methods but shed new light on the rationale behind the process, and ultimately with different rules; furthermore, using nonlinear upper models of the function in the step selection process can lead to actual fractional steps.

The Proximal Trajectory Algorithm in SVM Cross Validation

We propose a bilevel cross-validation scheme for support vector machine (SVM) model selection based on the construction of the entire regularization path. Since such path is a particular case of the more general proximal trajectory concept from nonsmooth optimization, we propose for its construction an algorithm based on solving a finite number of structured linear programs. Our methodology, differently from other approaches, works directly on the primal form of SVM. Numerical results are presented on binary data sets drawn from literature.

Edge detection by spherical separation

We describe an optimization-based method for tackling the classic image processing problem known as edge detection and we formulate it in the form of a classification one. The novelty of the approach is in the use of spherical separation as a classification tool in the image processing framework. Spherical separation consists in separating by means of a sphere two given discrete point-sets in a finite dimensional Euclidean space; in our context the two sets are the edge points and the non-edge points, respectively, in the digital representation of a given image. Assuming that the center of the sphere is fixed, the problem reduces to the minimization of a convex and nonsmooth function of just one variable, which can be effectively solved by means of an “ad hoc” bisection method. The results of our experiments on some edge detection benchmark images are provided.