Gaetano Zanghirati

A parallel solver for large quadratic programs in training support vector machines

This work is concerned with the solution of the convex quadratic programming problem arising in training the learning machines named support vector machines. The problem is subject to box constraints and to a single linear equality constraint; it is dense and, for many practical applications, it becomes a large-scale problem. Thus, approaches based on explicit storage of the matrix of the quadratic form are not practicable. Here we present an easily parallelizable approach based on a decomposition technique that splits the problem into a sequence of smaller quadratic programming subproblems. These subproblems are solved by a variable projection method that is well suited to a parallel implementation and is very effective in the case of Gaussian support vector machines. Performance results are presented on well known large-scale test problems, in scalar and parallel environments. The numerical results show that the approach is comparable on scalar machines with a widely used technique and can achieve good efficiency and scalability on a multiprocessor system.

Gradient projection methods for quadratic programs and applications in training support vector machines

Gradient projection methods based on the Barzilai–Borwein spectral steplength choices are considered for quadratic programming (QP) problems with simple constraints. Well-known nonmonotone spectral projected gradient methods and variable projection methods are discussed. For both approaches, the behavior of different combinations of the two spectral steplengths is investigated. A new adaptive steplength alternating rule is proposed, which becomes the basis for a generalized version of the variable projection method (GVPM). Convergence results are given for the proposed approach and its effectiveness is shown by means of an extensive computational study on several test problems, including the special quadratic programs arising in training support vector machines (SVMs). Finally, the GVPM behavior as inner QP solver in decomposition techniques for large-scale SVMs is also evaluated.

Towards real-time image deconvolution: application to confocal and STED microscopy

Although deconvolution can improve the quality of any type of microscope, the high computational time required has so far limited its massive spreading. Here we demonstrate the ability of the scaled-gradient-projection (SGP) method to provide accelerated versions of the most used algorithms in microscopy. To achieve further increases in efficiency, we also consider implementations on graphic processing units (GPUs). We test the proposed algorithms both on synthetic and real data of confocal and STED microscopy. Combining the SGP method with the GPU implementation we achieve a speed-up factor from about a factor 25 to 690 (with respect the conventional algorithm). The excellent results obtained on STED microscopy images demonstrate the synergy between super-resolution techniques and image-deconvolution. Further, the real-time processing allows conserving one of the most important property of STED microscopy, i.e the ability to provide fast sub-diffraction resolution recordings.

Parallel Interior-Point Method for Linear and Quadratic Programs with Special Structure

This paper concerns the use of iterative solvers in interior-point methods for linear and quadratic programming problems. We state an adaptive termination rule for the inner iterative scheme and we prove the global convergence of the obtained algorithm, exploiting the theory developed for inexact Newton methods. This approach is promising for problems with special structure on parallel computers. We present an application on Cray T3E/256 and SGI Origin 2000/64 arising in stochastic linear programming and robust optimization, where the constraint matrix is block-angular and extremely large.

Parallel decomposition approaches for training support vector machines

Publisher Summary This chapter discusses parallel decomposition techniques for solving the large quadratic programming (QP) problems arising in training support vector machines. A recent technique is improved by introducing an efficient solver for the inner QP sub-problems and a preprocessing step useful to hot start the decomposition strategy. The effectiveness of the proposed improvements is evaluated by solving large-scale benchmark problems on different parallel architectures. For a parallel solution of the QP sub-problem at each decomposition iteration, a generalized variable projection method is proposed based on an adaptive step-length selection. Since this solver has the same cost per iteration and a better convergence rate than the variable projection method used in, a remarkable time reduction is observed in the sub-problems solution. The computational experiments on well known large-scale test problems showed that the new decomposition approach, based on the above improvements, outperforms the technique, in both, in serial and parallel environments and can further achieve a better scalability.

A Cray T3E implementation of a parallel stochastic dynamic assets and liabilities management model

Asset and liability management (ALM) models represent an important tool for banks and finance companies to measure the volatility of expected revenues. These models ‐ usually static and deterministic to fit conventional computer resources ‐ may be much more useful if a dynamic stochastic simulation is adopted. This makes it possible to increase the precision of risk estimation. In this paper the parallelization strategy adopted to implement such a stochastic ALM code is described, together with porting and code performance issues. The very good timings obtained on a 128-proc Cray T3E are reported. Anyway the code is easily portable on other, possibly heterogeneous, high-performance computing platforms. ” 2000 Elsevier Science B.V. All rights reserved.

The ROI CT problem: a shearlet-based regularization approach

The possibility to significantly reduce the X-ray radiation dose and shorten the scanning time is particularly appealing, especially for the medical imaging community. Region- of-interest Computed Tomography (ROI CT) has this potential and, for this reason, is currently receiving increasing attention. Due to the truncation of projection images, ROI CT is a rather challenging problem. Indeed, the ROI reconstruction problem is severely ill-posed in general and naive local reconstruction algorithms tend to be very unstable. To obtain a stable and reliable reconstruction, under suitable noise circumstances, we formulate the ROI CT problem as a convex optimization problem with a regularization term based on shearlets, and possibly nonsmooth. For the solution, we propose and analyze an iterative approach based on the variable metric inexact line-search algorithm (VMILA). The reconstruction performance of VMILA is compared against different regularization conditions, in the case of fan-beam CT simulated data. The numerical tests show that our approach is insensitive to the location of the ROI and remains very stable also when the ROI size is rather small.

Shearlet-based regularized ROI reconstruction in fan beam computed tomography

Region-of-interest (ROI) reconstruction in computed tomography (CT) is a problem receiving increasing attention in the medical imaging community, due to its potential to lower exposure to X-ray radiation and to reduce the scanning time. Since the ROI reconstruction problem requires to deal with truncated projection images, classical CT reconstruction algorithms tend to become very unstable and the solution of this problem requires either ad hoc analytic formulas or more sophisticated numerical schemes. In this paper, we introduce a novel approach for ROI CT reconstruction, formulated as a convex optimization problem with a regularized functional based on shearlets or wavelets. Our numerical implementation consists of an iterative algorithm based on the scaled gradient projection method. As illustrated by numerical tests in the context of fan beam CT, our algorithm is insensitive to the location of the ROI and remains very stable also when the ROI size is rather small.

Some improvements to a parallel decomposition technique for training support vector machines

We consider a parallel decomposition technique for solving the large quadratic programs arising in training the learning methodology Support Vector Machine. At each iteration of the technique a subset of the variables is optimized through the solution of a quadratic programming subproblem. This inner subproblem is solved in parallel by a special gradient projection method. In this paper we consider some improvements to the inner solver: a new algorithm for the projection onto the feasible region of the optimization subproblem and new linesearch and steplength selection strategies for the gradient projection scheme. The effectiveness of the proposed improvements is evaluated, both in terms of execution time and relative speedup, by solving large-scale benchmark problems on a parallel architecture.

Global convergence of nonmonotone strategies in parallel methods for block-bordered nonlinear systems

In this paper we present a new method for solving block-bordered nonlinear systems of equations. This method is based on the modified Feng-Schnabel algorithm of G. Zanghirati (Global convergence extension of Feng-Schnabel algorithm for block bordered nonlinear systems, Technical report No. 252, Mathematics Department, University of Ferrara, 1997) for the selection of the search direction. The resulting technique is a nonmonotone strategy that we prove to be globally convergent. Furthermore, the multilevel Newton-like algorithm we propose maintains the intrinsic parallelism due to the sparsity structure of the problem, so it is very suitable for a parallel implementation on distributed memory multiprocessor architectures. A case study is given as a numerical example.