Fabio Rivieccio

Hyperparameter design criteria for support vector classifiers

Abstract The design of a support vector machine (SVM) consists in tuning a set of hyperparameter quantities, and requires an accurate prediction of the classifiers generalization performance. The paper describes the application of the maximal-discrepancy criterion to the hyperparameter-setting process, and points out the advantages of such an approach over existing theoretical frameworks. The resulting theoretical predictions are then compared with the k -fold cross-validation empirical method, which probably is the current best-performing approach to the SVM design problem. Experimental results on a wide range of real-world testbeds prove out that the features of the maximal-discrepancy method can notably narrow the gap that so far has separated theoretical and empirical estimates of a classifiers generalization error.

Quantum optimization for training support vector machines

Refined concepts, such as Rademacher estimates of model complexity and nonlinear criteria for weighting empirical classification errors, represent recent and promising approaches to characterize the generalization ability of Support Vector Machines (SVMs). The advantages of those techniques lie in both improving the SVM representation ability and yielding tighter generalization bounds. On the other hand, they often make Quadratic-Programming algorithms no longer applicable, and SVM training cannot benefit from efficient, specialized optimization techniques. The paper considers the application of Quantum Computing to solve the problem of effective SVM training, especially in the case of digital implementations. The presented research compares the behavioral aspects of conventional and enhanced SVMs; experiments in both a synthetic and real-world problems support the theoretical analysis. At the same time, the related differences between Quadratic-Programming and Quantum-based optimization techniques are considered.

K-fold generalization capability assessment for support vector classifiers

The problem of how to effectively implement k-fold cross-validation for support vector machines is considered. Indeed, despite the fact that this selection criterion is widely used due to its reasonable requirements in terms of computational resources and its good ability in identifying a well performing model, it is not clear how one should employ the committee of classifiers coming from the k folds for the task of on-line classification. Three methods are here described and tested, based respectively on: averaging, random choice and majority voting. Each of these methods is tested on a wide range of data-sets for different fold settings.

Training support vector machines: a quantum-computing perspective

Recent advances in characterizing the generalization ability of support vector machines (SVMs) exploit refined concepts, such as Rademacher estimates of model complexity and nonlinear criteria for weighting empirical errors. Those methods improve the SVM representation ability and tighten generalization bounds. On the other hand, quadratic-programming algorithms are no longer applicable, hence the SVM-training process cannot benefit from the notable efficiency featured by those specialized techniques. The paper considers the possibility of using quantum computing to solve the resulting problem of effective optimization, especially in the case of digital SV implementations. The behavioral aspects of conventional and enhanced SVMs are compared, supported by experiments in both a synthetic and a real-world problem. Likewise, the related differences between quadratic-programming and quantum-based optimization techniques are analyzed.

Automatic Hyperparameter Tuning for Support Vector Machines

This work describes the application of the Maximal Discrepancy (MD) criterion to the process of hyperparameter setting in SVMs and points out the advantages of such an approach over existing theoretical and practical frameworks.The resulting theoretical predictions are compared with a k-fold cross-validation empirical method on some benchmark datasets showing that the MD technique can be used for automatic SVM model selection.

A LEARNING-MACHINE BASED METHOD FOR THE SIMULATION OF COMBUSTION PROCESS IN AUTOMOTIVE I.C. ENGINES

In automotive applications problems related to management and diagnostics play an important role to improve engine performance and to reduce fuel consumption and pollutant emissions. In the design of control systems the use of theoretical models for the simulation of engine behaviour proved to be very useful, and it is apparent from the literature. However, since automotive engines have become very complex plants, their modelling requires a comprehensive description of the behaviour of many processes and components. Combustion process has a strong influence on performance and emissions, but its theoretical description can be hardly combined with the requirements of control-oriented models (especially as regards “real-time” applications). Two simplified theoretical models are proposed in the paper, based on a thermodynamic and a simplified approach respectively. In the first case a single-zone method was followed with the introduction of an apparent heat release rate (HRR) described as a superposition of two Wiebe functions. Coefficients of these burning functions are estimated by means of Learning Machines (LM), i.e. Support Vector Machines (SVM), trained from experimental data and then embedded in a Simulink® block. In order to make calculation time shorter, a simpler and faster model based on the application of SVM was defined to describe combustion process. Starting from experimental data, the proposed SVM was trained and implemented in a Simulink® block to evaluate exhaust gas temperature and bmep directly from engine operating parameters. Both blocks were defined to be easily embedded in engine simulation models for control-oriented applications. Results were promising for both models, showing very short computation time. A comparison of theoretical outputs with experimental data is reported in the paper, together with an application of both calculation procedures to a comprehensive model of a modern automotive turbocharged Diesel engine.© 2003 ASME

An Algorithm for Reducing the Number of Support Vectors

According to the Support Vector Machine algorithm, the task of classification depends on a subset of the original data-set, which is the set of Support Vectors (SVs). They are the only information needed to compute the discriminating function between the classes and, therefore, to classify new data. Since both the computational complexity and the memory requirements of the algorithm depend on the number of SVs, this property is very appealing from the point of view of hardware implementations. For this reason, many researchers have proposed new methods to reduce the number of SVs, even at the expenses of a larger error rate. We propose in this work a method which aims at finding a single point per each class, called archetype, which allows to reconstruct the classifier found by the SVM algorithm, without suffering any classification rate loss. The method is also extended to the case of non-linear classification by finding an approximation of the archetypes in the input space, which maintain the ability to classify the data with a moderate increase of the error rate.

Data mining tools: from web to grid architectures

The paradigm of Grid computing is establishing as a novel, reliable and effective method to exploit a pool of hardware resources and make them available to the users. Data-mining benefits from the Grid as it often requires to run time consuming algorithms on large amounts of data which maybe reside on a different resource from the one having the proper data-mining algorithms. Also, in recent times, machine learning methods have been available to the purposes of knowledge discovery, which is a topic of interest for a large community of users. The present work is an account of the evolution of the ways in which a user can be provided with a data-mining service: from a web interface to a Grid service, the exploitation of a complex resource from a technical and a user-friendliness point of view is considered. More specifically, the goal is to show the interest/advantage of running data mining algorithm on the Grid. Such an environment can employ computational and storage resources in an efficient way, making it possible to open data mining services to Grid users and providing services to business contexts.