Mathias M. Adankon

Model selection for the LS-SVM. Application to handwriting recognition

The support vector machine (SVM) is a powerful classifier which has been used successfully in many pattern recognition problems. It has also been shown to perform well in the handwriting recognition field. The least squares SVM (LS-SVM), like the SVM, is based on the margin-maximization principle performing structural risk minimization. However, it is easier to train than the SVM, as it requires only the solution to a convex linear problem, and not a quadratic problem as in the SVM. In this paper, we propose to conduct model selection for the LS-SVM using an empirical error criterion. Experiments on handwritten character recognition show the usefulness of this classifier and demonstrate that model selection improves the generalization performance of the LS-SVM.

Semisupervised Least Squares Support Vector Machine

The least squares support vector machine (LS-SVM), like the SVM, is based on the margin-maximization performing structural risk and has excellent power of generalization. In this paper, we consider its use in semisupervised learning. We propose two algorithms to perform this task deduced from the transductive SVM idea. Algorithm 1 is based on combinatorial search guided by certain heuristics while Algorithm 2 iteratively builds the decision function by adding one unlabeled sample at the time. In term of complexity, Algorithm 1 is faster but Algorithm 2 yields a classifier with a better generalization capacity with only a few labeled data available. Our proposed algorithms are tested in several benchmarks and give encouraging results, confirming our approach.

Help-Training for semi-supervised support vector machines

In this paper, we propose to reinforce the Self-Training strategy in semi-supervised mode by using a generative classifier that may help to train the main discriminative classifier to label the unlabeled data. We call this semi-supervised strategy Help-Training and apply it to training kernel machine classifiers as support vector machines (SVMs) and as least squares support vector machines. In addition, we propose a model selection strategy for semi-supervised training. Experimental results on both artificial and real problems demonstrate that Help-Training outperforms significantly the standard Self-Training. Moreover, compared to other semi-supervised methods developed for SVMs, our Help-Training strategy often gives the lowest error rate.

Optimizing resources in model selection for support vector machine

Tuning SVM kernel parameters is a an important step for achieving a high-performing learning machine. The usual automatic methods used to tune these parameters require an inversion of the Gram-Schmidt matrix or a resolution of an extra quadratic programming problem. In the case of a large dataset these methods require the addition of huge amounts of memory and a long CPU time to the already significant resources used in the SVM training. In this paper, we propose a fast method based on an approximation of the gradient of the empirical error along with incremental learning, which reduces the resources required both in terms of processing time and of storage space.

IBN SINA: a database for research on processing and understanding of Arabic manuscripts images

This paper describes the steps that have been undertaken in order to develop the IBN SINA database, which is designed to apply learning techniques in the processing and understanding of document images. The description of the preparation process, including preprocessing, feature extraction and labeling, is provided. The database has been evaluated using classification techniques, such as the SVM classifiers. In order to make the database compatible with these classifiers, the labels of the shapes have been translated into a set of bi-class problems. Promising results with the SVM classifiers have been obtained.

Genetic algorithm–based training for semi-supervised SVM

The Support Vector Machine (SVM) is an interesting classifier with excellent power of generalization. In this paper, we consider applying the SVM to semi-supervised learning. We propose using an additional criterion with the standard formulation of the semi-supervised SVM (S3VM) to reinforce classifier regularization. Since, we deal with nonconvex and combinatorial problem, we use a genetic algorithm to optimize the objective function. Furthermore, we design the specific genetic operators and certain heuristics in order to improve the optimization task. We tested our algorithm on both artificial and real data and found that it gives promising results in comparison with classical optimization techniques proposed in literature.

Semisupervised Learning Using Bayesian Interpretation: Application to LS-SVM

Bayesian reasoning provides an ideal basis for representing and manipulating uncertain knowledge, with the result that many interesting algorithms in machine learning are based on Bayesian inference. In this paper, we use the Bayesian approach with one and two levels of inference to model the semisupervised learning problem and give its application to the successful kernel classifier support vector machine (SVM) and its variant least-squares SVM (LS-SVM). Taking advantage of Bayesian interpretation of LS-SVM, we develop a semisupervised learning algorithm for Bayesian LS-SVM using our approach based on two levels of inference. Experimental results on both artificial and real pattern recognition problems show the utility of our method.

Optimizing resources in model selection for support vector machines

Tuning SVM kernel parameters is a an important step for achieving a high-performing learning machine. The usual automatic methods used to tune these parameters require an inversion of the Gram-Schmidt matrix or a resolution of an extra quadratic programming problem. In the case of a large dataset these methods require the addition of huge amounts of memory and a long CPU time to the already significant resources used in the SVM training. In this paper, we propose a fast method based on an approximation of the gradient of the empirical error along with incremental learning, which reduces the resources required both in terms of processing time and of storage space.

Prediction of scoliosis curve type based on the analysis of trunk surface topography

Scoliosis treatment strategy is generally chosen according to the severity and type of the spinal curve. Currently, the curve type is determined from X-rays whose acquisition can be harmful for the patient. We propose in this paper a system that can predict the scoliosis curve type based on the analysis of the surface of the trunk. The latter is acquired and reconstructed in 3D using a non invasive multi-head digitizing system. The deformity is described by the back surface rotation, measured on several cross-sections of the trunk. A classifier composed of three support vector machines was trained and tested using the data of 97 patients with scoliosis. A prediction rate of 72.2% was obtained, showing that the use of the trunk surface for a high-level scoliosis classification is feasible and promising.

Scoliosis Follow-Up Using Noninvasive Trunk Surface Acquisition

Adolescent idiopathic scoliosis (AIS) is a musculoskeletal pathology. It is a complex spinal curvature in a 3-D space that also affects the appearance of the trunk. The clinical follow-up of AIS is decisive for its management. Currently, the Cobb angle, which is measured from full spine radiography, is the most common indicator of the scoliosis progression. However, cumulative exposure to X-rays radiation increases the risk for certain cancers. Thus, a noninvasive method for the identification of the scoliosis progression from trunk shape analysis would be helpful. In this study, a statistical model is built from a set of healthy subjects using independent component analysis and genetic algorithm. Based on this model, a representation of each scoliotic trunk from a set of AIS patients is computed and the difference between two successive acquisitions is used to determine if the scoliosis has progressed or not. This study was conducted on 58 subjects comprising 28 healthy subjects and 30 AIS patients who had trunk surface acquisitions in upright standing posture. The model detects 93% of the progressive cases and 80% of the nonprogressive cases. Thus, the rate of false negatives, representing the proportion of undetected progressions, is very low, only 7%. This study shows that it is possible to perform a scoliotic patients follow-up using 3-D trunk image analysis, which is based on a noninvasive acquisition technique.