Gilles Gasso

Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming

This paper considers the problem of recovering a sparse signal representation according to a signal dictionary. This problem could be formalized as a penalized least-squares problem in which sparsity is usually induced by a lscr1-norm penalty on the coefficients. Such an approach known as the Lasso or Basis Pursuit Denoising has been shown to perform reasonably well in some situations. However, it was also proved that nonconvex penalties like the pseudo lscrq-norm with q < 1 or smoothly clipped absolute deviation (SCAD) penalty are able to recover sparsity in a more efficient way than the Lasso. Several algorithms have been proposed for solving the resulting nonconvex least-squares problem. This paper proposes a generic algorithm to address such a sparsity recovery problem for some class of nonconvex penalties. Our main contribution is that the proposed methodology is based on an iterative algorithm which solves at each iteration a convex weighted Lasso problem. It relies on the family of nonconvex penalties which can be decomposed as a difference of convex functions (DC). This allows us to apply DC programming which is a generic and principled way for solving nonsmooth and nonconvex optimization problem. We also show that several algorithms in the literature dealing with nonconvex penalties are particular instances of our algorithm. Experimental results demonstrate the effectiveness of the proposed generic framework compared to existing algorithms, including iterative reweighted least-squares methods.

Histogram of gradients of time-frequency representations for audio scene classification

Presents our entry to the Detection and Classification of Acoustic Scenes challenge. The approach we propose for classifying acoustic scenes is based on transforming the audio signal into a time-frequency representation and then in extracting relevant features about shapes and evolutions of time-frequency structures. These features are based on histogram of gradients that are subsequently fed to a multi-class linear support vector machines.

A multiple kernel framework for inductive semi-supervised SVM learning

We investigate the benefit of combining both cluster assumption and manifold assumption underlying most of the semi-supervised algorithms using the flexibility and the efficiency of multiple kernel learning. The multiple kernel version of Transductive SVM (a cluster assumption based approach) is proposed and it is solved based on DC (Difference of Convex functions) programming. Promising results on benchmark data sets and the BCI data analysis suggest and support the effectiveness of proposed work.

Batch and online learning algorithms for nonconvex neyman-pearson classification

We describe and evaluate two algorithms for Neyman-Pearson (NP) classification problem which has been recently shown to be of a particular importance for bipartite ranking problems. NP classification is a nonconvex problem involving a constraint on false negatives rate. We investigated batch algorithm based on DC programming and stochastic gradient method well suited for large-scale datasets. Empirical evidences illustrate the potential of the proposed methods.

Regularization Paths for ν -SVM and ν -SVR

This paper presents the ν-SVM and theν-SVR full regularization paths along with aleave-one-out inspired stopping criterion and an efficientimplementation. In the ν-SVR method, two parameters areprovided by the user: the regularization parameter Candνwhich settles the width of the ν-tube. Inthe classical ν-SVM method, parameter νisan lower bound on the number of support vectors in the solution.Based on the previous works of [1,2], extensions of regularizationpaths for SVM and SVR are proposed and permit to automaticallycompute the solution path by varying νor theregularization parameter.

New methods for the identification of a stable subspace model for dynamical systems

Dynamical system identification is a widely studied problem. Among all the available models, linear ones are probably the most used, thanks to their efficiency and the theoretical comprehension they allow on the real system. Stability, which is an attractive characteristic of systems, can be simply expressed for the linear systems. As some identification algorithms do not guarantee the stability of the calculated model, this property is used to extend efficiently these procedures to stable system identification. The different approaches proposed in the literature are studied and some new algorithms are proposed to solve this problem. The algorithms are then compared on a simple example to measure their performances.

Oblique principal subspace tracking on manifold

This paper addresses the problem of principal subspace tracking in presence of a colored noise. We propose to extend the YAST algorithm to handle such a case. We also propose a Riemannian framework that could benefit to other classical trackers. Finally, as a proof of concept, our method is compared to the only oblique tracker of the literature on a toy dataset.

Model Selection for Ranking SVM Using Regularization Path

This chapter deals with supervised learning problems under the ranking framework. Ranking algorithms are typically introduced as a tool for personalizing the order in which document recommendations or search results in the web, for example are presented. That is, the more important a result is to the user, the earlier it should be listed. To this end, two possible settings can be considered : i. the algorithm tries to interactively rearrange the results of one search such that relevant results come the closer to the top the more (implicit) feedback the user provides, ii. the algorithm tries to generalize over several queries and presents the results of one search in an order depending on the feedback obtained from previous searches. The first setting deals with an active learning while the second setting deals with a passive supervised learning. This kind of problems have gain major attention given the nowadays amount of available informations. This is without doubt a challenging task in the medium and large scale context. Several methods have been proposed to solve these problems. For the passive setting, the Rankboost algorithm (Freund et al. (2003)) is an adaptation from the Adaboost algorithm to the ranking problem. This is a boosting algorithm which works by iteratively building a linear combination of several “weak” algorithms to form a more accurate algorithm. The Pranking algorithm (Crammer & Singer (2001)) is an online version of the weighted algorithm. The SVRank and RankSVMalgorithms are the adaptation of the Support Vector machines for classification and regression, respectively, while the MPRank (Cortes et al. (2007)) is a magnitude-preserving algorithm, which searches not only to keep the relative position of each sample but also to preserve the distance given by the correct ordering. This last algorithm has as well the form of a regularization problem as the two previous with a different cost function. Later, the Ranking SVM (RankSVM) algorithm was proposed by Herbrich et al. (2000) and Joachims (2002) as an optimization problem with constraints given by the induced graph of the ordered queries’ results. This algorithm forms part of the family of kernel algorithms of the SVM type (Boser et al. (1992); Scholkopf & Smola (2002)). Kernel methods like the SVM or the ranking SVM solve optimization problems of the form O pe n A cc es s D at ab as e w w w .in te ch w eb .o rg

Sparsity regularization path for semi-supervised SVM

Using unlabeled data to unravel the structure of the data to leverage the learning process is the goal of semi supervised learning. A common way to represent this underlying structure is to use graphs. Flexibility of the maximum margin kernel framework allows to model graph smoothness and to build kernel machine for semi supervised learning such as Laplacian SVM [1]. But a common complaint of the practitioner is the long running time of these kernel algorithms for classification of new points. We provide an efficient way of alleviating this problem by using a LI penalization term and a regularization path algorithm to efficiently compute the solution. Empirical evidence shows the benefit of the algorithm.

Improved Model-Free Gait Recognition Based on Human Body Part

Gait recognition aims to identify people through the analysis of the way they walk. The challenge of model-free based gait recognition is to cope with various intra-class variations such as clothing variations, carrying conditions, and angle variations that adversely affect the recognition performance. This chapter proposes a method to select the most discriminative human body part based on group Lasso of motion to reduce the intra-class variation so as to improve the recognition performance. The proposed method is evaluated using CASIA gait database (dataset B), and the experimental results suggest that our method yields 88.75 % of Correct Classification Rate (CCR) when compared to existing state-of-the-art methods.