Yoram Singer

Efficient projections onto the l 1 -ball for learning in high dimensions

We describe efficient algorithms for projecting a vector onto the l1-ball. We present two methods for projection. The first performs exact projection in O(n) expected time, where n is the dimension of the space. The second works on vectors k of whose elements are perturbed outside the l1-ball, projecting in O(k log(n)) time. This setting is especially useful for online learning in sparse feature spaces such as text categorization applications. We demonstrate the merits and effectiveness of our algorithms in numerous batch and online learning tasks. We show that variants of stochastic gradient projection methods augmented with our efficient projection procedures outperform interior point methods, which are considered state-of-the-art optimization techniques. We also show that in online settings gradient updates with l1 projections outperform the exponentiated gradient algorithm while obtaining models with high degrees of sparsity.

Pegasos: primal estimated sub-gradient solver for SVM

We describe and analyze a simple and effective stochastic sub-gradient descent algorithm for solving the optimization problem cast by Support Vector Machines (SVM). We prove that the number of iterations required to obtain a solution of accuracy

A Stochastic Quasi-Newton Method for Large-Scale Optimization

{\epsilon}

Local collaborative ranking

is

Online multiclass learning by interclass hypothesis sharing

{\tilde{O}(1 / \epsilon)}

Individual Sequence Prediction Using Memory-Efficient Context Trees

, where each iteration operates on a single training example. In contrast, previous analyses of stochastic gradient descent methods for SVMs require