Jussi Kujala

Covering analysis of the greedy algorithm for partial cover

The greedy algorithm is known to have a guaranteed approximation performance in many variations of the well-known minimum set cover problem. We analyze the number of elements covered by the greedy algorithm for the minimum set cover problem, when executed for k rounds. This analysis quite easily yields in the p-partial cover problem over a ground set of m elements the harmonic approximation guarantee H(⌈pm⌉) for the number of required covering sets. Thus, we tie together the coverage analysis of the greedy algorithm for minimum set cover and its dual problem partial cover.

A Walk from 2-Norm SVM to 1-Norm SVM

This paper studies how useful the standard 2-norm regularized SVM is in approximating the 1-norm SVM problem. To this end, we examine a general method that is based on iteratively re-weighting the features and solving a 2-norm optimization problem. The convergence rate of this method is unknown. Previous work indicates that it might require an excessive number of iterations. We study how well we can do with just a small number of iterations. In theory the convergence rate is fast, except for coordinates of the current solution that are close to zero. Our empirical experiments confirm this. In many problems with irrelevant features, already one iteration is often enough to produce accuracy as good as or better than that of the 1-norm SVM. Hence, it seems that in these problems we do not need to converge to the 1-norm SVM solution near zero values. The benefit of this approach is that we can build something similar to the 1-norm regularized solver based on any 2-norm regularized solver. This is quick to implement and the solution inherits the good qualities of the solver such as scalability and stability.

Improved Algorithms for Univariate Discretization of Continuous Features

In discretization of a continuous variable its numerical value range is divided into a few intervals that are used in classification. For example, Naive Bayes can benefit from this processing. A commonly-used supervised discretization method is Fayyad and Iranis recursive entropy-based splitting of a value range. The technique uses ent-mdl as a model selection criterion to decide whether to accept the proposed split. We argue that theoretically the method is not always close to ideal for this application. Empirical experiments support our finding. We give a statistical rule that does not use the ad-hoc rule of Fayyad and Iranis approach to increase its performance. This rule, though, is quite time consuming to compute. We also demonstrate that a very simple Bayesian method performs better than ent-mdl as a model selection criterion.

Following the Perturbed Leader to Gamble at Multi-armed Bandits

Following the perturbed leader ( fpl ) is a powerful technique for solving online decision problems. Kalai and Vempala [1] rediscovered this algorithm recently. A traditional model for online decision problems is the multi-armed bandit. In it a gambler has to choose at each round one of the klevers to pull with the intention to minimize the cumulated cost. There are four versions of the nonstochastic optimization setting out of which the most demanding one is a game played against an adaptive adversary in the bandit setting. An adaptive adversary may alter its game strategy of assigning costs to decisions depending on the decisions chosen by the gambler in the past. In the bandit setting the gambler only gets to know the cost of the choice he made, rather than the costs of all available alternatives. In this work we show that the very straightforward and easy to implement algorithm Adaptive Bandit fpl can attain a regret of

Unsupervised Classifier Selection Based on Two-Sample Test

O(\sqrt{T \ln T})

Reducing splaying by taking advantage of working sets

against an adaptive adversary. This regret holds with respect to the best lever in hindsight and matches the previous best regret bounds of

Practical approximation of optimal multivariate discretization

O(\sqrt{T \ln T})

Approximation algorithms for minimizing empirical error by axis-parallel hyperplanes

.

The cost of offline binary search tree algorithms and the complexity of the request sequence

We propose a well-founded method of ranking a pool of mtrained classifiers by their suitability for the current input of ninstances. It can be used when dynamically selecting a single classifier as well as in weighting the base classifiers in an ensemble. No classifiers are executed during the process. Thus, the ninstances, based on which we select the classifier, can as well be unlabeled. This is rare in previous work. The method works by comparing the training distributions of classifiers with the input distribution. Hence, the feasibility for unsupervised classification comes with a price of maintaining a small sample of the training data for each classifier in the pool. In the general case our method takes time and space , where tis the size of the stored sample from the training distribution for each classifier. However, for commonly used Gaussian and polynomial kernel functions we can execute the method more efficiently. In our experiments the proposed method was found to be accurate.

Poketree: a dynamically competitive data structure with good worst-case performance

Access requests to keys stored into a data structure often exhibit locality of reference in practice. Such a regularity can be modeled, e.g., by working sets. In this paper we study to what extent can the existence of working sets be taken advantage of in splay trees. In order to reduce the number of costly splay operations we monitor for information on the current working set and its change. We introduce a simple algorithm which attempts to splay only when necessary. Under worst-case analysis the algorithm guarantees an amortized logarithmic bound. In empirical experiments it is 5% more efficient than randomized splay trees and at most 10% more efficient than the original splay tree. We also briefly analyze the usefulness of the commonly-used Zipfs distribution as a general model of locality of reference.