Balázs Kégl

Aggregate features and ADABOOST for music classification

We present an algorithm that predicts musical genre and artist from an audio waveform. Our method uses the ensemble learner ADABOOST to select from a set of audio features that have been extracted from segmented audio and then aggregated. Our classifier proved to be the most effective method for genre classification at the recent MIREX 2005 international contests in music information extraction, and the second-best method for recognizing artists. This paper describes our method in detail, from feature extraction to song classification, and presents an evaluation of our method on three genre databases and two artist-recognition databases. Furthermore, we present evidence collected from a variety of popular features and classifiers that the technique of classifying features aggregated over segments of audio is better than classifying either entire songs or individual short-timescale features.

Image denoising with complex ridgelets

In this paper, we propose a novel image denoising method by incorporating the dual-tree complex wavelets into the ordinary ridgelet transform. The approximate shift invariant property of the dual-tree complex wavelet and the high directional sensitivity of the ridgelet transform make the new method a very good choice for image denoising. We apply the digital complex ridgelet transform to denoise some standard images corrupted with additive white noise. Experimental results show that the new method outperforms VisuShrink, the ordinary ridgelet image denoising, and wiener2 filter both in terms of peak signal-to-noise ratio and in visual quality. In particular, our method preserves sharp edges better while removing white noise. Complex ridgelets could be applied to curvelet image denoising as well.

Bacterial evolution of antibiotic hypersensitivity

The evolution of resistance to a single antibiotic is frequently accompanied by increased resistance to multiple other antimicrobial agents. In sharp contrast, very little is known about the frequency and mechanisms underlying collateral sensitivity. In this case, genetic adaptation under antibiotic stress yields enhanced sensitivity to other antibiotics. Using large‐scale laboratory evolutionary experiments with Escherichia coli, we demonstrate that collateral sensitivity occurs frequently during the evolution of antibiotic resistance. Specifically, populations adapted to aminoglycosides have an especially low fitness in the presence of several other antibiotics. Whole‐genome sequencing of laboratory‐evolved strains revealed multiple mechanisms underlying aminoglycoside resistance, including a reduction in the proton‐motive force (PMF) across the inner membrane. We propose that as a side effect, these mutations diminish the activity of PMF‐dependent major efflux pumps (including the AcrAB transporter), leading to hypersensitivity to several other antibiotics. More generally, our work offers an insight into the mechanisms that drive the evolution of negative trade‐offs under antibiotic selection.

Boosting products of base classifiers

In this paper we show how to boost products of simple base learners. Similarly to trees, we call the base learner as a subroutine but in an iterative rather than recursive fashion. The main advantage of the proposed method is its simplicity and computational efficiency. On benchmark datasets, our boosted products of decision stumps clearly outperform boosted trees, and on the MNIST dataset the algorithm achieves the second best result among no-domain-knowledge algorithms after deep belief nets. As a second contribution, we present an improved base learner for nominal features and show that boosting the product of two of these new subset indicator base learners solves the maximum margin matrix factorization problem used to formalize the collaborative filtering task. On a small benchmark dataset, we get experimental results comparable to the semi-definite-programming-based solution but at a much lower computational cost.

Genome-wide analysis captures the determinants of the antibiotic cross-resistance interaction network

Understanding how evolution of antimicrobial resistance increases resistance to other drugs is a challenge of profound importance. By combining experimental evolution and genome sequencing of 63 laboratory-evolved lines, we charted a map of cross-resistance interactions between antibiotics in Escherichia coli, and explored the driving evolutionary principles. Here, we show that (1) convergent molecular evolution is prevalent across antibiotic treatments, (2) resistance conferring mutations simultaneously enhance sensitivity to many other drugs and (3) 27% of the accumulated mutations generate proteins with compromised activities, suggesting that antibiotic adaptation can partly be achieved without gain of novel function. By using knowledge on antibiotic properties, we examined the determinants of cross-resistance and identified chemogenomic profile similarity between antibiotics as the strongest predictor. In contrast, cross-resistance between two antibiotics is independent of whether they show synergistic effects in combination. These results have important implications on the development of novel antimicrobial strategies.

Privacy-preserving boosting

We describe two algorithms, BiBoost (Bipartite Boosting) and MultBoost (Multiparty Boosting), that allow two or more participants to construct a boosting classifier without explicitly sharing their data sets. We analyze both the computational and the security aspects of the algorithms. The algorithms inherit the excellent generalization performance of AdaBoost. Experiments indicate that the algorithms are better than AdaBoost executed separately by the participants, and that, independently of the number of participants, they perform close to AdaBoost executed using the entire data set.

Data-dependent margin-based generalization bounds for classification

We derive new margin-based inequalities for the probability of error of classifiers. The main feature of these bounds is that they can be calculated using the training data and therefore may be effectively used for model selection purposes. In particular, the bounds involve empirical complexities measured on the training data (such as the empirical fat-shattering dimension) as opposed to their worst-case counterparts traditionally used in such analyses. Also, our bounds appear to be sharper and more general than recent results involving empirical complexity measures. In addition, we develop an alternative data-based bound for the generalization error of classes of convex combinations of classifiers involving an empirical complexity measure that is easier to compute than the empirical covering number or fat-shattering dimension. We also show examples of efficient computation of the new bounds.

Invariant pattern recognition using contourlets and AdaBoost

In this paper, we propose new methods for palmprint classification and handwritten numeral recognition by using the contourlet features. The contourlet transform is a new two dimensional extension of the wavelet transform using multiscale and directional filter banks. It can effectively capture smooth contours that are the dominant features in palmprint images and handwritten numeral images. AdaBoost is used as a classifier in the experiments. Experimental results show that the contourlet features are very stable features for invariant palmprint classification and handwritten numeral recognition, and better classification rates are reported when compared with other existing classification methods.

Robust Regression by Boosting the Median

Most boosting regression algorithms use the weighted average of base regressors as their final regressor. In this paper we analyze the choice of the weighted median. We propose a general boosting algorithm based on this approach. We prove boosting-type convergence of the algorithm and give clear conditions for the convergence of the robust training error. The algorithm recovers \(\textsc{AdaBoost}\) and \(\textsc{AdaBoost}_\varrho\) as special cases. For boosting confidence-rated predictions, it leads to a new approach that outputs a different decision and interprets robustness in a different manner than the approach based on the weighted average. In the general, non-binary case we suggest practical strategies based on the analysis of the algorithm and experiments.

Combining and adapting software quality predictive models by genetic algorithms

The goal of quality models is to predict a quality factor starting from a set of direct measures. Selecting an appropriate quality model for a particular software is a difficult, non-trivial decision. In this paper, we propose an approach to combine and/or adapt existing models (experts) in such way that the combined/adapted model works well on the particular system. Test results indicate that the models perform significantly better than individual experts in the pool.