Süreyya Özögür-Akyüz

Infinite kernel learning via infinite and semi-infinite programming

As data become heterogeneous, multiple kernel learning methods may help to classify them. To overcome the drawback lying in its (multiple) finite choice, we propose a novel method of ‘infinite’ kernel combinations for learning problems with the help of infinite and semi-infinite optimizations. Looking at all the infinitesimally fine convex combinations of the kernels from an infinite kernel set, the margin is maximized subject to an infinite number of constraints with a compact index set and an additional (Riemann–Stieltjes) integral constraint due to the combinations. After a parametrization in the space of probability measures, we get a semi-infinite programming problem. We analyse regularity conditions (reduction ansatz) and discuss the type of density functions in the constraints and the bilevel optimization problem derived. Our proposed approach is implemented with the conceptual reduction method and tested on homogeneous and heterogeneous data; this yields a better accuracy than a single-kernel learning for the heterogeneous data. We analyse the structure of problems obtained and discuss structural frontiers, trade-offs and research challenges.

A review on data mining and continuous optimization applications in computational biology and medicine.

An emerging research area in computational biology and biotechnology is devoted to mathematical modeling and prediction of gene-expression patterns; it nowadays requests mathematics to deeply understand its foundations. This article surveys data mining and machine learning methods for an analysis of complex systems in computational biology. It mathematically deepens recent advances in modeling and prediction by rigorously introducing the environment and aspects of errors and uncertainty into the genetic context within the framework of matrix and interval arithmetics. Given the data from DNA microarray experiments and environmental measurements, we extract nonlinear ordinary differential equations which contain parameters that are to be determined. This is done by a generalized Chebychev approximation and generalized semi-infinite optimization. Then, time-discretized dynamical systems are studied. By a combinatorial algorithm which constructs and follows polyhedra sequences, the region of parametric stability is detected. In addition, we analyze the topological landscape of gene-environment networks in terms of structural stability. As a second strategy, we will review recent model selection and kernel learning methods for binary classification which can be used to classify microarray data for cancerous cells or for discrimination of other kind of diseases. This review is practically motivated and theoretically elaborated; it is devoted to a contribution to better health care, progress in medicine, a better education, and more healthy living conditions.

Guest editorial: model selection and optimization in machine learning

Machine learning methods largely benefit from optimization techniques in order to find an optimal model for future predictions and decisions. The interplay of machine learning and optimization methods is much like operations research (OR). Optimization, also called mathematical programming, is a subfield of OR. Both machine learning and OR are concerned with modeling of systems related to real-world problems. In machine learning (ML), the common practice is to use classical optimization techniques. However, due to massive and large-scale data sets faced in real world problems, optimization becomes a challenging task and traditional approaches cannot keep up with expectations. Accordingly, optimization methods adapted or integrated for machine learning tasks are needed to make ML more feasible for real world data sets. Another important challenging task is model selection. Because of the mathematical structure of the optimization model, there are parameters to be searched offline for the training data. Statistical model selection methods such as crossvalidation can be very time consuming when the size and the dimension of the training data is large. Thus, developing powerful model selection methods is an important factor for the feasibility of the algorithm solving the optimization problem. This special issue on “Model Selection and Optimization in ML” was inspired from the stream “Model Selection and Optimization Techniques in Machine Learning” organized by Kristiaan Pelckmans and Sureyya Ozogur-Akyuz at 23rd European Conference on Operational Research held in Bonn, Germany, July 5–8, 2009.

Pattern analysis for the prediction of fungal pro-peptide cleavage sites

Support vector machines (SVMs) have many applications in investigating biological data from gene expression arrays to understanding EEG signals of sleep stages. In this paper, we have developed an application that will support the prediction of the pro-peptide cleavage site of fungal extracellular proteins which display mostly a monobasic or dibasic processing site. Many of the secretory proteins and peptides are synthesized as inactive precursors and they become active after post-translational processing. A collection of fungal pro-protein sequences are used as a training data set. A specifically designed kernel is expressed as an application of the well-known Gaussian kernel via feature spaces defined for our problem. Rather than fixing the kernel parameters with cross validation or other methods, we introduce a novel approach that simultaneously performs model selection together with the test of accuracy and testing confidence levels. This leads us to higher accuracy at significantly reduced training times. The results of the server ProP1.0 which predicts pro-peptide cleavage sites are compared with the results of this study. A similar mathematical approach may be adapted to pro-peptide cleavage prediction in other eukaryotes.

MODELLING OF KERNEL MACHINES BY INFINITE AND SEMI‐INFINITE PROGRAMMING

In Machine Learning (ML) algorithms, one of the crucial issues is the representation of the data. As the data become heterogeneous and large‐scale, single kernel methods become insufficient to classify nonlinear data. The finite combinations of kernels are limited up to a finite choice. In order to overcome this discrepancy, we propose a novel method of “infinite” kernel combinations for learning problems with the help of infinite and semi‐infinite programming regarding all elements in kernel space. Looking at all infinitesimally fine convex combinations of the kernels from the infinite kernel set, the margin is maximized subject to an infinite number of constraints with a compact index set and an additional (Riemann‐Stieltjes) integral constraint due to the combinations. After a parametrization in the space of probability measures, it becomes semi‐infinite. We analyze the regularity conditions which satisfy the Reduction Ansatz and discuss the type of distribution functions within the structure of the co...

Prediction with the SVM Using Test Point Margins.

Support vector machines (SVMs) carry out binary classification by constructing a maximal margin hyperplane between the two classes of observed (training) examples and then classifying test points according to the half-spaces in which they reside (irrespective of the distances that may exist between the test examples and the hyperplane). Cross-validation involves finding the one SVM model together with its optimal parameters that minimizes the training error and has good generalization in the future. In contrast, in this chapter we collect all of the models found in the model selection phase and make predictions according to the model whose hyperplane achieves the maximum separation from a test point. This directly corresponds to the L ∞ norm for choosing SVM models at the testing stage. Furthermore, we also investigate other more general techniques corresponding to different L p norms and show how these methods allow us to avoid the complex and timeconsuming paradigm of cross-validation. Experimental results demonstrate this advantage, showing significant decreases in computational time as well as competitive generalization error.

Automated X-Ray Image Annotation

Advances in the medical imaging technology has lead to an exponential growth in the number of digital images that needs to be acquired, analyzed, classified, stored and retrieved in medical centers. As a result, medical image classification and retrieval has recently gained high interest in the scientific community. Despite several attempts, such as the yearly-held ImageCLEF Medical Image Annotation Challenge, the proposed solutions are still far from being sufficiently accurate for real-life implementations.