Michael Kemmler

One-class classification with Gaussian processes

Detecting instances of unknown categories is an important task for a multitude of problems such as object recognition, event detection, and defect localization. This article investigates the use of Gaussian process (GP) priors for this area of research. Focusing on the task of one-class classification, we analyze different measures derived from GP regression and approximate GP classification. We also study important theoretical connections to other approaches and discuss their underlying assumptions. Experiments are performed using a large number of datasets and different image kernel functions. Our findings show that our approaches can outperform the well-known support vector data description approach indicating the high potential of Gaussian processes for one-class classification. Furthermore, we show the suitability of our methods in the area of attribute prediction, defect localization, bacteria recognition, and background subtraction. These applications and experiments highlight the easy applicability of our method as well as its state-of-the-art performance compared to established methods.

Kernel Null Space Methods for Novelty Detection

Detecting samples from previously unknown classes is a crucial task in object recognition, especially when dealing with real-world applications where the closed-world assumption does not hold. We present how to apply a null space method for novelty detection, which maps all training samples of one class to a single point. Beside the possibility of modeling a single class, we are able to treat multiple known classes jointly and to detect novelties for a set of classes with a single model. In contrast to modeling the support of each known class individually, our approach makes use of a projection in a joint subspace where training samples of all known classes have zero intra-class variance. This subspace is called the null space of the training data. To decide about novelty of a test sample, our null space approach allows for solely relying on a distance measure instead of performing density estimation directly. Therefore, we derive a simple yet powerful method for multi-class novelty detection, an important problem not studied sufficiently so far. Our novelty detection approach is assessed in comprehensive multi-class experiments using the publicly available datasets Caltech-256 and Image Net. The analysis reveals that our null space approach is perfectly suited for multi-class novelty detection since it outperforms all other methods.

One-class classification with gaussian processes

Detecting instances of unknown categories is an important task for a multitude of problems such as object recognition, event detection, and defect localization. This paper investigates the use of Gaussian process (GP) priors for this area of research. Focusing on the task of one-class classification for visual object recognition, we analyze different measures derived from GP regression and approximate GP classification. Experiments are performed using a large set of categories and different image kernel functions. Our findings show that the well-known Support Vector Data Description is significantly outperformed by at least two GP measures which indicates high potential of Gaussian processes for one-class classification.

Automatic identification of novel bacteria using Raman spectroscopy and Gaussian processes

Raman spectroscopy is successfully used for the reliable classification of complex biological samples. Much effort concentrates on the accurate prediction of known categories for highly relevant tasks in a wide area of applications such as cancer detection and bacteria recognition. However, the resulting recognition systems cannot always be directly used in practice since unseen samples might not belong to classes present in the training set. Our work aims to tackle this problem of novelty detection using a recently proposed approach based on Gaussian processes. By learning novelty scores for a large bacteria Raman dataset comprising 50 different strains, we analyze the behavior of this method on an independent dataset which includes known as well as unknown categories. Our experiment reveals that non-parametric methods such as Gaussian processes can be successfully applied to the task of finding unknown bacterial strains, leading to encouraging results motivating their further utilization in this area.

Large-scale gaussian process multi-class classification for semantic segmentation and facade recognition

This paper deals with the task of semantic segmentation, which aims to provide a complete description of an image by inferring a pixelwise labeling. While pixelwise classification is a suitable approach to achieve this goal, state-of-the-art kernel methods are generally not applicable since training and testing phase involve large amounts of data. We address this problem by presenting a method for large-scale inference with Gaussian processes. Standard limitations of Gaussian process classifiers in terms of speed and memory are overcome by pre-clustering the data using decision trees. This leads to a breakdown of the entire problem into several independent classification tasks whose complexity is controlled by the maximum number of training examples allowed in the tree leaves. We additionally propose a technique which allows for computing multi-class probabilities by incorporating uncertainties of the classifier estimates. The approach provides pixelwise semantics for a wide range of applications and different image types such as those from scene understanding, defect localization, and remote sensing. Our experiments are performed with a facade recognition application that shows the significant performance gain achieved by our method compared to previous approaches.

Efficient Gaussian process classification using random decision forests

Gaussian Processes are powerful tools in machine learning which offer wide applicability in regression and classification problems due to their non-parametric and non-linear behavior. However, one of their main drawbacks is the training time complexity which scales cubically with the number of samples. Our work addresses this issue by combining Gaussian Processes with Randomized Decision Forests to enable fast learning. An important advantage of our method is its simplicity and the ability to directly control the trade-off between classification performance and computation speed. Experiments on an indoor place recognition task show that our method can handle large training sets in reasonable time while retaining a good classification accuracy.

Global Context Extraction for Object Recognition Using a Combination of Range and Visual Features

It has been highlighted by many researchers, that the use of context information as an additional cue for high-level object recognition is important to close the gap between human and computer vision. We present an approach to context extraction in the form of global features for place recognition. Based on an uncalibrated combination of range data of a time-of-flight (ToF) camera and images obtained from a visual sensor, our system is able to classify the environment in predefined places (e.g. kitchen, corridor, office) by representing the sensor data with various global features. Besides state-of-the-art feature types, such as power spectrum models and Gabor filters, we introduce histograms of surface normals as a new representation of range images. An evaluation with different classifiers shows the potential of range data from a ToF camera as an additional cue for this task.

Large-scale Gaussian process classification using random decision forests

Gaussian processes are powerful modeling tools in machine learning which offer wide applicability for regression and classification tasks due to their non-parametric and non-linear behavior. However, one of their main drawbacks is the training time complexity which scales cubically with the number of examples. Our work addresses this issue by combining Gaussian processes with random decision forests to enable fast learning. An important advantage of our method is its simplicity and the ability to directly control the tradeoff between classification performance and computational speed. Experiments on an indoor place recognition task and on standard machine learning benchmarks show that our method can handle large training sets of up to three million examples in reasonable time while retaining good classification accuracy.

Classification of microorganisms via Raman spectroscopy using Gaussian processes

Automatic categorization of microorganisms is a complex task which requires advanced techniques to achieve accurate performance. In this paper, we aim at identifying microorganisms based on Raman spectroscopy. Empirical studies over the last years show that powerful machine learning methods such as Support Vector Machines (SVMs) are suitable for this task. Our work focuses on the Gaussian process (GP) classifier which is new to this field, provides fully probabilistic outputs and allows for efficient hyperparameter optimization. We also investigate the incorporation of prior knowledge regarding possible signal variations where known concepts from invariant kernel theory are transferred to the GP framework. In order to validate the suitability of the GP classifier, a comparison with state-of-the-art learners is conducted on a large-scale Raman spectra dataset, showing that the GP classifier significantly outperforms all other tested classifiers including SVM. Our results further show that incorporating prior knowledge leads to a significant performance gain when small amounts of training data are used.

Segmentation of microorganism in complex environments

In this paper, we tackle the problem of finding microorganisms in bright field microscopy images, which is an important and challenging step in various tasks, like classifying soil textures. Apart from bacteria or fungi, these images can contain impurities such as sand particles, which increase the difficulty of microbe detection. Following a semantic segmentation approach, where a label is inferred for each pixel, we achieve encouraging classification results on a database containing five different types of microbes. We review and evaluate multiple techniques including segment classification, conditional random field models, and level set approaches.