Tilman Lange

Stability-based validation of clustering solutions

Data clustering describes a set of frequently employed techniques in exploratory data analysis to extract natural group structure in data. Such groupings need to be validated to separate the signal in the data from spurious structure. In this context, finding an appropriate number of clusters is a particularly important model selection question. We introduce a measure of cluster stability to assess the validity of a cluster model. This stability measure quantifies the reproducibility of clustering solutions on a second sample, and it can be interpreted as a classification risk with regard to class labels produced by a clustering algorithm. The preferred number of clusters is determined by minimizing this classification risk as a function of the number of clusters. Convincing results are achieved on simulated as well as gene expression data sets. Comparisons to other methods demonstrate the competitive performance of our method and its suitability as a general validation tool for clustering solutions in real-world problems.

A Resampling Approach to Cluster Validation

The concept of cluster stability is introduced as a means for assessing the validity of data partitionings found by clustering algorithms. It allows us to explicitly quantify the quality of a clustering solution, without being dependent on external information. The principle of maximizing the cluster stability can be interpreted as choosing the most self-consistent data partitioning. We present an empirical estimator for the theoretically derived stability index, based on imitating independent sample-sets by way of resampling. Experiments on both toy-examples and real-world problems effectively demonstrate that the proposed validation principle is highly suited for model selection.

Learning with constrained and unlabelled data

Classification problems abundantly arise in many computer vision tasks eing of supervised, semi-supervised or unsupervised nature. Even when class labels are not available, a user still might favor certain grouping solutions over others. This bias can be expressed either by providing a clustering criterion or cost function and, in addition to that, by specifying pairwise constraints on the assignment of objects to classes. In this work, we discuss a unifying formulation for labelled and unlabelled data that can incorporate constrained data for model fitting. Our approach models the constraint information by the maximum entropy principle. This modeling strategy allows us (i) to handle constraint violations and soft constraints, and, at the same time, (ii) to speed up the optimization process. Experimental results on face classification and image segmentation indicates that the proposed algorithm is computationally efficient and generates superior groupings when compared with alternative techniques.

An Approach for the Classification of Urban Building Structures Based on Discriminant Analysis Techniques

Recognition of urban structures is of interest in cartography and urban modelling. While a broad range of typologies of urban patterns have been published in the last century, relatively little research on the automated recognition of such structures exists. This work presents a sample-based approach for the recognition of five types of urban structures: (1) inner city areas, (2) industrial and commercial areas, (3) urban areas, (4) suburban areas and (5) rural areas. The classification approach is based only on the characterisation of building geometries with morphological measures derived from perceptual principles of Gestalt psychology. Thereby, size, shape and density of buildings are evaluated. After defining the research questions we develop the classification methodology and evaluate the approach with respect to several aspects. The experiments focus on the impact of different classification algorithms, correlations and contributions of measures, parameterisation of buffer-based indices, and mode filtering. In addition to that, we investigate the influence of scale and regional factors. The results show that the chosen approach is generally successful. It turns out that scale, algorithm parameterisation, and regional heterogeneity of building structures substantially influence the classification performance.

Model Order Selection and Cue Combination for Image Segmentation

Model order selection and cue combination are both difficult open problems in the area of clustering. In this work we build upon stability-based approaches to develop a new method for automatic model order selection and cue combination with applications to visual grouping. Novel features of our approach include the ability to detect multiple stable clusterings (instead of only one), a simpler means of calculating stability that does not require training a classifier, and a new characterization of the space of stabilities for a continuum of segmentations that provides for an efficient sampling scheme. Our contribution is a framework for visual grouping that frees the user from the hassles of parameter tuning and model order selection: the input is an image, the output is a shortlist of segmentations.

Bayesian class discovery in microarray datasets

A novel approach to class discovery in gene expression datasets is presented. In the context of clinical diagnosis, the central goal of class discovery algorithms is to simultaneously find putative (sub-)types of diseases and to identify informative subsets of genes with disease-type specific expression profile. Contrary to many other approaches in the literature, the method presented implements a wrapper strategy for feature selection, in the sense that the features are directly selected by optimizing the discriminative power of the used partitioning algorithm. The usual combinatorial problems associated with wrapper approaches are overcome by a Bayesian inference mechanism. On the technical side, we present an efficient optimization algorithm with guaranteed local convergence property. The only free parameter of the optimization method is selected by a resampling-based stability analysis. Experiments with Leukemia and Lymphoma datasets demonstrate that our method is able to correctly infer partitions and corresponding subsets of genes which both are relevant in a biological sense. Moreover, the frequently observed problem of ambiguities caused by different but equally high-scoring partitions is successfully overcome by the model selection method proposed.

Combining partitions by probabilistic label aggregation

Data clustering represents an important tool in exploratory data analysis. The lack of objective criteria render model selection as well as the identification of robust solutions particularly difficult. The use of a stability assessment and the combination of multiple clustering solutions represents an important ingredient to achieve the goal of finding useful partitions. In this work, we propose a novel way of combining multiple clustering solutions for both, hard and soft partitions: the approach is based on modeling the probability that two objects are grouped together. An efficient EM optimization strategy is employed in order to estimate the model parameters. Our proposal can also be extended in order to emphasize the signal more strongly by weighting individual base clustering solutions according to their consistency with the prediction for previously unseen objects. In addition to that, the probabilistic model supports an out-of-sample extension that (i) makes it possible to assign previously unseen objects to classes of the combined solution and (ii) renders the efficient aggregation of solutions possible. In this work, we also shed some light on the usefulness of such combination approaches. In the experimental result section, we demonstrate the competitive performance of our proposal in comparison with other recently proposed methods for combining multiple classifications of a finite data set.

Image Segmentation by Networks of Spiking Neurons

A network of leaky integrate-and-fire (IAF) neurons is proposed to segment gray-scale images. The network architecture with local competition between neurons that encode segment assignments of image blocks is motivated by a histogram clustering approach to image segmentation. Lateral excitatory connections between neighboring image sites yield a local smoothing of segments. The mean firing rate of class membership neurons encodes the image segmentation. A weight modification scheme is proposed that estimates segment-specific prototypical histograms. The robustness properties of the network implementation make it amenable to an analog VLSI realization. Results on synthetic and real-world images demonstrate the effectiveness of the architecture.

Adaptive Feature Selection in Image Segmentation

Most image segmentation algorithms optimize some mathematical similarity criterion derived from several low-level image features. One possible way of combining different types of features, e.g. color- and texture features on different scales and/or different orientations, is to simply stack all the individual measurements into one high-dimensional feature vector. Due to the nature of such stacked vectors, however, only very few components (e.g. those which are defined on a suitable scale) will carry information that is relevant for the actual segmentation task. We present an approach to combining segmentation and adaptive feature selection that overcomes this relevance determination problem. All free model parameters of this method are selected by a resampling-based stability analysis. Experiments demonstrate that the built-in feature selection mechanism leads to stable and meaningful partitions of the images.

Weakly Supervised Cell Nuclei Detection and Segmentation on Tissue Microarrays of Renal Clear Cell Carcinoma

Renal cell carcinoma (RCC) is one of the ten most frequent malignancies in Western societies and can be diagnosed by histological tissue analysis. Current diagnostic rules rely on exact counts of cancerous cell nuclei which are manually counted by pathologists. We propose a complete imaging pipeline for the automated analysis of tissue microarrays of renal cell cancer. At its core, the analysis system consists of a novel weakly supervised classification method, which is based on an iterative morphological algorithm and a soft-margin support vector machine. The lack of objective ground truth labels to validate the system requires the combination of expert knowledge of pathologists. Human expert annotations of more than 2000 cell nuclei from 9 different RCC patients are used to demonstrate the superior performance of the proposed algorithm over existing cell nuclei detection approaches.