Hans-Hermann Bock

Two-mode clustering methods: a structured overview.

In this paper we present a structured overview of methods for two-mode clustering, that is, methods that provide a simultaneous clustering of the rows and columns of a rectangular data matrix. Key structuring principles include the nature of row, column and data clusters and the type of model structure or associated loss function. We illustrate with analyses of symptom data on archetypal psychiatric patients.

On some significance tests in cluster analysis

We investigate the properties of several significance tests for distinguishing between the hypothesisH of a “homogeneous” population and an alternativeA involving “clustering” or “heterogeneity,” with emphasis on the case of multidimensional observationsx 1, ...,x n eℝ p . Four types of test statistics are considered: the (s-th) largest gap between observations, their mean distance (or similarity), the minimum within-cluster sum of squares resulting from a k-means algorithm, and the resulting maximum F statistic. The asymptotic distributions underH are given forn→∞ and the asymptotic power of the tests is derived for neighboring alternatives.

Probabilistic models in cluster analysis

Abstract This paper discusses cluster analysis in a probabilistic and inferential framework as opposed to more exploratory, heuristic or algorithmic approaches. It presents a broad survey on probabilistic models for partition-type, hierarchical and tree-like clustering structures and points to the relevant literature. It is shown how suitable clustering criteria or grouping methods may be derived from these models in the case of vector-valued data, dissimilarity matrices and similarity relations. In particular, we discuss hypothesis testing for homogeneity or for a grouping structure, the asymptotic distribution of test statistics, the use of random graph theory and combinatorial methods for simulating random dendrograms. Our presentation of hierarchies includes, e.g., Markovian branching processes and phylogenetic inference based on molecular sequence data.

Clustering Methods: A History of k -Means Algorithms

This paper surveys some historical issues related to the well-known k-means algorithm in cluster analysis. It shows to which authors the different versions of this algorithm can be traced back, and which were the underlying applications. We sketch various generalizations (with references also to Diday’s work) and thereby underline the usefulness of the k-means approach in data analysis.

Probabilistic Aspects in Cluster Analysis

Cluster analysis provides methods and algorithms for partitioning a set of objects O = 1,…, n (or data vectors x1,…, xn ∈ R p ) into a suitable number of classes C1,…,Cm ⊆ O such that these classes are homogeneous and each of them comprizes only objects which are’similar’ in some sense. The historical evolution shows a surprising trend from an algorithmic, heuristic and applications oriented point of view (Sokal/Sneath 1963) to a more basic, theory oriented investigation of the structural, mathematical and statistical properties of clustering methods. Nowadays, the questions to be answered are of the type’How many clusters are there ?’,’Is there a classification structure ?’,’Is the calculated classification adequate ?’,’Which are the strongest clusters ?’ etc.

On the Interface between Cluster Analysis, Principal Component Analysis, and Multidimensional Scaling

This paper shows how methods of cluster analysis, principal component analysis, and multidimensional scaling may be combined in order to obtain an optimal fit between a classification underlying some set of objects 1,…,n and its visual representation in a low-dimensional euclidean space ℝs. We propose several clustering criteria and corresponding k-means-like algorithms which are based either on a probabilistic model or on geometrical considerations leading to matrix approximation problems. In particular, a MDS-clustering strategy is presented for-displaying not only the n objects using their pairwise dissimilarities, but also the detected clusters and their average distances.

Clustering and Neural Networks

This paper considers the usage of neural networks for the construction of clusters and classifications from given data and discusses, conversely, the use of clustering methods in neural network algorithms. We survey related work in the fields of k-means clustering, stochastic approximation, Kohonen maps, Hopfield networks and multi-layer perceptrons. We propose various new approaches, reveal the asymptotic behaviour of Kohonen maps, and point to possible extensions.

Classification and Clustering: Problems for the Future

This paper reviews various basic achievements in classification during the last fifteen years and points to a series of unsolved mathematical, statistical and applied problems. It suggests the investigation of new methodological aspects, a better adaptation between methods and applications, the extension of cluster and data analysis into fields like information processing, machine learning and artificial intelligence, and a formal investigation of information retrieval problems in the clustering and database framework. Furthermore, we comment on computational aspects and software tools required for future applications.

Simultaneous Visualization and Clustering Methods as an Alternative to Kohonen Maps

Kohonen maps are often used for visualizing high-dimensional feature vectors in lowdimensional space. This approach is often recommended for supporting the clustering of data. In this paper an alternative approach is proposed which is more in the lines of multivariate statistics and provides a simultaneous visualization and clustering of data. This approach combines projection and embedding methods (such as principal components or multidimensional scaling) with clustering criteria and corresponding optimization algorithms. Four distinct methods are proposed: projection pursuit clustering for quantitative data vectors, two MDS clustering methods for dissimilarity data (either with or without a representation of classes) and a group difference scaling method (known from. literature).

Clustering and Neural Network Approaches

This paper describes how clustering problems can be resolved by neural network (NN) approaches such as Hopfield nets, multi-layer perceptrons, and Kohonen’s ’self-organizing maps’ (SOMs). We emphasize the close relationship between the NN approach and classical clustering methods. In particular, we show how SOMs are derived by stochastic approximation from a new continuous version (K-criterion) of a finite-sample clustering criterion proposed by Anouar et al. (1997). In this framework we determine the asymptotic behaviour of Kohonen’s method, design a new finite-sample version of the SOM approach of the k-means type, and propose various generalizations along the lines of classical ’regression clustering’, ’principal component clustering’, and ’maximum-likelihood clustering’.