Angela Y. Wu

An efficient k-means clustering algorithm: analysis and implementation

In k-means clustering, we are given a set of n data points in d-dimensional space R/sup d/ and an integer k and the problem is to determine a set of k points in Rd, called centers, so as to minimize the mean squared distance from each data point to its nearest center. A popular heuristic for k-means clustering is Lloyds (1982) algorithm. We present a simple and efficient implementation of Lloyds k-means clustering algorithm, which we call the filtering algorithm. This algorithm is easy to implement, requiring a kd-tree as the only major data structure. We establish the practical efficiency of the filtering algorithm in two ways. First, we present a data-sensitive analysis of the algorithms running time, which shows that the algorithm runs faster as the separation between clusters increases. Second, we present a number of empirical studies both on synthetically generated data and on real data sets from applications in color quantization, data compression, and image segmentation.

An optimal algorithm for approximate nearest neighbor searching fixed dimensions

Consider a set of <italic>S</italic> of <italic>n</italic> data points in real <italic>d</italic>-dimensional space, R<supscrpt>d</supscrpt>, where distances are measured using any Minkowski metric. In nearest neighbor searching, we preprocess <italic>S</italic> into a data structure, so that given any query point <italic>q</italic><inline-equation> <f>∈</f></inline-equation> R<supscrpt>d</supscrpt>, is the closest point of S to <italic>q</italic> can be reported quickly. Given any positive real ε, data point <italic>p</italic> is a (1 +ε)-<italic>approximate nearest neighbor</italic> of <italic>q</italic> if its distance from <italic>q</italic> is within a factor of (1 + ε) of the distance to the true nearest neighbor. We show that it is possible to preprocess a set of <italic>n</italic> points in R<supscrpt>d</supscrpt> in <italic>O(dn</italic> log <italic>n</italic>) time and <italic>O(dn)</italic> space, so that given a query point <italic> q</italic> <inline-equation> <f>∈</f></inline-equation> R<supscrpt>d</supscrpt>, and ε > 0, a (1 + ε)-approximate nearest neighbor of <italic>q</italic> can be computed in <italic>O</italic>(<italic>c</italic><subscrpt><italic>d</italic>, ε</subscrpt> log <italic>n</italic>) time, where <italic>c<subscrpt>d,ε</subscrpt></italic>≤<italic>d</italic> <inline-equation> <f><fen lp=ceil>1 + 6d/<g>e</g><rp post=ceil></fen></f></inline-equation>;<supscrpt>d</supscrpt> is a factor depending only on dimension and ε. In general, we show that given an integer <italic>k</italic> ≥ 1, (1 + ε)-approximations to the <italic>k</italic> nearest neighbors of <italic>q</italic> can be computed in additional <italic>O(kd</italic> log <italic>n</italic>) time.

A local search approximation algorithm for k-means clustering

In k-means clustering we are given a set of n data points in d-dimensional space Rd and an integer k, and the problem is to determine a set of k points in ÓC;d, called centers, to minimize the mean squared distance from each data point to its nearest center. No exact polynomial-time algorithms are known for this problem. Although asymptotically efficient approximation algorithms exist, these algorithms are not practical due to the extremely high constant factors involved. There are many heuristics that are used in practice, but we know of no bounds on their performance.We consider the question of whether there exists a simple and practical approximation algorithm for k-means clustering. We present a local improvement heuristic based on swapping centers in and out. We prove that this yields a (9+ε)-approximation algorithm. We show that the approximation factor is almost tight, by giving an example for which the algorithm achieves an approximation factor of (9-ε). To establish the practical value of the heuristic, we present an empirical study that shows that, when combined with Lloyds algorithm, this heuristic performs quite well in practice.

Embedding of tree networks into hypercubes

Abstract The hypercube is a good host graph for the embedding of networks of processors because of its low degree and low diameter. Graphs such as trees and arrays can be embedded into a hypercube with small dilation and expansion costs, but there are classes of graphs which can be embedded into a hypercube only with large expansion cost or large dilation cost.

The analysis of a simple k -means clustering algorithm

Abstract : K-means clustering is a very popular clustering technique which is used in numerous applications. Given a set of n data points in R(exp d) and an integer k, the problem is to determine a set of k points R(exp d), called centers, so as to minimize the mean squared distance from each data point to its nearest center. A popular heuristic for k-means clustering is Lloyds algorithm. In this paper, we present a simple and efficient implementation of Lloyds k-means clustering algorithm, which we call the filtering algorithm. This algorithm is very easy to implement. It differs from most other approaches in that it precomputes a kd-tree data structure for the data points rather than the center points. We establish the practical efficiency of the filtering algorithm in two ways. First, we present a data-sensitive analysis of the algorithms running time. Second, we have implemented the algorithm and performed a number of empirical studies, both on synthetically generated data and on real data from applications in color quantization, compression, and segmentation.

On the Area of Overlap of Translated Polygons

Given two simple polygonsPandQin the plane and a translation vectort?R2, thearea-of-overlapfunction ofPandQis the function Ar(t) = Area(P? (t+Q)), wheret+QdenotesQtranslated byt. This function has a number of applications in areas such as motion planning and object recognition. We present a number of mathematical results regarding this function. We also provide efficient algorithms for computing a representation of this function and for tracing contour curves of constant area-of-overlap.

Computation of geometric properties from the medial axis transform in O ( n log n ) time

Abstract The digital medial axis transform (MAT) represents an image subset S as the union of maximal upright squares contained in S . Brute-force algorithms for computing geometric properties of S from its MAT require time O(n 2 ) , where n is the number of squares. Over the past few years, however, algorithms have been developed that compute properties for a union of upright rectangles in time O(n log n) , which makes the use of the MAT much more attractive. We review these algorithms and also present efficient algorithms for computing union-of-rectangle representations of derived sets (union, intersection, complement) and for conversion between the union of rectangles and other representations of a subset.

Parallel Image Processing

This chapter reviews basic work on parallel image processing and analysis, with emphasis on work done at the Computer Vision Laboratory at the University of Maryland. It describes parallel computers suitable for image processing tasks, including meshes, pyramids, and hypercubes, and discusses parallel algorithms for pixel-level and region-level processing.

Parallel processing of regions represented by linear quadtrees

We show how computation of geometric properties of a region represented by a linear quadtree can be speeded up by about a factor of p by using a p -processor CREW PRAM model of parallel computation. Similar speedups are obtained for computing the union and intersection of two regions, and the complement of a region, using linear quadtree representations.

On the Least Trimmed Squares Estimator

The linear least trimmed squares (LTS) estimator is a statistical technique for fitting a linear model to a set of points. Given a set of n points in ℝd and given an integer trimming parameter h≤n, LTS involves computing the (d−1)-dimensional hyperplane that minimizes the sum of the smallest h squared residuals. LTS is a robust estimator with a 50xa0%-breakdown point, which means that the estimator is insensitive to corruption due to outliers, provided that the outliers constitute less than 50xa0% of the set. LTS is closely related to the well known LMS estimator, in which the objective is to minimize the median squared residual, and LTA, in which the objective is to minimize the sum of the smallest 50xa0% absolute residuals. LTS has the advantage of being statistically more efficient than LMS. Unfortunately, the computational complexity of LTS is less understood than LMS. In this paper we present new algorithms, both exact and approximate, for computing the LTS estimator. We also present hardness results for exact and approximate LTS. Axa0number of our results apply to the LTA estimator as well.