Nathan S. Netanyahu

An efficient k-means clustering algorithm: analysis and implementation

In k-means clustering, we are given a set of n data points in d-dimensional space R/sup d/ and an integer k and the problem is to determine a set of k points in Rd, called centers, so as to minimize the mean squared distance from each data point to its nearest center. A popular heuristic for k-means clustering is Lloyds (1982) algorithm. We present a simple and efficient implementation of Lloyds k-means clustering algorithm, which we call the filtering algorithm. This algorithm is easy to implement, requiring a kd-tree as the only major data structure. We establish the practical efficiency of the filtering algorithm in two ways. First, we present a data-sensitive analysis of the algorithms running time, which shows that the algorithm runs faster as the separation between clusters increases. Second, we present a number of empirical studies both on synthetically generated data and on real data sets from applications in color quantization, data compression, and image segmentation.

A local search approximation algorithm for k-means clustering

In k-means clustering we are given a set of n data points in d-dimensional space Rd and an integer k, and the problem is to determine a set of k points in ÓC;d, called centers, to minimize the mean squared distance from each data point to its nearest center. No exact polynomial-time algorithms are known for this problem. Although asymptotically efficient approximation algorithms exist, these algorithms are not practical due to the extremely high constant factors involved. There are many heuristics that are used in practice, but we know of no bounds on their performance.We consider the question of whether there exists a simple and practical approximation algorithm for k-means clustering. We present a local improvement heuristic based on swapping centers in and out. We prove that this yields a (9+ε)-approximation algorithm. We show that the approximation factor is almost tight, by giving an example for which the algorithm achieves an approximation factor of (9-ε). To establish the practical value of the heuristic, we present an empirical study that shows that, when combined with Lloyds algorithm, this heuristic performs quite well in practice.

The analysis of a simple k -means clustering algorithm

Abstract : K-means clustering is a very popular clustering technique which is used in numerous applications. Given a set of n data points in R(exp d) and an integer k, the problem is to determine a set of k points R(exp d), called centers, so as to minimize the mean squared distance from each data point to its nearest center. A popular heuristic for k-means clustering is Lloyds algorithm. In this paper, we present a simple and efficient implementation of Lloyds k-means clustering algorithm, which we call the filtering algorithm. This algorithm is very easy to implement. It differs from most other approaches in that it precomputes a kd-tree data structure for the data points rather than the center points. We establish the practical efficiency of the filtering algorithm in two ways. First, we present a data-sensitive analysis of the algorithms running time. Second, we have implemented the algorithm and performed a number of empirical studies, both on synthetically generated data and on real data from applications in color quantization, compression, and segmentation.

Efficient algorithms for robust feature matching

Abstract One of the basic building blocks in any point-based registration scheme involves matching feature points that are extracted from a sensed image to their counterparts in a reference image. This leads to the fundamental problem of point matching: Given two sets of points, find the (affine) transformation that transforms one point set so that its distance from the other point set is minimized. Because of measurement errors and the presence of outlying data points, it is important that the distance measure between the two point sets be robust to these effects. We measure distances using the partial Hausdorff distance. Point matching can be a computationally intensive task, and a number of theoretical and applied approaches have been proposed for solving this problem. In this paper, we present two algorithmic approaches to the point matching problem, in an attempt to reduce its computational complexity, while still providing a guarantee of the quality of the final match. Our first method is an approximation algorithm, which is loosely based on a branch-and-bound approach due to Huttenlocher and Rucklidge, (Technical Report 1321, Dept. of Computer Science, Cornell University, Ithaca, 1992; Proc. IEEE Conf. on Computer vision and Pattern Recognition, New York, 1993, pp. 705–706). We show that by varying the approximation error bounds, it is possible to achieve a tradeoff between the quality of the match and the running time of the algorithm. Our second method involves a Monte Carlo method for accelerating the search process used in the first algorithm. This algorithm operates within the framework of a branch-and-bound procedure, but employs point-to-point alignments to accelerate the search. We show that this combination retains many of the strengths of branch-and-bound search, but provides significantly faster search times by exploiting alignments. With high probability, this method succeeds in finding an approximately optimal match. We demonstrate the algorithms’ performances on both synthetically generated data points and actual satellite images.

A FAST IMPLEMENTATION OF THE ISODATA CLUSTERING ALGORITHM

Clustering is central to many image processing and remote sensing applications. ISODATA is one of the most popular and widely used clustering methods in geoscience applications, but it can run slowly, particularly with large data sets. We present a more efficient approach to ISODATA clustering, which achieves better running times by storing the points in a kd-tree and through a modification of the way in which the algorithm estimates the dispersion of each cluster. We also present an approximate version of the algorithm which allows the user to further improve the running time, at the expense of lower fidelity in computing the nearest cluster center to each point. We provide both theoretical and empirical justification that our modified approach produces clusterings that are very similar to those produced by the standard ISODATA approach. We also provide empirical studies on both synthetic data and remotely sensed Landsat and MODIS images that show that our approach has significantly lower running times.

Image Registration for Remote Sensing

Foreword Jon A. Benediktsson Part I. The Importance of Image Registration for Remote Sensing: 1. Introduction Jacqueline Le Moigne, Nathan S. Netanyahu and Roger D. Eastman 2. Influence of image registration on validation efforts Bin Tan and Curtis E. Woodcock 3. Survey of image registration methods Roger D. Eastman, Nathan S. Netanyahu and Jacqueline Le Moigne Part II. Similarity Metrics for Image Registration: 4. Fast correlation and phase correlation Harold S. Stone 5. Matched filtering techniques Qin-Sheng Chen 6. Image registration using mutual information Arlene A. Cole-Rhodes and Pramod K. Varshney Part III. Feature Matching and Strategies for Image Registration: 7. Registration of multiview images A. Ardeshir Goshtasby 8. New approaches to robust, point-based image registration David M. Mount, Nathan S. Netanyahu and San Ratanasanya 9. Condition theory for image registration and post-registration error estimation Charles S. Kenney, B. S. Manjunath, Marco Zuliani and Kaushal Solanki 10. Feature-based image to image registration Venu M. Govindu and Rama Chellappa 11. On the use of wavelets for image registration Jacqueline Le Moigne, Ilya Zavorin and Harold S. Stone 12. Gradient descent approaches to image registration Arlene A. Cole-Rhodes and Roger D. Eastman 13. Bounding the performance of image registration Min Xu and Parmod K. Varshney Part IV. Applications and Operational Systems: 14. Multi-temporal and multi-sensor image registration Jacqueline Le Moigne, Arlene A. Cole-Rhodes, Roger D. Eastman, Nathan S. Netanyahu, Harold S. Stone, Ilya Zavorin and Jeffrey T. Morisette 15. Georegistration of meteorological images James L. Carr 16. Challenges, solutions, and applications of accurate multi-angle image registration: lessons learned from MISR Veljko M. Jovanovic, David J. Diner and Roger Davies 17. Automated AVHRR image navigation William J. Emery, R. Ian Crocker and Daniel G. Baldwin 18. Landsat image geocorrection and registration James C. Storey 19. Automatic and precise orthorectification of SPOT images Simon Baillarin, Aurelie Bouillon and Marc Bernard 20. Geometry of the VEGETATION sensor Sylvia Sylvander 21. Accurate MODIS global geolocation through automated ground control image matching Robert E. Wolfe and Masahiro Nishihama 22. SeaWIFS operational geolocation assessment system Frederick S. Patt Part V. Conclusion: 23. Concluding remarks Jacqueline Le Moigne, Nathan S. Netanyahu and Roger D. Eastman Glossary Index.

A RANDOMIZED ALGORITHM FOR SLOPE SELECTION

A set of n distinct points in the plane defines lines by joining each pair of distinct points. The median slope of these O(n2) lines was proposed by Theil as a robust estimator for the slope of the line of best fit for the points. We present a randomized algorithm for selecting the k-th smallest slope of such a set of lines which runs in expected O(n log n) time. An efficient implementation of the algorithm and practical experience with the algorithm are discussed.

Improved algorithms for robust point pattern matching and applications to image registration

Given two images of roughly the same scene, image registration is the process of determining the transformation that most nearly maps one image to another. This problem is of particular interest in remote sensing applications, where it is known that two images correspond to roughly the same gecgraphic region, but the exact alignment between the images io not known. There are many approaches to image registration. We will consider an approach based on extracting a Ret of point features from each of the two images, and thus reducing the problem to a point pattern matching problem. Because of measurement errors and the presence of outlying data points in either of the images, it is important that the diotance measure between two point sets be robust to theeo cffecto. We will measure distances using the partial Hauodorff distance, An important element of image registration applications is that the search begins with a priori information on the bounds of transformation, and a good algorithm should be able to take advantage of this information. Point matching can be a computationally intensive task, and there have been a number of algorithms and approaches proposed for solving this problem, both from theoretical and applied standpoints. One common approach is based on a *Dopartmont of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park, MaryInnd. Email: mountQce.umd.edu. The support of the Nationnl Science Foundation under grant CCR-9712379 is gratefully acknowledged. tCentcr for Automation Research, University of Maryland, Collego Park, and Center of Excellence in Space Data and Information Sciences (CESDIS), Code 930.5, Space Data and Computing Division, NASA Goddard Space Flight Center, Greenbolt, Mnr.vlnnd. EmaiL nathanocf ar .umd. edu. The SUP port of tha Applied Information Sciences Branch (AISB), Code 935, NASAIGSFC, under contract NAS 5555-37 is greatfully acknowledged. SUniversities Space Research Association/CESDIS, Code 030,5, Space Data and Computing Division, NASA/GSFC, Grconbolt, Maryland. Email: lemoigneQcesdis.gsfc.nasa.gov. The nupport of the AISB, Code 935, NASA/GSFC is greatfully acknowledged, jkr&sjon tomnko dj&l orhardcopies ofnjl orpartoftiworkfm pcmond or clormom we in grated without fee qrovided that copies rue not mndc or &ributcd for profit or comn~ercld advantage and that copjm beor Ihin notiw and the full citation on the f~page. Tf copy odwwi~rice, to rcpubliQ to post on servers or to redlstnbue to W require3 prior rpeoitio permission Mdlor a fffi. SCCJ 98 Minneapolis M~IUV.XO~~ USA Copyri&t /KM 1998 O-89791-9734/98/ 6...

A nonparametric method for fitting a straight line to a noisy image

5.00 geometric branch-and-bound search of transformation space and another is based on using point alignments to derive the matching transformation. The former has the advantage that it can provide guarantees on the accuracy of the final match, and that it naturally uses any a priori information to bound the search. The latter has the advantage of simplicity and speed. We introduce a new practical approximation algorithm, which we call bounded alignment, for robust point pattern matching. This algorithm is a novel combination of these two approaches. Our algorithm operates within the framework of the branch-and-bound, but employs point-to-point alignments to accelerate the search. We show that this combination retains many of the strengths of branch-and-bound search, but provides significantly faster search times by esplaiting alignments. We have implemented the algorithm and have demonstrated its performance on both synthetically generated data points and actual satellite images.

First evaluation of automatic image registration methods

In fitting a straight line to a noisy image, the least-squares method becomes highly unreliable either when the noise distribution is nonnormal or when it is contaminated by outliers. The authors propose a nonparametric method, the median of the intercepts, to overcome these difficulties. This method is free of assumptions about the noise distribution and insensitive to outliers, and it does not require quantization of the parameter space. Thus, unlike the Hough transform, its outcome does not depend on the bin size. The method is efficient and its implementation does not involve practical difficulties such as local minima or poor convergence of iterative procedures. >