James C. Bezdek

Efficient Implementation of the Fuzzy c-Means Clustering Algorithms

This paper reports the results of a numerical comparison of two versions of the fuzzy c-means (FCM) clustering algorithms. In particular, we propose and exemplify an approximate fuzzy c-means (AFCM) implementation based upon replacing the necessary ``exact variates in the FCM equation with integer-valued or real-valued estimates. This approximation enables AFCM to exploit a lookup table approach for computing Euclidean distances and for exponentiation. The net effect of the proposed implementation is that CPU time during each iteration is reduced to approximately one sixth of the time required for a literal implementation of the algorithm, while apparently preserving the overall quality of terminal clusters produced. The two implementations are tested numerically on a nine-band digital image, and a pseudocode subroutine is given for the convenience of applications-oriented readers. Our results suggest that AFCM may be used to accelerate FCM processing whenever the feature space is comprised of tuples having a finite number of integer-valued coordinates.

Segmentation of a Thematic Mapper Image Using the Fuzzy c-Means Clusterng Algorthm

In this paper, a segmentation procedure that utilizes a clustering algorithm based upon fuzzy set theory is developed. The procedure operates in a nonparametric unsupervised mode. The feasibility of the methodology is demonstrated by segmenting a six-band Landsat-4 digital image with 324 scan lines and 392 pixels per scan line. For this image, 100-percent ground cover information is available for estimating the quality of segmentation. About 80 percent of the imaged area contains corn and soybean fields near the peak of their growing season. The remaining 20 percent of the image contains 12 different types of ground cover classes that appear in regions of diffferent sizes and shapes. The segmentation method uses the fuzzy c-means algorithm in two stages. The large number of clusters resulting from this segmentation process are then merged by use of a similarity measure on the cluster centers. Results are presented to show that this two-stage process leads to separation of corn and soybean, and of several minor classes that would otherwise be overwhelmed in any practical one-stage clustering.

Generalized k -nearest neighbor rules

Abstract This paper discusses a suitable framework for generalizing the k-nearest neighbor (k-NNR) algorithms to cases where the design labels are not necessarily crisp, i.e., not binary-valued. The proposed framework imbeds all crisp k-NNRs into a larger structure of fuzzy k-NNRs. The resultant model enables neighborhood voting to be a continuous function of local labels at a point to be classified. We emphasize that the decision itself may be crisp even when a fuzzy k-NNR is utilized. The usefulness of this extension of the conventional technique is illustrated by comparing the observed error rates of four classifiers (the hard k-NNR, two fuzzy k-NNRs, and a fuzzy 1-nearest prototype rule (1-NPR) on three data sets: Andersons Iris data, and samples from (synthetic) univariate and bivariate normal mixtures. Our conclusions: all four designs yield comparable (usually within 4%) error rates; the Fuzzy c-Means (FCM) based k-NNR is usually the best design; the FCM/1-NPR is the most efficient and perhaps most useful of the four designs; and finally, that generalized NNRs are an important and useful extension of the conventional ones.

Curvature and Tangential Deflection of Discrete Arcs: A Theory Based on the Commutator of Scatter Matrix Pairs and Its Application to Vertex Detection in Planar Shape Data

This paper introduces a new theory for the tangential deflection and curvature of plane discrete curves. Our theory applies to discrete data in either rectangular boundary coordinate or chain coded formats: its rationale is drawn from the statistical and geometric properties associated with the eigenvalue-eigenvector structure of sample covariance matrices. Specifically, we prove that the nonzero entry of the commutator of a piar of scatter matrices constructed from discrete arcs is related to the angle between their eigenspaces. And further, we show that this entry is-in certain limiting cases-also proportional to the analytical curvature of the plane curve from which the discrete data are drawn. These results lend a sound theoretical basis to the notions of discrete curvature and tangential deflection; and moreover, they provide a means for computationally efficient implementation of algorithms which use these ideas in various image processing contexts. As a concrete example, we develop the commutator vertex detection (CVD) algorithm, which identifies the location of vertices in shape data based on excessive cummulative tangential deflection; and we compare its performance to several well established corner detectors that utilize the alternative strategy of finding (approximate) curvature extrema.

Local convergence of the fuzzy c-Means algorithms☆

Abstract Much understanding has recently been gained concerning global convergence properties of the fuzzy c -Means (FCM) family of clustering algorithms. These global convergence properties, which hold for all iteration sequences, guarantee that every FCM iteration sequence converges, at least along a subsequence, to a stationary point of an FCM objective function. In this paper we prove a local convergence property, that is, a property pertaining to iteration sequences started near a solution. Specifically, a simple result is proved which shows that whenever an FCM algorithm is started sufficiently near a minimizer of the corresponding objective function, then the iteration sequence must converge to that particular minimizer. The result guarantees that once captured by the local neighborhood of a minimizer, the succeeding iterate sequence will not escape—thus, infinite oscillation of such a sequence cannot occur. The rate of convergence of the sequence to such a point is also discussed.

Knowledge-assisted document retrieval: I. The natural-language interface

Description du modele conceptuel et du traitement des recherches en langage naturel dans les systemes dinformation automatises. Une interface basee sur la technique des ensembles flous est proposee pour gerer les incertitudes liees a la semantique du langage naturel

Some Non-Standard Clustering Algorithms

This paper is a (non-exhaustive) survey of the theory of fuzzy relations and partitions as it has been applied to various clustering algorithms. More specifically, the structural models discussed will be object and relational criterion functions, convex decompositions, numerical transitive closures, and generalized k-nearest neighbor rules. We first discuss the role clustering plays in the development of pattern recognition systems, which generally involve feature analysis, clustering, and classifier design. Then selected clustering algorithms based on each of the above methodologies will be reviewed. Recent applications from various fields which use these algorithms are documented in the references.

Heuristics for intermediate level road finding algorithms

This paper deals with detection of road networks in aerial imagery. In natural images, roadlike segments extracted by low level operators are highly fragmented. The main focus of this work is the development of an intermediate processing stage that addresses the task of partitioning and connecting the roadlike fragments. Heuristics based upon generally observed properties of roadlike features are used to cluster and integrate segments that appear perceptually continuous. The segments are represented as lists and processed symbolically. The proposed methodology is exemplified using three data sets: a suite of four synthetic segment sets which were used to test the sensitivity of the algorithm to thresholds and noise; a circularly polarized millimeter wave (TABILS 5) radar pseudo image; and a thematic mapper (LANDSAT) image.

Transitive closures of fuzzy thesauri for information-retrieval systems

In this paper we represent a thesaurus (R) for an information system as the sum of two fuzzy relations,S(synonyms) and G(generalizations). The max-star completion of R is defined as R¯*, the max-star transitive closure of R. We interpret R¯*, which extends the concept-pair fuzzy relation R initially provided by an expert, as a linguistic completion of the thesaurus. Six max-star completions, corresponding to six well-known T- norms, are defined, analysed, and numerically illustrated on a nine-term dictionary. The application of our results in the context of document retrieval is this: one may use R¯* as a means of effecting replacements of terms appearing in a natural-language document request. The weights (R¯*)ij can be used to diminish or increase ones confidence in the degree of support being developed for each document considered relevant to a given query. The ijth element of R¯* can be regarded as the ultimate extent to which term j can be “reached” from term i; the values in R¯* thus represent degrees of confidence in max-star transitive chains.

Knowledge-assisted document retrieval: II. The retrieval process

This article presents our conceptual model of the retrieval process of a document-retrieval system. The retrieval mechanism input is an unambiguous intermediate form of a user query generated by the language processor using the method described previously. Our retrieval mechanism uses a two-step procedure. In the first step a list of documents pertinent to the query are obtained from the document database, and then an evidence-combination scheme is used to compute the degree of support between the query and individual documents. The second step uses a ranking procedure to obtain a final degree of support for each document chosen, as a function of individual degrees of support associated with one or more parts of the query. The end result is a set of document citations presented to the user in ranked order in response to the information request. Numerical examples are given to illustrate various facets of the overall system, which has been prototypically implemented in modular form to test system response to changes in model parameters.