Katsuhiro Honda

Fuzzy clustering for categorical multivariate data

This paper proposes a new fuzzy clustering algorithm for categorical multivariate data. The conventional fuzzy clustering algorithms form fuzzy clusters so as to minimize the total distance from cluster centers to data points. However, they cannot be applied to the case where only cooccurrence relations among individuals and categories are given and the criterion to obtain clusters is not available. The proposed method enables us to handle that kind of data set by maximizing the degree of aggregation among clusters. The clustering results by the proposed method show similarity to those of correspondence analysis or Hayashis (1952) quantification method Type III. Numerical examples show the usefulness of our method.

Regularized linear fuzzy clustering and probabilistic PCA mixture models

Fuzzy c-means (FCM)-type fuzzy clustering approaches are closely related to Gaussian mixture models (GMMs) and EM-like algorithms have been used in FCM clustering with regularized objective functions. Especially, FCM with regularization by Kullback-Leibler information (KLFCM) is a fuzzy counterpart of GMMs. In this paper, we propose to apply probabilistic principal component analysis (PCA) mixture models to linear clustering following a discussion on the relationship between local PCA and linear fuzzy clustering. Although the proposed method is a kind of the constrained model of KLFCM, the algorithm includes the fuzzy c-varieties (FCV) algorithm as a special case, and the algorithm can be regarded as a modified FCV algorithm with regularization by K-L information. Numerical experiments demonstrate that the proposed clustering algorithm is more flexible than the maximum likelihood approaches and is useful for capturing local substructures properly.

Linear fuzzy clustering techniques with missing values and their application to local principal component analysis

In this paper, we propose two methods for partitioning an incomplete data set with missing values into several linear fuzzy clusters by extracting local principal components. One is an extension of fuzzy c-varieties clustering that can be regarded as the algorithm for the local principal component analysis of fuzzy covariance matrices. The other is a simultaneous application of fuzzy clustering and principal component analysis of fuzzy correlation matrices. Both methods estimate prototypes ignoring only missing values and they need no preprocessing of data such as the elimination of samples with missing values or the imputation of missing elements. Numerical examples show that the methods provide useful tools for interpretation of the local structures of a database.

Fuzzy c-means clustering with regularization by K-L information

The Gaussian mixture model or Gaussian mixture density decomposition(GMDD) use the likelihood function as a measure of fit. We show that just the same algorithm as the GMDD can be derived from a modified objective function of fuzzy c-means (FCM) clustering with the regularizer by K-L information, only when the parameter /spl lambda/ equals 2. Although the fixed-point iteration scheme of FCM is similar to that of the GMDD, the FCM has more flexible structure since the algorithm is based on the objective function method. In a slightly different manner such as installing a deterministic annealing or an addition of Gustafson and Kessels (1979) constraint, the proposed algorithm is likely to provide more valid clustering results.

Fuzzy PCA-Guided Robust

This paper proposes a new approach to robust clustering, in which a robust k-means partition is derived by using a noise-rejection mechanism based on the noise-clustering approach. The responsibility weight of each sample for the k-means process is estimated by considering the noise degree of the sample, and cluster indicators are calculated in a fuzzy principal-component-analysis (PCA) guided manner, where fuzzy PCA-guided robust k-means is performed by considering responsibility weights of samples. Then, the proposed method achieves cluster-core estimation in a deterministic way. The validity of the derived cluster cores is visually assessed through distance-sensitive ordering, which considers responsibility weights of samples. Numerical experiments demonstrate that the proposed method is useful for capturing cluster cores by rejecting noise samples, and we can easily assess cluster validity by using cluster-crossing curves.

k

Abstract Automated collaborative filtering is a popular technique for reducing information overload and the task is to predict missing values in a data matrix. Extraction of local linear models is a useful technique for predicting the missing values. Linear models featuring local structures of the high-dimensional incomplete data set are estimated by a modified linear fuzzy clustering algorithm. Fuzzy c -varieties (FCV) is a linear fuzzy clustering algorithm that estimates local principal component vectors as the vectors spanning prototypes of clusters. Least squares techniques, however, often fail to account for “outliers”, which are common in real applications. In this paper, a technique for making the FCV algorithm robust to intra-sample outliers is proposed. The objective function based on the lower rank approximation of the data matrix is minimized by a robust M-estimation algorithm that is similar to FCM-type iterative procedures. In numerical experiments, the diagnostic power of the filtering system is shown to be improved by predicting missing values using robust local linear models.

-Means Clustering

Automated collaborative filtering is a popular technique for reducing information overload. In this paper, we propose a new approach for the collaborative filtering using local principal components. The new method is based on a simultaneous approach to principal component analysis and fuzzy clustering with an incomplete data set including missing values. In the simultaneous approach, we extract local principal components by using lower rank approximation of the data matrix. The missing values are predicted using the approximation of the data matrix. In numerical experiment, we apply the proposed technique to the recommendation system of background designs of stationery for word processor.

Component-wise robust linear fuzzy clustering for collaborative filtering

The most prevailing approach now for parking lot vehicle detection system is to use sensor-based techniques such as ultrasound and infrared-light sensors. A few engineering firms provide camera-based systems, which are only for underground and indoor parking lots due to the poor accuracy of the detector. The main impediments to the camera-based system in applying to outdoor parking lots are adherent rain drops on the lens in the rain, glaring sun light and dark shadows in the daytime, and low-light intensity and back-lighting in the nighttime. To date, no camera-based detecting systems for outdoor parking lots have been in practical use. This paper reports on the performance of the detector based on the fuzzy c-means (FCM) clustering and the hyperparameter tuning by particle swarm optimization (PSO). The new system was introduced to an underground parking lot in Tokyo in early October 2009 and achieved the detection rate (sensitivity/specificity) of 99.9%. The system was also tested at an outdoor (rooftop) parking lot for a period of two months and achieved 99.6%. The performance clearly surpassed the initial goal of the project. In terms of classification accuracy, the FCM classifier is better than the support vector machine (SVM) and the computation time for training is an order of magnitude smaller than that of SVM.

Collaborative Filtering Using Principal Component Analysis and Fuzzy Clustering

A novel membership function and a fuzzy clustering approach derived from a viewpoint of iteratively reweighted least square (IRLS) techniques resolve the problem of singularity in the regular fuzzy c-means (FCM) clustering. An FCM classifier using the membership function and Mahalanobis distances makes class memberships of outliers less clear-cut, which thus resolve the problem of classification based on normal populations or normal mixtures. The ways of handling singular covariance matrices and missing values are also furnished, which improve the generalization capability of the classifier. Computational experiments show high classification performance on several well-known benchmark data sets

Improvement in the performance of camera based vehicle detector for parking lot

This paper considers a new approach to user-item clustering for collaborative filtering problems that achieves personalized recommendation. When user-item relations are given by an alternative process, personalized recommendation is performed by finding user-item neighborhoods (co-clusters) from a rectangular relational data matrix, in which users and items have mutually positive relations. In the proposed approach, user-item clusters are extracted one by one in a sequential manner via a structural balancing technique, used in conjunction with the sequential fuzzy cluster extraction method.