Chang-Dong Wang

SVStream: A Support Vector-Based Algorithm for Clustering Data Streams

In this paper, we propose a novel data stream clustering algorithm, termed SVStream, which is based on support vector domain description and support vector clustering. In the proposed algorithm, the data elements of a stream are mapped into a kernel space, and the support vectors are used as the summary information of the historical elements to construct cluster boundaries of arbitrary shape. To adapt to both dramatic and gradual changes, multiple spheres are dynamically maintained, each describing the corresponding data domain presented in the data stream. By allowing for bounded support vectors (BSVs), the proposed SVStream algorithm is capable of identifying overlapping clusters. A BSV decaying mechanism is designed to automatically detect and remove outliers (noise). We perform experiments over synthetic and real data streams, with the overlapping, evolving, and noise situations taken into consideration. Comparison results with state-of-the-art data stream clustering methods demonstrate the effectiveness and efficiency of the proposed method.

Robust Ensemble Clustering Using Probability Trajectories

Although many successful ensemble clustering approaches have been developed in recent years, there are still two limitations to most of the existing approaches. First, they mostly overlook the issue of uncertain links, which may mislead the overall consensus process. Second, they generally lack the ability to incorporate global information to refine the local links. To address these two limitations, in this paper, we propose a novel ensemble clustering approach based on sparse graph representation and probability trajectory analysis. In particular, we present the elite neighbor selection strategy to identify the uncertain links by locally adaptive thresholds and build a sparse graph with a small number of probably reliable links. We argue that a small number of probably reliable links can lead to significantly better consensus results than using all graph links regardless of their reliability. The random walk process driven by a new transition probability matrix is utilized to explore the global information in the graph. We derive a novel and dense similarity measure from the sparse graph by analyzing the probability trajectories of the random walkers, based on which two consensus functions are further proposed. Experimental results on multiple real-world datasets demonstrate the effectiveness and efficiency of our approach.

Locally Weighted Ensemble Clustering

Due to its ability to combine multiple base clusterings into a probably better and more robust clustering, the ensemble clustering technique has been attracting increasing attention in recent years. Despite the significant success, one limitation to most of the existing ensemble clustering methods is that they generally treat all base clusterings equally regardless of their reliability, which makes them vulnerable to low-quality base clusterings. Although some efforts have been made to (globally) evaluate and weight the base clusterings, yet these methods tend to view each base clustering as an individual and neglect the local diversity of clusters inside the same base clustering. It remains an open problem how to evaluate the reliability of clusters and exploit the local diversity in the ensemble to enhance the consensus performance, especially, in the case when there is no access to data features or specific assumptions on data distribution. To address this, in this paper, we propose a novel ensemble clustering approach based on ensemble-driven cluster uncertainty estimation and local weighting strategy. In particular, the uncertainty of each cluster is estimated by considering the cluster labels in the entire ensemble via an entropic criterion. A novel ensemble-driven cluster validity measure is introduced, and a locally weighted co-association matrix is presented to serve as a summary for the ensemble of diverse clusters. With the local diversity in ensembles exploited, two novel consensus functions are further proposed. Extensive experiments on a variety of real-world datasets demonstrate the superiority of the proposed approach over the state-of-the-art.

Incremental Support Vector Clustering

Support vector clustering (SVC) is a flexible clustering method inspired by support vector machines (SVM). Due to its advantage in discovering clusters of arbitrary shapes, it has been widely used in many applications. However, one bottleneck which restricts the scalability of the method is its significantly high time complexity. Both of its two main stages, namely, sphere construction and cluster labeling, are quite time-consuming. Although some methods have been developed to speedup cluster labeling, it is still an intractable task to construct a sphere for a large-scale dataset. To this end, we propose a novel incremental support vector clustering (ISVC) algorithm, which constructs a sphere incrementally and efficiently. In our approach, by taking the data as arriving over time in chunks, the support vectors of the historical data and the data points of the new chunk are used to learn an updated sphere. Theoretical analysis has shown that the proposed ISVC algorithm can generate completely the same clustering results as SVC with much lower time and memory consumption. Experimental results on large-scale datasets have validated the theoretical analysis.

Exploiting the Wisdom of Crowd: A Multi-granularity Approach to Clustering Ensemble

There are three levels of granularity in a clustering ensemble system, namely, base clusterings, clusters, and instances. In this paper, we propose a novel clustering ensemble approach which integrates information from different levels of granularity into a unified graph model. The normalized crowd agreement index (NCAI) is presented for estimating the quality of base clusterings in an unsupervised manner. The source aware connected triple (SACT) method is proposed for inter-cluster link analysis. By treating the clusters and the instances altogether as nodes, we formulate the ensemble of base clusterings and multiple levels of relationship among them into a bipartite graph. The final consensus clustering is obtained via an efficient graph partitioning algorithm. Experiments are conducted on four real-world datasets from UCI Machine Learning Repository. Experimental results demonstrate the effectiveness of our approach for solving the clustering ensemble problem.

Ensembling over-segmentations

Due to the high diversity of image data, image segmentation is still a very challenging problem after decades of development. Each segmentation algorithm has its merits as well as its drawbacks. Instead of segmenting images via conventional techniques, inspired by the idea of the ensemble clustering technique that combines a set of weak clusterers to obtain a strong clusterer, we propose to achieve a consensus segmentation by fusing evidence accumulated from multiple weak segmentations (or over-segmentations). We present a novel image segmentation approach which exploits multiple over-segmentations and achieves segmentation results by hierarchical region merging. The cross-region evidence accumulation (CREA) mechanism is designed for collecting information among over-segmentations. The pixel-pairs across regions are treated as a bag of independent voters and the cumulative votes from multiple over-segmentations are fused to estimate the coherency of adjacent regions. We further integrate the brightness, color, and texture cues for measuring the appearance similarity between regions in an over-segmentation, which, together with the CREA information, are utilized for making the region merging decisions. Experiments are conducted on multiple public datasets, which demonstrate the superiority of our approach in terms of both effectiveness and efficiency when compared to the state-of-the-art.

Multi-view Unit Intact Space Learning

Multi-view learning is a hot research topic in different research fields. Recently, a model termed multi-view intact space learning has been proposed and drawn a large amount of attention. The model aims to find the latent intact representation of data by integrating information from different views. However, the model has two obvious shortcomings. One is that the model needs to tune two regularization parameters. The other is that the optimization algorithm is too time-consuming. Based on the unit intact space assumption, we propose an improved model, termed multi-view unit intact space learning, without introducing any prior parameters. Besides, an efficient algorithm based on proximal gradient scheme is designed to solve the model. Extensive experiments have been conducted on four real-world datasets to show the effectiveness of our method.

Multi-view Proximity Learning for Clustering

In recent years, multi-view clustering has become a hot research topic due to the increasing amount of multi-view data. Among existing multi-view clustering methods, proximity-based method is a typical class and achieves much success. Usually, these methods need proximity matrices as inputs, which can be constructed by some nearest-neighbors-based approaches. However, in this way, neither the intra-view cluster structure nor the inter-view correlation is considered in constructing proximity matrices. To address this issue, we propose a novel method, named multi-view proximity learning. By introducing the idea of representative, our model can consider both the relations between data objects and the cluster structure within individual views. Besides, the spectral-embedding-based scheme is adopted for modeling the correlations across different views, i.e. the view consistency and complement properties. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of our method.

TW-Co-k-means: Two-level weighted collaborative k-means for multi-view clustering

Abstract Multi-view clustering has attracted an increasing amount of attention in recent years due to its ability to analyze data from multiple sources or views. Despite significant success, there are still two challenging problems in multi-view clustering, namely, (i) how to satisfy the consistency across different views while preserving the diversity within each view, and (ii) how to weight the different views and the features in each view w.r.t. their importance to improve the clustering result. In this paper, to simultaneously tackle these two problems, we propose a novel multi-view clustering approach termed Two-level Weighted Collaborative k-means (TW-Co-k-means). A new objective function is designed for multi-view clustering, which exploits the distinctive information in each view while taking advantage of the complementariness and consistency across different views in a collaborative manner. The views and the features in each view are assigned with weights that reflect their importance. We introduce an iterative optimization method to optimize the objective function and thereby achieve the final clustering result. Experimental results on multiple real-world datasets demonstrate the effectiveness of our approach.

An item orientated recommendation algorithm from the multi-view perspective

Abstract In the traditional recommendation algorithms, items are recommended to users on the basis of users’ preferences to improve selling efficiency, which however cannot always raise revenues for manufacturers of particular items. Assume that, a manufacturer has a limited budget for an item’s advertisement, with this budget, it is only possible for him to market this item to limited users. How to select the most suitable users that will increase advertisement revenue? It seems to be an insurmountable problem to the existing recommendation algorithms. To address this issue, a new item orientated recommendation algorithm from the multi-view perspective is proposed in this paper. Different from the existing recommendation algorithms, this model provides the target items with the users that are the most possible to purchase them. The basic idea is to simultaneously calculate the relationships between items and the rating differences between users from a multi-view model in which the purchasing records of each user are regarded as a view and each record is seen as a node in a view. The experimental results show that our proposed method outperforms the state-of-the-art methods in the scenario of item orientated recommendation.