Mark Kliger

Adaptive evolutionary clustering

In many practical applications of clustering, the objects to be clustered evolve over time, and a clustering result is desired at each time step. In such applications, evolutionary clustering typically outperforms traditional static clustering by producing clustering results that reflect long-term trends while being robust to short-term variations. Several evolutionary clustering algorithms have recently been proposed, often by adding a temporal smoothness penalty to the cost function of a static clustering method. In this paper, we introduce a different approach to evolutionary clustering by accurately tracking the time-varying proximities between objects followed by static clustering. We present an evolutionary clustering framework that adaptively estimates the optimal smoothing parameter using shrinkage estimation, a statistical approach that improves a naïve estimate using additional information. The proposed framework can be used to extend a variety of static clustering algorithms, including hierarchical, k-means, and spectral clustering, into evolutionary clustering algorithms. Experiments on synthetic and real data sets indicate that the proposed framework outperforms static clustering and existing evolutionary clustering algorithms in many scenarios.

Clustering with a new distance measure based on a dual-rooted tree

Abstract This paper introduces a novel distance measure for clustering high dimensional data based on the hitting time of two Minimal Spanning Trees (MST) grown sequentially from a pair of points by Prim’s algorithm. When the proposed measure is used in conjunction with spectral clustering, we obtain a powerful clustering algorithm that is able to separate neighboring non-convex shaped clusters and to account for local as well as global geometric features of the data set. Remarkably, the new distance measure is a true metric even if the Prim algorithm uses a non-metric dissimilarity measure to compute the edges of the MST. This metric property brings added flexibility to the proposed method. In particular, the method is applied to clustering non Euclidean quantities, such as probability distributions or spectra, using the Kullback–Leibler divergence as a base measure. We reduce computational complexity by applying consensus clustering to a small ensemble of dual rooted MSTs. We show that the resultant consensus spectral clustering with dual rooted MST is competitive with other clustering methods, both in terms of clustering performance and computational complexity. We illustrate the proposed clustering algorithm on public domain benchmark data for which the ground truth is known, on one hand, and on real-world astrophysical data on the other hand.

Tracking communities in dynamic social networks

The study of communities in social networks has attracted considerable interest from many disciplines. Most studies have focused on static networks, and in doing so, have neglected the temporal dynamics of the networks and communities. This paper considers the problem of tracking communities over time in dynamic social networks. We propose a method for community tracking using an adaptive evolutionary clustering framework. We apply the method to reveal the temporal evolution of communities in two real data sets. In addition, we obtain a statistic that can be used for identifying change points in the network.

Evolutionary spectral clustering with adaptive forgetting factor

Many practical applications of clustering involve data collected over time. In these applications, evolutionary clustering can be applied to the data to track changes in clusters with time. In this paper, we consider an evolutionary version of spectral clustering that applies a forgetting factor to past affinities between data points and aggregates them with current affinities. We propose to use an adaptive forgetting factor and provide a method to automatically choose this forgetting factor at each time step. We evaluate the performance of the proposed method through experiments on synthetic and real data and find that, with an adaptive forgetting factor, we are able to obtain improved clustering performance compared to a fixed forgetting factor.

A regularized graph layout framework for dynamic network visualization

Many real-world networks, including social and information networks, are dynamic structures that evolve over time. Such dynamic networks are typically visualized using a sequence of static graph layouts. In addition to providing a visual representation of the network structure at each time step, the sequence should preserve the mental map between layouts of consecutive time steps to allow a human to interpret the temporal evolution of the network. In this paper, we propose a framework for dynamic network visualization in the on-line setting where only present and past graph snapshots are available to create the present layout. The proposed framework creates regularized graph layouts by augmenting the cost function of a static graph layout algorithm with a grouping penalty, which discourages nodes from deviating too far from other nodes belonging to the same group, and a temporal penalty, which discourages large node movements between consecutive time steps. The penalties increase the stability of the layout sequence, thus preserving the mental map. We introduce two dynamic layout algorithms within the proposed framework, namely dynamic multidimensional scaling and dynamic graph Laplacian layout. We apply these algorithms on several data sets to illustrate the importance of both grouping and temporal regularization for producing interpretable visualizations of dynamic networks.

On the generation of correlated time series with a given probability density function

Abstract The approach presented provides very good results in modelling non-Gaussian time series. It is validated by direct numerical simulation of bimodal and Nakagami distributed sequences with approximately exponential correlation function.

A shrinkage approach to tracking dynamic networks

The analysis of network data is of interest to many disciplines, ranging from sociology to computer science. Recent interest has shifted from static networks to dynamic networks, which evolve over time. A fundamental problem in the analysis of dynamic networks is tracking long-term trends, which are obscured by short-term variations. In this paper, we propose a method for minimum mean-squared error tracking of dynamic networks using a recursive shrinkage estimation framework that accounts for the spatial correlation in the network. Unlike model-based tracking methods such as the Kalman filter, the proposed method does not require knowledge about the network dynamics. We demonstrate that the proposed method is able to track dynamic networks effectively through experiments on simulated and real networks.

Strong consistency of the over- and underdetermined LSE of 2-D exponentials in white noise

We consider the problem of least squares estimation of the parameters of two-dimensional (2-D) exponential signals observed in the presence of an additive noise field, when the assumed number of exponentials is incorrect. We consider both the case where the number of exponential signals is underestimated, and the case where the number of exponential signals is overestimated. In the case where the number of exponential signals is underestimated, we prove the almost sure convergence of the least squares estimates (LSE) to the parameters of the dominant exponentials. In the case where the number of exponential signals is overestimated, the estimated parameter vector obtained by the least squares estimator contains a subvector that converges almost surely to the correct parameters of the exponentials.

Combining multiple partitions created with a graph-based construction for data clustering

This paper focusses on a new clustering method called evidence accumulation clustering with dual rooted prim tree cuts (EAC-DC), based on the principle of cluster ensembles also known as “combining multiple clustering methods”. A simple weak clustering algorithm is introduced based upon the properties of dual rooted minimal spanning trees and it is extended to multiple rooted trees. Co-association measures are proposed that account for the cluster sets obtained by these methods. These are exploited in order to obtain new ensemble consensus clustering algorithms. The EAC-DC methodology applied to both real and synthetic data sets demonstrates the superiority of the proposed methods.

Strong consistency of the over- and under-determined LSE of 2-D exponentials in white noise

We consider the problem of least squares estimation of the parameters of 2D exponential signals observed in the presence of an additive noise field, when the assumed number of exponentials is incorrect. We consider both the case where the number of exponential signals is under-estimated, and the case where the number of exponential signals is over-estimated. In the case where the number of exponential signals is under-estimated we prove the almost sure convergence of the least squares estimates to the parameters of the dominant exponentials. In the case where the number of exponential signals is over-estimated, the estimated parameter vector obtained by the least squares estimator contains a sub-vector that converges almost surely to the correct parameters of the exponentials.