Anastasios Zouzias

Randomized Extended Kaczmarz For Solving Least Squares

We present a randomized iterative algorithm that exponentially converges in the mean square to the minimum

Randomized Dimensionality Reduction for

\ell_2

k

-norm least squares solution of a given linear system of equations. The expected number of arithmetic operations required to obtain an estimate of given accuracy is proportional to the squared condition number of the system multiplied by the number of nonzero entries of the input matrix. The proposed algorithm is an extension of the randomized Kaczmarz method that was analyzed by Strohmer and Vershynin.

-Means Clustering

We study the topic of dimensionality reduction for k-means clustering. Dimensionality reduction encompasses the union of two approaches: 1) feature selection and 2) feature extraction. A feature selection-based algorithm for k-means clustering selects a small subset of the input features and then applies k-means clustering on the selected features. A feature extraction-based algorithm for k-means clustering constructs a small set of new artificial features and then applies k-means clustering on the constructed features. Despite the significance of k-means clustering as well as the wealth of heuristic methods addressing it, provably accurate feature selection methods for k-means clustering are not known. On the other hand, two provably accurate feature extraction methods for k-means clustering are known in the literature; one is based on random projections and the other is based on the singular value decomposition (SVD). This paper makes further progress toward a better understanding of dimensionality reduction for k-means clustering. Namely, we present the first provably accurate feature selection method for k-means clustering and, in addition, we present two feature extraction methods. The first feature extraction method is based on random projections and it improves upon the existing results in terms of time complexity and number of features needed to be extracted. The second feature extraction method is based on fast approximate SVD factorizations and it also improves upon the existing results in terms of time complexity. The proposed algorithms are randomized and provide constant-factor approximation guarantees with respect to the optimal k-means objective value.

Fast distributed smoothing of relative measurements

We consider the problem of estimation from noisy relative measurements in a network. In previous work, a distributed scheme for obtaining least-squares (LS) estimates was developed based on the Jacobi algorithm; in a synchronous version, the algorithm was shown to converge exponentially and bounds on the rate of convergence have been obtained. In this paper, we design and analyze a new class of distributed asynchronous smoothing algorithms based on a randomized version of Kaczmarz algorithm for solving linear systems. One of the proposed schemes applies Randomized Kaczmarz directly to the noisy linear system, whereas the other one operates on the normal equations for LS estimation. We analyze the expected convergence rate of the proposed algorithms depending solely on properties of the network topology. Inspired by the analytical insights, we propose a distributed smoothing algorithm, namely Randomized Kaczmarz Over-smoothing (RKO), which has demonstrated significant improvement over existing protocols in terms of both convergence speedup and energy savings.

Near Optimal Dimensionality Reductions That Preserve Volumes

Let Pbe a set of npoints in Euclidean space and let 0 < i¾?< 1. A well-known result of Johnson and Lindenstrauss states that there is a projection of Ponto a subspace of dimension

A matrix hyperbolic cosine algorithm and applications

\mathcal{O}(\epsilon^{-2} \log n)

Hidden Cliques and the Certification of the Restricted Isometry Property

such that distances change by at most a factor of 1 + i¾?. We consider an extension of this result. Our goal is to find an analogous dimension reduction where not only pairs but all subsets of at most kpoints maintain their volume approximately. More precisely, we require that sets of size s≤ kpreserve their volumes within a factor of (1 + i¾?)si¾? 1. We show that this can be achieved using

A note on element-wise matrix sparsification via a matrix-valued Bernstein inequality

\mathcal{O}(\max\{\frac{k}{\epsilon},\epsilon^{-2}\log n\})

Unsupervised sparse matrix co-clustering for marketing and sales intelligence

dimensions. This in particular means that for