Naman Agarwal

Finding approximate local minima faster than gradient descent

We design a non-convex second-order optimization algorithm that is guaranteed to return an approximate local minimum in time which scales linearly in the underlying dimension and the number of training examples. The time complexity of our algorithm to find an approximate local minimum is even faster than that of gradient descent to find a critical point. Our algorithm applies to a general class of optimization problems including training a neural network and other non-convex objectives arising in machine learning.

Multisection in the Stochastic Block Model Using Semidefinite Programming

We consider the problem of identifying underlying community-like structures in graphs. Toward this end, we study the stochastic block model (SBM) on k-clusters: a random model on n = km vertices, partitioned in k equal sized clusters, with edges sampled independently across clusters with probability q and within clusters with probability p, p > q. The goal is to recover the initial “hidden” partition of [n]. We study semidefinite programming (SDP)-based algorithms in this context. In the regime \(p = \frac {\alpha \log (m)}{m}\) and \(q = \frac {\beta \log (m)}{m}\), we show that a certain natural SDP-based algorithm solves the problem of exact recovery in the k-community SBM, with high probability, whenever \(\sqrt {\alpha } - \sqrt {\beta } > \sqrt {1}\), as long as \(k=o(\log n)\). This threshold is known to be the information theoretically optimal. We also study the case when \(k=\theta (\log (n))\). In this case however, we achieve recovery guarantees that no longer match the optimal condition \(\sqrt {\alpha } - \sqrt {\beta } > \sqrt {1}\), thus leaving achieving optimality for this range an open question.

On the Expansion of Group-Based Lifts

A

Second Order Stochastic Optimization in Linear Time.

k

Finding Approximate Local Minima for Nonconvex Optimization in Linear Time.

-lift of an

Second-Order Stochastic Optimization for Machine Learning in Linear Time

n

Finding Local Minima for Nonconvex Optimization in Linear Time

-vertex base graph

Unique Games on the Hypercube

G

cpSGD: Communication-efficient and differentially-private distributed SGD

is a graph

Lower Bounds for Higher-Order Convex Optimization

H