Rasul Tutunov

On convergence rate of Accelerated Dual Descent Algorithm

We expand and refine the convergence rate analysis of the Accelerated Dual Descent (ADD) Algorithm, a fast distributed solution to the convex network flow optimization problem. ADD uses local information to compute an approximate Newton direction for the dual problem. It has been previously shown that the quality of the approximation depends on a network invariant related to the expansion properties of the underlying graph. The main result of this work is to characterize this information spreading parameter in terms of the network structure and properties of the primal objective. The three phase convergence Theorem for ADD is revisited and bounds which explicitly depend on the network structure are presented. Additionally, an upper bound on the number of iterations required to reach the terminal phase is proven. We explore what type of graphs work best with ADD by examining our characterization of the information spreading coefficient in the context of commonly studied network structures.

An exact distributed newton method for reinforcement learning

In this paper, we propose a distributed second-order method for reinforcement learning. Our approach is the fastest in literature so-far as it outperforms state-of-the-art methods, including ADMM, by significant margins. We achieve this by exploiting the sparsity pattern of the dual Hessian and transforming the problem of computing the Newton direction to one of solving a sequence of symmetric diagonally dominant system of equations. We validate the above claim both theoretically and empirically. On the theoretical side, we prove that similar to exact Newton, our algorithm exhibits super-linear convergence within a neighborhood of the optimal solution. Empirically, we demonstrate the superiority of this new method on a set of benchmark reinforcement learning tasks.

Identifiability of links and nodes in multi-agent systems under the agreement protocol

In this paper, the question of identifying various links and nodes in the network based on the observed agent dynamics is addressed. The focus is on a multi-agent network that evolves under the linear agreement protocol. The results help determine if various components of the network are distinguishable from each other, based on the choice of initial conditions and the observed output responses. Identifiability of links and nodes are studied separately, and in each case, the role of symmetries in the network information flow graph are analyzed. Examples are provided to elucidate the results.

Fast, accurate second order methods for network optimization

Dual descent methods are commonly used to solve network flow optimization problems, since their implementation can be distributed over the network. These algorithms, however, often exhibit slow convergence rates. Approximate Newton methods which compute descent directions locally have been proposed as alternatives to accelerate the convergence rates of conventional dual descent. The effectiveness of these methods, is limited by the accuracy of such approximations. In this paper, we propose an efficient and accurate distributed second order method for network flow problems. Our approach utilizes the sparsity pattern of the dual Hessian to approximate the the Newton direction using a novel distributed solver for symmetric diagonally dominant linear equations. We analyze the properties of the proposed algorithm and show that superlinear convergence within a neighborhood of the optimal value. We finally demonstrate the effectiveness of the approach in a set of experiments.