Franck Iutzeler

Asynchronous distributed optimization using a randomized alternating direction method of multipliers

Consider a set of networked agents endowed with private cost functions and seeking to find a consensus on the minimizer of the aggregate cost. A new class of random asynchronous distributed optimization methods is introduced. The methods generalize the standard Alternating Direction Method of Multipliers (ADMM) to an asynchronous setting where isolated components of the network are activated in an uncoordinated fashion. The algorithms rely on the introduction of randomized Gauss-Seidel iterations of Douglas-Rachford splitting leading to an asynchronous algorithm based on the ADMM. Convergence to the sought minimizers is provided under mild connectivity conditions.

Explicit Convergence Rate of a Distributed Alternating Direction Method of Multipliers

Consider a set of N agents seeking to solve distributively the minimization problem inf_∞ Σ_n=1^N f_n(x) where the convex functions f_n are local to the agents. The popular Alternating Direction Method of Multipliers has the potential to handle distributed optimization problems of this kind. We provide a general reformulation of the problem and obtain a class of distributed algorithms which encompass various network architectures. The rate of convergence of our method is considered. It is assumed that the infimum of the problem is reached at a point x_*, the functions f_n are twice differentiable at this point and Σ ∇²f_n(x_*) > 0 in the positive definite ordering of symmetric matrices. With these assumptions, it is shown that the convergence to the consensus x_* is linear and the exact rate is provided. Application examples where this rate can be optimized with respect to the ADMM free parameter ρ are also given.

Analysis of Max-Consensus Algorithms in Wireless Channels

In this paper, we address the problem of estimating the maximal value over a sensor network using wireless links between them. We introduce two heuristic algorithms and analyze their theoretical performance. More precisely, i) we prove that their convergence time is finite with probability one, ii) we derive an upper-bound on their mean convergence time, and iii) we exhibit a bound on their convergence time dispersion.

Analysis of Sum-Weight-Like Algorithms for Averaging in Wireless Sensor Networks

Distributed estimation of the average value over a Wireless Sensor Network has recently received a lot of attention. Most papers consider single variable sensors and communications with feedback (e.g., peer-to-peer communications). However, in order to use efficiently the broadcast nature of the wireless channel, communications without feedback are advocated. To ensure the convergence in this feedback-free case, the recently-introduced Sum-Weight-like algorithms which rely on two variables at each sensor are a promising solution. In this paper, the convergence towards the consensus over the average of the initial values is analyzed in depth. Furthermore, it is shown that the squared error decreases exponentially with the time. In addition, a powerful algorithm relying on the Sum-Weight structure and taking into account the broadcast nature of the channel is proposed.

A Coordinate Descent Primal-Dual Algorithm and Application to Distributed Asynchronous Optimization

Based on the idea of randomized coordinate descent of α-averaged operators, a randomized primal-dual optimization algorithm is introduced, where a random subset of coordinates is updated at each iteration. The algorithm builds upon a variant of a recent (deterministic) algorithm proposed by Vũ and Condat that includes the well-known Alternating Direction Method of Multipliers as a particular case. The obtained algorithm is used to solve asynchronously a distributed optimization problem. A network of agents, each having a separate cost function containing a differentiable term, seek to find a consensus on the minimum of the aggregate objective. The method yields an algorithm where at each iteration, a random subset of agents wake up, update their local estimates, exchange some data with their neighbors, and go idle. Numerical results demonstrate the attractive performance of the method. The general approach can be naturally adapted to other situations where coordinate descent convex optimization algorithms are used with a random choice of the coordinates.

New broadcast based distributed averaging algorithm over wireless sensor networks

The distributed estimation of the average value of the sensors initial measures is one of the most popular issues in the Wireless Sensor Networks (WSN) area. In WSNs, broadcasting data seems natural to exchange information quickly because of the broadcast nature of the Wireless channel. Nevertheless, although broadcast-based algorithms converge faster than pairwise algorithms, the obtained consensus is not necessarily the true average. By the means of additional side-information exchange, we propose a broadcast-based algorithm converging rapidly to the true average. The convergence of this new algorithm is established and its convergence speed is exhibited. We remark that the proposed algorithm outperforms the existing ones.

A stochastic primal-dual algorithm for distributed asynchronous composite optimization

Consider a network where each agent has a private composite function (e.g. the sum of a smooth and a non-smooth function). The problem we address here is to And a minimize! of the aggregate cost (the sum of the agents functions) in a distributed manner. In this paper, we combine recent results on primal-dual optimization and coordinate descent to propose an asynchronous distributed algorithm for composite optimization.

Linear convergence rate for distributed optimization with the alternating direction method of multipliers

Consider the problem of distributed optimization where a network of N agents cooperate to solve a minimization problem of the form inf_x Σ_n=1^N f_n(x) where function fn is convex and known only by agent n. The Alternating Direction Method of Multipliers (ADMM) has shown to be particularly efficient to solve this kind of problem. In this paper, we assume that there exists a unique minimum x_* and that the functions f_n are twice differentiable at x_* and verify Σ_n=1^N ∇²f_n(x_*) > 0 where the inequality is taken in the positive definite ordering. Under these assumptions, we prove the linear convergence of the distributed ADMM to the consensus over x_* and derive a tight convergence rate. Finally, we give examples where one can derive the ADMM hyper-parameter ρ corresponding to the optimal rate.

Distributed estimation of the maximum value over a Wireless Sensor Network

This paper focuses on estimating the maximum of the initial measures in a Wireless Sensor Network. Two different algorithms are studied: the RANDOM GOSSIP, relying on pairwise exchanges between the nodes, and the BROADCAST in which each sensor sends its value to all its neighbors; both are asynchronous and distributed. We prove the convergence of these algorithms and provide tight bounds for their convergence speed.