João F. C. Mota

D-ADMM: A Communication-Efficient Distributed Algorithm for Separable Optimization

We propose a distributed algorithm, named Distributed Alternating Direction Method of Multipliers (D-ADMM), for solving separable optimization problems in networks of interconnected nodes or agents. In a separable optimization problem there is a private cost function and a private constraint set at each node. The goal is to minimize the sum of all the cost functions, constraining the solution to be in the intersection of all the constraint sets. D-ADMM is proven to converge when the network is bipartite or when all the functions are strongly convex, although in practice, convergence is observed even when these conditions are not met. We use D-ADMM to solve the following problems from signal processing and control: average consensus, compressed sensing, and support vector machines. Our simulations show that D-ADMM requires less communications than state-of-the-art algorithms to achieve a given accuracy level. Algorithms with low communication requirements are important, for example, in sensor networks, where sensors are typically battery-operated and communicating is the most energy consuming operation.

Distributed Basis Pursuit

We propose a distributed algorithm for solving the optimization problem Basis Pursuit (BP). BP finds the least ℓ1-norm solution of the underdetermined linear system Ax = b and is used, for example, in compressed sensing for reconstruction. Our algorithm solves BP on a distributed platform such as a sensor network, and is designed to minimize the communication between nodes. The algorithm only requires the network to be connected, has no notion of a central processing node, and no node has access to the entire matrix A at any time. We consider two scenarios in which either the columns or the rows of A are distributed among the compute nodes. Our algorithm, named D-ADMM, is a decentralized implementation of the alternating direction method of multi- pliers. We show through numerical simulation that our algorithm requires considerably less communications between the nodes than the state-of-the-art algorithms.

Distributed ADMM for model predictive control and congestion control

Many problems in control can be modeled as an optimization problem over a network of nodes. Solving them with distributed algorithms provides advantages over centralized solutions, such as privacy and the ability to process data locally. In this paper, we solve optimization problems in networks where each node requires only partial knowledge of the problems solution. We explore this feature to design a decentralized algorithm that allows a significant reduction in the total number of communications. Our algorithm is based on the Alternating Direction of Multipliers (ADMM), and we apply it to distributed Model Predictive Control (MPC) and TCP/IP congestion control. Simulation results show that the proposed algorithm requires less communications than previous work for the same solution accuracy.

Compressed sensing with side information: Geometrical interpretation and performance bounds

We address the problem of Compressed Sensing (CS) with side information. Namely, when reconstructing a target CS signal, we assume access to a similar signal. This additional knowledge, the side information, is integrated into CS via ℓ₁-ℓ₁ and ℓ₁-ℓ₂ minimization. We then provide lower bounds on the number of measurements that these problems require for successful reconstruction of the target signal. If the side information has good quality, the number of measurements is significantly reduced via ℓ₁-ℓ₁ minimization, but not so much via ℓ₁-ℓ₂ minimization. We provide geometrical interpretations and experimental results illustrating our findings.

Distributed Optimization With Local Domains: Applications in MPC and Network Flows

We consider a network where each node has exclusive access to a local cost function. Our contribution is a communication-efficient distributed algorithm that finds a vector x* minimizing the sum of all the functions. We make the additional assumption that the functions have intersecting local domains, i.e., each function depends only on some components of the variable. Consequently, each node is interested in knowing only some components of x*, not the entire vector. This allows improving communication-efficiency. We apply our algorithm to distributed model predictive control (D-MPC) and to network flow problems and show, through experiments on large networks, that the proposed algorithm requires less communications to converge than prior state-of-the-art algorithms.

Basis Pursuit in sensor networks

Basis Pursuit (BP) finds a minimum ℓ1-norm vector z that satisfies the underdetermined linear system Mz = b, where the matrix M and vector b are given. Lately, BP has attracted attention because of its application in compressed sensing, where it is used to reconstruct signals by finding the sparsest solutions of linear systems. In this paper, we propose a distributed algorithm to solve BP. This means no central node is used for the processing and no node has access to all the data: the rows of M and the vector b are distributed over a set of interconnected compute nodes. A typical scenario is a sensor network. The novelty of our method is in using an optimal first-order method to solve an augmented Lagrangian-based reformulation of BP. We implemented our algorithm in a computer cluster, and show that it can solve problems that are too large to be stored in and processed by a single node.

Dynamic sparse state estimation using ℓ 1 -ℓ 1 minimization: Adaptive-rate measurement bounds, algorithms and applications

We propose a recursive algorithm for estimating time-varying signals from a few linear measurements. The signals are assumed sparse, with unknown support, and are described by a dynamical model. In each iteration, the algorithm solves an ℓ1-ℓ1 minimization problem and estimates the number of measurements that it has to take at the next iteration. These estimates are computed based on recent theoretical results for ℓ1-ℓ1 minimization. We also provide sufficient conditions for perfect signal reconstruction at each time instant as a function of an algorithm parameter. The algorithm exhibits high performance in compressive tracking on a real video sequence, as shown in our experimental results.

Adaptive-Rate Reconstruction of Time-Varying Signals With Application in Compressive Foreground Extraction

We propose and analyze an online algorithm for reconstructing a sequence of signals from a limited number of linear measurements. The signals are assumed sparse, with unknown support, and evolve over time according to a generic nonlinear dynamical model. Our algorithm, based on recent theoretical results for l1 - l1 minimization, is recursive and computes the number of measurements to be taken at each time on-the-fly. As an example, we apply the algorithm to online compressive video foreground extraction, a problem stated as follows: given a set of measurements of a sequence of images with a static background, simultaneously reconstruct each image while separating its foreground from the background. The performance of our method is illustrated on sequences of real images. We observe that it allows a dramatic reduction in the number of measurements or reconstruction error with respect to state-of-the-art compressive background subtraction schemes.

Agrafo: a visual interface for grouping and browsing digital photos

With the growing popularity of digital cameras, the organization, browsing, management and grouping of photos become a problem of every photograph (professional or amateur), because their collections easily achieve the order of thousands. Here, we present a system to automate these processes, which relies on photo information, such as, semantic features (extracted from content), meta-information and low level.

Reference-based compressed sensing: A sample complexity approach

We address the problem of reference-based compressed sensing: reconstruct a sparse signal from few linear measurements using as prior information a reference signal, a signal similar to the signal we want to reconstruct. Access to reference signals arises in applications such as medical imaging, e.g., through prior images of the same patient, and compressive video, where previously reconstructed frames can be used as reference. Our goal is to use the reference signal to reduce the number of required measurements for reconstruction. We achieve this via a reweighted ℓ1-ℓ1 minimization scheme that updates its weights based on a sample complexity bound. The scheme is simple, intuitive and, as our experiments show, outperforms prior algorithms, including reweighted ℓ1 minimization, ℓ1-ℓ1 minimization, and modified CS.