Sindri Magnusson

On the Convergence of Alternating Direction Lagrangian Methods for Nonconvex Structured Optimization Problems

Nonconvex and structured optimization problems arise in many engineering applications that demand scalable and distributed solution methods. The study of the convergence properties of these methods is, in general, difficult due to the nonconvexity of the problem. In this paper, two distributed solution methods that combine the fast convergence properties of augmented Lagrangian-based methods with the separability properties of alternating optimization are investigated. The first method is adapted from the classic quadratic penalty function method and is called the alternating direction penalty method (ADPM). Unlike the original quadratic penalty function method, where single-step optimizations are adopted, ADPM uses an alternating optimization which, in turn, makes it scalable. The second method is the well-known alternating direction method of multipliers (ADMM). It is shown that ADPM for nonconvex problems asymptotically converges to a primal feasible point under mild conditions and an additional condition ensuring that it asymptotically reaches the standard first-order necessary conditions for local optimality is introduced. In the case of the ADMM, novel sufficient conditions under which the algorithm asymptotically reaches the standard first-order necessary conditions are established. Based on this, complete convergence of the ADMM for a class of low-dimensional problems is characterized. Finally, the results are illustrated by applying ADPM and ADMM to a nonconvex localization problem in wireless-sensor networks.

A Distributed Approach for the Optimal Power-Flow Problem Based on ADMM and Sequential Convex Approximations

For operating electrical power networks, the Optimal Power Flow (OPF) problem plays a central role. The problem is nonconvex and NP hard. Therefore, designing efficient solution algorithms is crucial, though their global optimality is not guaranteed. Existing semi-definite programming relaxation based approaches are restricted to OPF problems for which zero duality holds, whereas for non-convex problems there is a lack of solution methods of provable performance. In this paper, an efficient novel method to address the general nonconvex OPF problem is investigated. The proposed method is based on alternating direction method of multipliers combined with sequential convex approximations. The global OPF problem is decomposed into smaller problems associated to each bus of the network, the solutions of which are coordinated via a light communication protocol. Therefore, the proposed method is highly scalable. The convergence properties of the proposed algorithm are mathematically and numerically substantiated.

Distributed resource allocation using one-way communication with applications to power networks

Typical coordination schemes for future power grids require two-way communications. Since the number of end power-consuming devices is large, the bandwidth requirements for such two-way communication schemes may be prohibitive. Motivated by this observation, we study distributed coordination schemes that require only one-way limited communications. In particular, we investigate how dual descent distributed optimization algorithm can be employed in power networks using one-way communication. In this iterative algorithm, system coordinators broadcast coordinating (or pricing) signals to the users/devices who update power consumption based on the received signal. Then system coordinators update the coordinating signals based on the physical measurement of the aggregate power usage. We provide conditions to guarantee the feasibility of the aggregated power usage at each iteration so as to avoid blackout. Furthermore, we prove the convergence of algorithms under these conditions, and establish its rate of convergence. We illustrate the performance of our algorithms using numerical simulations. These results show that one-way limited communication may be viable for coordinating/operating the future smart grids.

Extensions of Fast-Lipschitz Optimization

The need of fast distributed solvers for optimization problems in networked systems has motivated the recent development of the Fast-Lipschitz optimization framework. In such an optimization, problems satisfying certain qualifying conditions, such as monotonicity of the objective function and contractivity of the constraints, have a unique optimal solution obtained via fast distributed algorithms that compute the fixed point of the constraints. This paper extends the set of problems for which the Fast-Lipschitz framework applies. Existing assumptions on the problem form are relaxed and new and generalized qualifying conditions are established by novel results based on Lagrangian duality. It is shown for which cases of more constraints than decision variables, and less constraints than decision variables Fast-Lipschitz optimization applies. New results are obtained by imposing non strict monotonicity of the objective functions. The extended Fast-Lipschitz framework is illustrated by a number of examples, including network optimization and optimal control problems.

Robustness analysis for an online decentralized descent power allocation algorithm

As independent service providers shift from conventional energy to renewable energy sources, the power distribution system will likely experience increasingly significant fluctuation in supply, given the uncertain and intermittent nature of renewable sources like wind and solar energy. These fluctuations in power generation, coupled with time-varying consumer demands of electricity and the massive scale of power distribution networks present the need to not only design real-time decentralized power allocation algorithms, but also characterize how effective they are given fast-changing consumer demands and power generation capacities. In this paper, we present an Online Decentralized Dual Descent (OD3) power allocation algorithm and determine (in the worst case) how much of observed social welfare and price volatility can be explained by fluctuations in generation capacity and consumer demand. Convergence properties and performance guarantees of the OD3 algorithm are analyzed by characterizing the difference between the online decision and the optimal decision. The theoretical results in the paper are validated and illustrated by numerical experiments using real data.

On Variability of Renewable Energy and Online Power Allocation

As electric power system operators shift from conventional energy to renewable energy sources, power distribution systems will experience increasing fluctuations in supply. These fluctuations present the need to not only design online decentralized power allocation algorithms, but also characterize how effective they are given fast-changing consumer demand and generation. In this paper, we present an online decentralized dual descent (OD3) power allocation algorithm and determine (in the worst case) how much of observed social welfare can be explained by fluctuations in generation capacity and consumer demand. Convergence properties and performance guarantees of the OD3 algorithm are analyzed by characterizing the difference between the online decision and the optimal decision. We demonstrate validity and accuracy of the theoretical results in the paper through numerical experiments using real power generation data.

On the convergence of an alternating direction penalty method for nonconvex problems

This paper investigates convergence properties of scalable algorithms for nonconvex and structured optimization. We consider a method that is adapted from the classic quadratic penalty function method, the Alternating Direction Penalty Method (ADPM). Unlike the original quadratic penalty function method, in which single-step optimizations are adopted, ADPM uses alternating optimization, which in turn is exploited to enable scalability of the algorithm. A special case of ADPM is a variant of the well known Alternating Direction Method of Multipliers (ADMM), where the penalty parameter is increased to infinity. We show that due to the increasing penalty, the ADPM asymptotically reaches a primal feasible point under mild conditions. Moreover, we give numerical evidence that demonstrates the potential of the ADPM for computing local optimal points when the penalty is not updated too aggressively.

Convergence of limited communications gradient methods

Distributed control and decision making increasingly play a central role in economical and sustainable operation of cyber-physical systems. Nevertheless, the full potential of the technology has not yet been fully exploited in practice due to communication limitations of real-world infrastructures. This work investigates the fundamental properties of gradient methods for distributed optimization, where gradient information is communicated at every iteration, when using limited number of communicated bits. In particular, a general class of quantized gradient methods are studied where the gradient direction is approximated by a finite quantization set. Conditions on the quantization set are provided that are necessary and sufficient to guarantee the ability of these methods to minimize any convex objective function with Lipschitz continuous gradient and a nonempty, bounded set of optimizers. Moreover, a lower bound on the cardinality of the quantization set is provided, along with specific examples of minimal quantizations. Furthermore, convergence rate results are established that connect the fineness of the quantization and number of iterations needed to reach a predefined solution accuracy. The results provide a bound on the number of bits needed to achieve the desired accuracy. Finally, an application of the theory to resource allocation in power networks is demonstrated, and the theoretical results are substantiated by numerical simulations.

Convergence of Limited Communication Gradient Methods

Distributed optimization increasingly plays a central role in economical and sustainable operation of cyber-physical systems. Nevertheless, the complete potential of the technology has not yet been fully exploited in practice due to communication limitations posed by the real-world infrastructures. This work investigates fundamental properties of distributed optimization based on gradient methods, where gradient information is communicated using a limited number of bits. In particular, a general class of quantized gradient methods are studied, where the gradient direction is approximated by a finite quantization set. Sufficient and necessary conditions are provided on such a quantization set to guarantee that the methods minimize any convex objective function with Lipschitz continuous gradient and a nonempty and bounded set of optimizers. A lower bound on the cardinality of the quantization set is provided, along with specific examples of minimal quantizations. Convergence rate results are established that connect the fineness of the quantization and the number of iterations needed to reach a predefined solution accuracy. Generalizations of the results to a relevant class of constrained problems using projections are considered. Finally, the results are illustrated by simulations of practical systems.

A distributed approach for the optimal power flow problem

The optimal power flow (OPF) problem, which plays a central role in operating electrical networks is considered. The problem is nonconvex and is in fact NP hard. Therefore, designing efficient algorithms of practical relevance is crucial, though their global optimality is not guaranteed. Existing semi-definite programming relaxation based approaches are restricted to OPF problems where zero duality holds. In this paper, an efficient novel method to address the general nonconvex OPF problem is investigated. The proposed method is based on alternating direction method of multipliers combined with sequential convex approximations. The global OPF problem is decomposed into smaller problems associated to each bus of the network, the solutions of which are coordinated via a light communication protocol. Therefore, the proposed method is highly scalable. The convergence properties of the proposed algorithm are mathematically substantiated. Finally, the proposed algorithm is evaluated on a number of test examples, where the convergence properties of the proposed algorithm are numerically substantiated and the performance is compared with a global optimal method.