Euhanna Ghadimi

Optimal parameter selection for the alternating direction method of multipliers (ADMM) : quadratic problems

The alternating direction method of multipliers (ADMM) has emerged as a powerful technique for large-scale structured optimization. Despite many recent results on the convergence properties of ADMM, a quantitative characterization of the impact of the algorithm parameters on the convergence times of the method is still lacking. In this paper we find the optimal algorithm parameters that minimize the convergence factor of the ADMM iterates in the context of ℓ2-regularized minimization and constrained quadratic programming. Numerical examples show that our parameter selection rules significantly outperform existing alternatives in the literature.

Low power, low delay: opportunistic routing meets duty cycling

Traditionally, routing in wireless sensor networks consists of two steps: First, the routing protocol selects a next hop, and, second, the MAC protocol waits for the intended destination to wake up and receive the data. This design makes it difficult to adapt to link dynamics and introduces delays while waiting for the next hop to wake up. In this paper we introduce ORW, a practical opportunistic routing scheme for wireless sensor networks. In a duty-cycled setting, packets are addressed to sets of potential receivers and forwarded by the neighbor that wakes up first and successfully receives the packet. This reduces delay and energy consumption by utilizing all neighbors as potential forwarders. Furthermore, this increases resilience to wireless link dynamics by exploiting spatial diversity. Our results show that ORW reduces radio duty-cycles on average by 50% (up to 90% on individual nodes) and delays by 30% to 90% when compared to the state of the art.

Opportunistic Routing in Low Duty-Cycle Wireless Sensor Networks

Opportunistic routing is widely known to have substantially better performance than unicast routing in wireless networks with lossy links. However, wireless sensor networks are usually duty cycled, that is, they frequently enter sleep states to ensure long network lifetime. This renders existing opportunistic routing schemes impractical, as they assume that nodes are always awake and can overhear other transmissions. In this article we introduce ORW, a practical opportunistic routing scheme for wireless sensor networks. ORW uses a novel opportunistic routing metric, EDC, that reflects the expected number of duty-cycled wakeups that are required to successfully deliver a packet from source to destination. We devise distributed algorithms that find the EDC-optimal forwarding and demonstrate using analytical performance models and simulations that EDC-based opportunistic routing results in significantly reduced delay and improved energy efficiency compared to traditional unicast routing. In addition, we evaluate the performance of ORW in both simulations and testbed-based experiments. Our results show that ORW reduces radio duty cycles on average by 50% (up to 90% on individual nodes) and delays by 30% to 90% when compared to the state-of-the-art.

Multi-Step Gradient Methods for Networked Optimization

This paper explores the use of accelerated gradient methods in networked optimization. Optimal algorithm parameters and associated convergence rates are derived for distributed resource allocation and consensus problems, and the practical performance of the accelerated gradient algorithms are shown to outperform alternatives in the literature. Since the optimal parameters for the accelerated gradient method depends on upper and lower bounds of the Hessian, we study how errors in these estimates influence the convergence rate of the algorithm. This analysis identifies, among other things, cases where erroneous estimates of the Hessian bounds cause the accelerated method to have slower convergence than the corresponding (non-accelerated) gradient method. An application to Internet congestion control illustrates these issues.

Accelerated gradient methods for networked optimization

This paper explores the use of accelerated gradient methods in networked optimization. Optimal algorithm parameters and associated convergence rates are derived for distributed resource allocation and consensus problems, and the practical performance of the accelerated gradient algorithms are shown to outperform alternatives in the literature. Since the optimal parameters for the accelerated gradient method depends on upper and lower bounds of the Hessian, we study how errors in these estimates influence the convergence rate of the algorithm. This analysis identifies, among other things, cases where erroneous estimates of the Hessian bounds cause the accelerated method to have slower convergence than the corresponding (non-accelerated) gradient method. An application to Internet congestion control illustrates these issues.

An analytical model of delay in multi-hop wireless ad hoc networks

Several analytical models of different wireless networking schemes such as wireless LANs and meshes have been reported in the literature. To the best of our knowledge, all these models fail to address the accurate end-to-end delay analysis of multi-hop wireless networks under unsaturated traffic condition considering the hidden and exposed terminal situation. In an effort to gain deep understanding of delay, this paper firstly proposes a new analytical model to predict accurate media access delay by obtaining its distribution function in a single wireless node. The interesting point of having the media access delay distribution is its generality that not only enables us to derive the average delay which has been reported in almost most of the previous studies as a special case but also facilitates obtaining higher moments of delay such as variance and skewness to capture the QoS parameters such as jitters in recently popular multimedia applications. Secondly, using the obtained single node media access delay distribution, we extend our modeling approach to investigate the delay in multi-hop networks. Moreover, probabilities of collisions in both hidden and exposed terminal conditions have been calculated. The validity of the model is demonstrated by comparing results predicted by the analytical model against those obtained through simulation experiments.

Optimal scaling of the ADMM algorithm for distributed quadratic programming

This paper addresses the optimal scaling of the ADMM method for distributed quadratic programming. Scaled ADMM iterations are first derived for generic equality-constrained quadratic problems and then applied to a class of distributed quadratic problems. In this setting, the scaling corresponds to the step-size and the edge-weights of the underlying communication graph. We optimize the convergence factor of the algorithm with respect to the step-size and graph edge-weights. Explicit analytical expressions for the optimal convergence factor and the optimal step-size are derived. Numerical simulations illustrate our results.

A metric for opportunistic routing in duty cycled wireless sensor networks

Opportunistic routing is widely known to have substantially better performance than traditional unicast routing in wireless networks with lossy links. However, wireless sensor networks are heavily duty-cycled, i.e. they frequently enter deep sleep states to ensure long network life-time. This renders existing opportunistic routing schemes impractical, as they assume that nodes are always awake and can overhear other transmissions. In this paper, we introduce a novel opportunistic routing metric that takes duty cycling into account. By analytical performance modeling and simulations, we show that our routing scheme results in significantly reduced delay and improved energy efficiency compared to traditional unicast routing. The method is based on a new metric, EDC, that reflects the expected number of duty cycled wakeups that are required to successfully deliver a packet from source to destination. We devise distributed algorithms that find the EDC-optimal forwarding, i.e. the optimal subset of neighbors that each node should permit to forward its packets. We compare the performance of the new routing with ETX-optimal single path routing in both simulations and testbed-based experiments.

On the Optimal Step-Size Selection for the Alternating Direction Method of Multipliers

The alternating direction method of multipliers is a powerful technique for structured large-scale optimization that has recently found applications in a variety of fields including networked optimization, estimation, compressed sensing and multi-agent systems. While applications of this technique have received a lot of attention, there is a lack of theoretical support for how to set the algorithm parameters, and its step-size is typically tuned experimentally. In this paper we consider three different formulations of the algorithm and present explicit expressions for the step-size that minimizes the convergence rate. We also compare our method with one of the existing step-size selection techniques for consensus applications.

Global convergence of the Heavy-ball method for convex optimization

This paper establishes global convergence and provides global bounds of the rate of convergence for the Heavy-ball method for convex optimization. When the objective function has Lipschitz-continuous gradient, we show that the Cesáro average of the iterates converges to the optimum at a rate of O(1/k) where k is the number of iterations. When the objective function is also strongly convex, we prove that the Heavy-ball iterates converge linearly to the unique optimum. Numerical examples validate our theoretical findings.