Hamid Reza Feyzmahdavian

Exponential Stability of Homogeneous Positive Systems of Degree One With Time-Varying Delays

While the asymptotic stability of positive linear systems in the presence of bounded time delays has been thoroughly investigated, the theory for nonlinear positive systems is considerably less well-developed. This technical note presents a set of conditions for establishing delay-independent stability and bounding the decay rate of a significant class of nonlinear positive systems which includes positive linear systems as a special case. Specifically, when the time delays have a known upper bound, we derive necessary and sufficient conditions for exponential stability of: a) continuous-time positive systems whose vector fields are homogeneous and cooperative and b) discrete-time positive systems whose vector fields are homogeneous and order-preserving. We then present explicit expressions that allow us to quantify the impact of delays on the decay rate and show that the best decay rate of positive linear systems that our bounds provide can be found via convex optimization. Finally, we extend the results to general linear systems with time-varying delays.

Contractive Interference Functions and Rates of Convergence of Distributed Power Control Laws

The standard interference functions introduced by Yates have been very influential on the analysis and design of distributed power control laws. While powerful and versatile, the framework has some drawbacks: the existence of fixed-points has to be established separately, and no guarantees are given on the rate of convergence of the iterates. This paper introduces contractive interference functions, a slight reformulation of the standard interference functions that guarantees the existence and uniqueness of fixed-points along with linear convergence of iterates. We show that many power control laws from the literature are contractive and derive, sometimes for the first time, analytical convergence rate estimates for these algorithms. We also prove that contractive interference functions converge when executed totally asynchronously and, under the assumption that the communication delay is bounded, derive an explicit bound on the convergence time penalty due to increased delay. Finally, we demonstrate that although standard interference functions are, in general, not contractive, they are all para-contractions with respect to a certain metric. Similar results for two-sided scalable interference functions are also derived.

ASYMPTOTIC STABILITY AND DECAY RATES OF HOMOGENEOUS POSITIVE SYSTEMS WITH BOUNDED AND UNBOUNDED DELAYS

There are several results on the stability of nonlinear positive systems in the presence of time delays. However, most of them assume that the delays are constant. This paper considers time-varying, possibly unbounded, delays and establishes asymptotic stability and bounds the decay rate of a significant class of nonlinear positive systems which includes positive linear systems as a special case. Specifically, we present a necessary and sufficient condition for delay-independent stability of continuous-time positive systems whose vector fields are cooperative and homogeneous. We show that global asymptotic stability of such systems is independent of the magnitude and variation of the time delays. For various classes of time delays, we are able to derive explicit expressions that quantify the decay rates of positive systems. We also provide the corresponding counterparts for discrete-time positive systems whose vector fields are nondecreasing and homogeneous.

A delayed proximal gradient method with linear convergence rate

This paper presents a new incremental gradient algorithm for minimizing the average of a large number of smooth component functions based on delayed partial gradients. Even with a constant step size, which can be chosen independently of the maximum delay bound and the number of objective function components, the expected objective value is guaranteed to converge linearly to within some ball around the optimum. We derive an explicit expression that quantifies how the convergence rate depends on objective function properties and algorithm parameters such as step-size and the maximum delay. An associated upper bound on the asymptotic error reveals the trade-off between convergence speed and residual error. Numerical examples confirm the validity of our results.

Stability and Performance of Continuous-Time Power Control in Wireless Networks

This paper develops a comprehensive stability analysis framework for general classes of continuous-time power control algorithms under heterogeneous time-varying delays. Our first set of results establish global asymptotic stability of power control laws involving two-sided scalable interference functions, and include earlier work on standard interference functions as a special case. We then consider contractive interference functions and demonstrate that the associated continuous-time power control laws always have unique fixed points which are exponentially stable, even under bounded heterogeneous time-varying delays. For this class of interference functions, we present explicit bounds on the decay rate that allow us to quantify the impact of delays on the convergence time of the algorithm. When interference functions are linear, we also prove that contractivity is necessary and sufficient for exponential stability of continuous-time power control algorithms with time-varying delays. Finally, numerical simulations illustrate the validity of our theoretical results.

Global convergence of the Heavy-ball method for convex optimization

This paper establishes global convergence and provides global bounds of the rate of convergence for the Heavy-ball method for convex optimization. When the objective function has Lipschitz-continuous gradient, we show that the Cesáro average of the iterates converges to the optimum at a rate of O(1/k) where k is the number of iterations. When the objective function is also strongly convex, we prove that the Heavy-ball iterates converge linearly to the unique optimum. Numerical examples validate our theoretical findings.

An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization

Mini-batch optimization has proven to be a powerful paradigm for large-scale learning. However, the state of the art mini-batch algorithms assume synchronous operation or cyclic update orders. When worker nodes are heterogeneous (due to different computational capabilities, or different communication delays), synchronous and cyclic operations are inefficient since they will leave workers idle waiting for the slower nodes to complete their work. We propose an asynchronous mini-batch algorithm for regularized stochastic optimization problems that eliminates idle waiting and allows workers to run at their maximal update rates. We show that the time necessary to compute an ϵ-optimal solution is asymptotically O(1/ϵ2), and the algorithm enjoys near-linear speedup if the number of workers is O(1/√ϵ). Theoretical results are confirmed in real implementations on a distributed computing infrastructure.

Asymptotic stability and decay rates of positive linear systems with unbounded delays

There are several results on the stability analysis of positive linear systems in the presence of constant or time-varying delays. However, most existing results assume that the delays are bounded. This paper studies the stability of discrete-time positive linear systems with unbounded delays. We provide a set of easily verifiable necessary and sufficient conditions for delay-independent stability of positive linear systems subject to a general class of heterogeneous time-varying delays. For two particular classes of unbounded delays, explicit expressions that bound the decay rate of the system are presented. We demonstrate that the best bound on the decay rate that our results can guarantee can be found via convex optimization. Finally, the validity of the results is demonstrated via a numerical example.

On the rate of convergence of continuous-time linear positive systems with heterogeneous time-varying delays

In this work, a set of conditions are presented for establishing exponential stability and bounds on the convergence rates of both general and positive linear systems with heterogeneous time-varying delays. First, a sufficient condition for delay-independent exponential stability of general linear systems is derived. When the time delays have a known upper bound, we present an explicit expression that bounds the decay rate of the system. We demonstrate that the best decay rate that our bound can guarantee can be easily found via convex optimization techniques. Finally, for positive linear systems, we show that the stability condition that we have developed is also necessary. The validity of the results is demonstrated via numerical examples.

Asymptotic and exponential stability of general classes of continuous-time power control laws in wireless networks

This paper develops a comprehensive stability analysis framework for continuous-time power control algorithms in wireless networks under bounded time-varying communication delays. Our first set of results establish global asymptotic stability of power control laws involving two-sided scalable interference functions, and include earlier work on standard interference functions as a special case. We then consider contractive interference functions and demonstrate that the associated continuous-time power control laws always have unique fixed points, which are exponentially stable even in the presence of bounded heterogeneous time-varying delays. For this class of interference functions, we derive an explicit bound on the decay rate that allows us to quantify the impact of delays on the convergence time of the algorithm. Numerical simulations illustrate our theoretical results.