Asymptotic convergence of constrained primal-dual dynamics
aa r X i v : . [ m a t h . O C ] O c t Asymptotic convergenceof constrained primal-dual dynamics
Ashish Cherukuri a , Enrique Mallada b , Jorge Cort´es a a Department of Mechanical and Aerospace Engineering, University of California, SanDiego, CA 92093, USA b Department of Computational and Mathematical Sciences, California Institute ofTechnology, Pasadena, CA 91125, USA
Abstract
This paper studies the asymptotic convergence properties of the primal-dualdynamics designed for solving constrained concave optimization problems us-ing classical notions from stability analysis. We motivate the need for thisstudy by providing an example that rules out the possibility of employingthe invariance principle for hybrid automata to study asymptotic conver-gence. We understand the solutions of the primal-dual dynamics in theCaratheodory sense and characterize their existence, uniqueness, and con-tinuity with respect to the initial condition. We use the invariance principlefor discontinuous Caratheodory systems to establish that the primal-dual op-timizers are globally asymptotically stable under the primal-dual dynamicsand that each solution of the dynamics converges to an optimizer.
Keywords: primal-dual dynamics; constrained optimization; saddle points;discontinuous dynamics; Caratheodory solutions
1. Introduction
The (constrained) primal-dual dynamics is a widespread continuous-timealgorithm for determining the primal and dual solutions of an inequalityconstrained convex (or concave) optimization problem. This dynamics, first
Email addresses: [email protected] (Ashish Cherukuri), [email protected] (Enrique Mallada), [email protected] (Jorge Cort´es)
Preprint submitted to Elsevier October 9, 2015 ntroduced in the pioneering works [1, 2], has been used in multiple ap-plications, including network resource allocation problems for wireless sys-tems [3, 4, 5] and distributed stabilization and optimization of power net-works [6, 7, 8, 9].Our objective in this paper is to provide a rigorous treatment of the con-vergence analysis of the primal-dual dynamics using classical notions fromstability analysis. Since this dynamics has a discontinuous right-hand side,the standard Lyapunov or LaSalle-based stability results for nonlinear sys-tems, see e.g. [10], are not directly applicable. This observation is at thebasis of the direct approach to establish convergence taken in [1], where theevolution of the distance of the solution of the primal-dual dynamics to an ar-bitrary primal-dual optimizer is approximated using power series expansionsand its monotonic evolution is concluded by analyzing the local behavioraround a saddle point of the terms in the series. Instead, [3] takes an indi-rect approach to establish convergence, modeling the primal-dual dynamicsas a hybrid automaton as defined in [11], and invoking a generalized LaSalleInvariance Principle to establish asymptotic convergence. However, the hy-brid automaton that corresponds to the primal-dual dynamics is in generalnot continuous, thereby not satisfying a key requirement of the invarianceprinciple stated in [11], and invalidating this route to establish convergence.The first contribution of this paper is an example that illustrates this point.Our second contribution is an alternative proof strategy to arrive at the sameconvergence results of [3].For the problem setup, we consider an inequality constrained concave op-timization problem described by continuously differentiable functions withlocally Lipschitz gradients. Since the primal-dual dynamics has a discontin-uous right-hand side, we specify the notion of solution in the Caratheodorysense (note that this does not necessarily preclude the study of other notionsof solution). We show that the primal-dual dynamics is a particular case ofa projected dynamical system and, using results from [12], we establish thatCaratheodory solutions exist, are unique, and are continuous with respect tothe initial condition. Using these properties, we show that the omega-limitset of any solution of the primal-dual dynamics is invariant under the dynam-ics. Finally, we employ the invariance principle for Caratheodory solutionsof discontinuous dynamical systems from [13] to show that the primal-dualoptimizers are globally asymptotically stable under the primal-dual dynam-ics and that each solution of the dynamics converges to an optimizer. We2elieve the use of classical notions of stability and Lyapunov methods pro-vides a conceptually simple and versatile approach that can also be invokedin characterizing other properties of the dynamics.The paper is organized as follows. Section 2 presents basic notation andpreliminary notions on discontinuous dynamical systems. Section 3 intro-duces the primal-dual dynamics and motivates with an example the needfor a convergence analysis with classical stability tools. Section 4 presentsthe main convergence results. Finally, Section 5 gathers our conclusions andideas for future work.
2. Preliminaries
This section introduces notation and basic concepts about discontinuousand projected dynamical systems.
We let R , R ≥ , R > , and Z ≥ be the set of real, nonnegative real, positivereal, and positive integer numbers, respectively. We denote by k · k the 2-norm on R n . The open ball of radius δ > x ∈ R n is representedby B δ ( x ). Given x ∈ R n , x i denotes the i -th component of x . For x, y ∈ R n , x ≤ y if and only if x i ≤ y i for all i ∈ { , . . . , n } . We use the shorthandnotation n = (0 , . . . , ∈ R n . For a real-valued function V : R n → R and α >
0, we denote the sublevel set of V by V − ( ≤ α ) = { x ∈ R n | V ( x ) ≤ α } .For scalars a, b ∈ R , the operator [ a ] + b is defined as[ a ] + b = ( a, if b > , max { , a } , if b = 0 . For vectors a, b ∈ R n , [ a ] + b denotes the vector whose i -th component is [ a i ] + b i , i ∈ { , . . . , n } . For a set S ∈ R n , its interior, closure, and boundary aredenoted by int( S ), cl( S ), and bd( S ), respectively. Given two sets X and Y ,a set-valued map f : X ⇒ Y associates to each point in X a subset of Y . Amap f : R n → R m is locally Lipschitz at x ∈ R n if there exist δ x , L x > k f ( y ) − f ( y ) k ≤ L x k y − y k for any y , y ∈ B δ x ( x ). If f is locallyLipschitz at every x ∈ K ⊂ R n , then we simply say that f is locally Lipschitzon K . The map f is Lipschitz on K ⊂ R n if there exists a constant L > k f ( x ) − f ( y ) k ≤ L k x − y k for any x, y ∈ K . Note that if f is locallyLipschitz on R n , then it is Lipschitz on every compact set K ⊂ R n . The map f is locally bounded if for each x ∈ R n there exists constants M x , ǫ x > k f ( y ) k ≤ M x for all y ∈ B ǫ x ( x ). Here we present basic concepts on discontinuous dynamical systems fol-lowing [13, 14]. Let f : R n → R n be Lebesgue measurable and locallybounded and consider the differential equation˙ x = f ( x ) . (1)A map γ : [0 , T ) → R n is a (Caratheodory) solution of (1) on the interval[0 , T ) if it is absolutely continuous on [0 , T ) and satisfies ˙ γ ( t ) = f ( γ ( t ))almost everywhere in [0 , T ). A set S ⊂ R n is invariant under (1) if everysolution starting from any point in S remains in S . For a solution γ of (1)defined on the time interval [0 , ∞ ), the omega-limit set Ω( γ ) is defined byΩ( γ ) = { y ∈ R n | ∃{ t k } ∞ k =1 ⊂ [0 , ∞ ) with lim k →∞ t k = ∞ and lim k →∞ γ ( t k ) = y } . If the solution γ is bounded, then Ω( γ ) = ∅ by the Bolzano-Weierstrass theo-rem [15]. These notions allow us to characterize the asymptotic convergenceproperties of the solutions of (1) via invariance principles. Given a continu-ously differentiable function V : R n → R , the Lie derivative of V along (1)at x ∈ R n is L f V ( x ) = ∇ V ( x ) ⊤ f ( x ). The next result is a simplified versionof [13, Proposition 3] which is sufficient for our convergence analysis later. Proposition 2.1. (Invariance principle for discontinuous Caratheodory sys-tems):
Let
S ∈ R n be compact and invariant. Assume that, for each point x ∈ S , there exists a unique solution of (1) starting at x and that its omega-limit set is invariant too. Let V : R n → R be a continuously differentiablemap such that L f V ( x ) ≤ for all x ∈ S . Then, any solution of (1) startingat S converges to the largest invariant set in cl( { x ∈ S | L f V ( x ) = 0 } ) .2.3. Projected dynamical systems Projected dynamical systems are a particular class of discontinuous dy-namical systems. Here, following [12], we gather some basic notions that4ill be useful later to establish continuity with respect to the initial condi-tion of the solutions of the primal-dual dynamics. Let
K ⊂ R n be a closedconvex set. Given a point y ∈ R n , the (point) projection of y onto K isproj K ( y ) = argmin z ∈K k z − y k . Note that proj K ( y ) is a singleton and themap proj K is Lipschitz on R n with constant L = 1 [16, Proposition 2.4.1].Given x ∈ K and v ∈ R n , the (vector) projection of v at x with respectto K is Π K ( x, v ) = lim δ → + proj K ( x + δv ) − xδ . Given a vector field f : R n → R n and a closed convex polyhedron K ⊂ R n ,the associated projected dynamical system is˙ x = Π K ( x, f ( x )) , x (0) ∈ K , (2)Note that, at any point x in the interior of K , we have Π K ( x, f ( x )) = f ( x ).At any boundary point of K , the projection operator restricts the flow ofthe vector field f such that the solutions of (2) remain in K . Therefore, ingeneral, (2) is a discontinuous dynamical system. The next result summa-rizes conditions under which the (Caratheodory) solutions of the projectedsystem (2) exist, are unique, and continuous with respect to the initial con-dition. Proposition 2.2. (Existence, uniqueness, and continuity with respect tothe initial condition [12, Theorem 2.5]):
Let f : R n → R n be Lipschitz on aclosed convex polyhedron K ⊂ R n . Then, (i) (existence and uniqueness): for any x ∈ K , there exists a uniquesolution t x ( t ) of the projected system (2) with x (0) = x definedover the domain [0 , ∞ ) , (ii) (continuity with respect to the initial condition): given a sequence ofpoints { x k } ∞ k =1 ⊂ K with lim k →∞ x k = x , the sequence of solutions { t γ k ( t ) } ∞ k =1 of (2) with γ k (0) = x k for all k , converge to the solution t γ ( t ) of (2) with γ (0) = x uniformly on every compact set of [0 , ∞ ) .
3. Problem statement
This section reviews the primal-dual dynamics for solving constrainedoptimization problems and justifies the need to rigorously characterize its5onvergence properties. Consider the concave optimization problem on R n ,maximize f ( x ) , (3a)subject to g ( x ) ≤ m , (3b)where the continuously differentiable functions f : R n → R and g : R n → R m are strictly concave and convex, respectively, and have locally Lipschitzgradients. The Lagrangian of the problem (3) is given as L ( x, λ ) = f ( x ) − λ ⊤ g ( x ) , (4)where λ ∈ R m is the Lagrange multiplier corresponding to the inequalityconstraint (3b). Note that the Lagrangian is concave in x and convex (infact linear) in λ . Assume that the Slater’s conditions is satisfied for theproblem (3), that is, there exists x ∈ R n such that g ( x ) < m . Under thisassumption, the duality gap between the primal and dual optimizers is zeroand a point ( x ∗ , λ ∗ ) ∈ R n × R m ≥ is a primal-dual optimizer of (3) if and onlyif it is a saddle point of L over the domain R n × R m ≥ , i.e., L ( x, λ ) ≤ L ( x ∗ , λ ∗ ) and L ( x ∗ , λ ) ≥ L ( x ∗ , λ ∗ ) , for all x ∈ R n and λ ∈ R m ≥ . For convenience, we denote the set of saddlepoints of L (equivalently the primal-dual optimizers) by X × Λ ⊂ R n × R m .Note that since f is strictly concave, the set X is a singleton. Furthermore,( x ∗ , λ ∗ ) is a primal-dual optimizer if and only if it satisfies the followingKarush-Kuhn-Tucker (KKT) conditions (cf. [17, Chapter 5]), ∇ f ( x ∗ ) − m X i =1 ( λ ∗ ) i ∇ g i ( x ∗ ) = 0 , (5a) g ( x ∗ ) ≤ m , λ ∗ ≥ m , λ ⊤∗ g ( x ∗ ) = 0 . (5b)Given this characterization of the solutions of the optimization problem, itis natural to consider the primal-dual dynamics on R n × R m ≥ to find them˙ x = ∇ x L ( x, λ ) = ∇ f ( x ) − m X i =1 λ i ∇ g i ( x ) , (6a)˙ λ = [ −∇ λ L ( x, λ )] + λ = [ g ( x )] + λ . (6b)6hen convenient, we use the notation X p-d : R n × R m ≥ → R n × R m to referto the dynamics (6). Given that the primal-dual dynamics is discontinuous,we consider solutions in the Caratheodory sense. The reason for this is that,with this notion of solution, a point is an equilibrium of (6) if and only if itsatisfies the KKT conditions (5).Our objective is to establish that the solutions of (6) exist and asymp-totically converge to a solution of the concave optimization problem (3) us-ing classical notions and tools from stability analysis. Our motivation forthis aim comes from the conceptual simplicity and versatility of Lyapunov-like methods and their amenability for performing robustness analysis andstudying generalizations of the dynamics. One way of tackling this prob-lem, see e.g., [3], is to interpret the dynamics as a state-dependent switchedsystem, formulate the latter as a hybrid automaton as defined in [11], andthen employ the invariance principle for hybrid automata to characterize itsasymptotic convergence properties. However, this route is not valid in gen-eral because one of the key assumptions required by the invariance principlefor hybrid automata is not satisfied by the primal-dual dynamics. The nextexample justifies this claim. Example 3.1. (The hybrid automaton corresponding to the primal-dual dy-namics is not continuous):
Consider the concave optimization problem (3)on R with f ( x ) = − ( x − and g ( x ) = x −
1, whose set of primal-dualoptimizers is X × Λ = { (1 , } . The associated primal-dual dynamics takesthe form ˙ x = − x − − xλ, (7a)˙ λ = [ x − + λ . (7b)We next formulate this dynamics as a hybrid automaton as defined in [11,Definition II.1]. The idea to build the hybrid automaton is to divide thestate space R × R ≥ into two domains over which the vector field (7) iscontinuous. To this end, we define two modes represented by the discretevariable q , taking values in Q = { , } . The value q = 1 represents the modewhere the projection in (7b) is active and q = 2 represents the mode whereit is not. Formally, the projection is active at ( x, λ ) if [ g ( x )] + λ = g ( x ), i.e, λ = 0 and g ( x ) <
0. The hybrid automaton is then given by the collection H = ( Q, X, f,
Init , D, E, G, R ), where Q = { q } is the set of discrete variables,7aking values in Q ; X = { x, λ } is the set of continuous variables, takingvalues in X = R × R ≥ ; the vector field f : Q × X → T X is defined by f (1 , ( x, λ )) = (cid:20) − x − − xλ (cid:21) ,f (2 , ( x, λ )) = (cid:20) − x − − xλx − (cid:21) ;Init = X is the set of initial conditions; D : Q ⇒ X specifies the domain ofeach discrete mode, D (1) = ( − , × { } , D (2) = X \ D (1) , i.e., the dynamics is defined by the vector field ( x, λ ) → f (1 , ( x, λ )) over D (1) and by ( x, λ ) → f (2 , ( x, λ )) over D (2); E = { (1 , , (2 , } is the set ofedges specifying the transitions between modes; the guard map G : Q ⇒ X specifies when a solution can jump from one mode to the other, G (1 ,
2) = { (1 , , ( − , } , G (2 ,
1) = ( − , × { } , i.e., G ( q, q ′ ) is the set of points where a solution jumps from mode q tomode q ′ ; and, finally, the reset map R : Q × X ⇒ X specifies that the stateis preserved after a jump from one mode to another, R ((1 , , ( x, λ )) = R ((2 , , ( x, λ )) = { ( x, λ ) } . We are now ready to show that the hybrid automaton is not continuous inthe sense defined by [11, Definition III.3]. This notion plays a key role in thestudy of omega-limit sets and their stability, and is in fact a basic assump-tion of the invariance principle developed in [11, Theorem IV.1]. Roughlyspeaking, H is continuous if two executions of H starting close to one an-other remain close to one another. An execution of H consists of a tuple( τ, q, x ), where τ is a hybrid time trajectory (a sequence of intervals spec-ifying where mode transitions and continuous evolution take place), q is amap that gives the discrete mode of the execution at each interval of τ , and x is a set of differentiable maps that represent the evolution of the contin-uous state of the execution along intervals of τ . A necessary condition fortwo executions to “remain close” is to have the time instants of transitionsin their mode for the executions (if there are any) close to one another.8 igure 1: An illustration depicting the vector field (7) in the range ( x, λ ) ∈ [0 , . × [0 , . x (0) , λ (0)) with x (0) < λ (0) > λ > t ′ when ( x ( t ′ ) , λ ( t ′ )) = (1 , To disprove the continuity of H , it is enough then to show that there ex-ist two executions that start arbitrarily close and yet experience their firstmode transitions at time instants that are not arbitrarily close. Select aninitial condition ( x (0) , λ (0)) ∈ (0 , × (0 , ∞ ) that gives rise to a solutionof (7) that remains in the set (0 , × (0 , ∞ ) for a finite time interval (0 , t ′ ), t ′ >
0, satisfies ( x ( t ′ ) , λ ( t ′ )) = (1 , , ∞ ) × (0 , ∞ )for some finite time interval ( t ′ , T ), T > t ′ . The existence of such a solutionbecomes clear by plotting the vector field (7), see Figure 1. Note that byconstruction, this also corresponds to an execution of the hybrid automaton H that starts and remains in domain D (2) for the time interval [0 , T ] andso it does not encounter any jumps in its discrete mode. Specifically, forthis execution, the hybrid time trajectory is the interval [0 , T ], the discretemode q is always 2 and the continuous state evolves as t ( x ( t ) , λ ( t )). Fur-ther, by observing the vector field, we deduce that in every neighborhoodof ( x (0) , λ (0)), there exists a point (˜ x (0) , ˜ λ (0)) such that a solution of (7) t (˜ x ( t ) , ˜ λ ( t )) starting at (˜ x (0) , ˜ λ (0)) reaches the set (0 , × { } in finitetime t >
0, remains in (0 , × { } for a finite time interval [ t , t ], and thenenters the set (1 , ∞ ) × (0 , ∞ ) upon reaching the point (1 , x < x (0) and ˜ λ < λ (0). The execution of H corresponding9o this solution starts in D (2), enters D (1) in finite time t , and returns to D (2) at time t . Specifically, the hybrid time trajectory consists of three in-tervals { [0 , t ] , [ t , t ] , [ t , T ′ ] } , where we assume T ′ > t . The discrete mode q takes value 2 for the interval [0 , t ], 1 for the interval [ t , t ], and 2 for theinterval [ t , T ′ ]. The continuous state t (˜ x ( t ) , ˜ λ ( t )) takes the same valuesas the solution of (7) explained above. Thus, the value of the discrete vari-able representing the mode of the execution switches from 2 to 1 and backto 2, whereas the execution corresponding to the solution of (7) starting at( x (0) , λ (0)) never switches mode. This shows that the hybrid automaton isnot continuous. • Interestingly, even though the hybrid automaton H described in Exam-ple 3.1 is not continuous, one can infer from Figure 1 that two solutions of (7)remain close to each other if they start close enough. This suggests that con-tinuity with respect to the initial condition might hold provided this notionis formalized the way it is done for traditional nonlinear systems (and not asdone for hybrid automata where both discrete and continuous states have tobe aligned). The next section shows that this in fact is the case. This, alongwith the existence and uniqueness of solutions, allows us to characterize theasymptotic convergence properties of the primal-dual dynamics.
4. Convergence analysis of primal-dual dynamics
In this section we show that the solutions of the primal-dual dynam-ics (6) asymptotically converge to a solution of the constrained optimizationproblem (3). Our proof strategy is to employ the invariance principle forCaratheodory solutions of discontinuous dynamical systems stated in Propo-sition 2.1. Our first step is then to verify that all its hypotheses hold.We start by stating a useful monotonicity property of the primal-dualdynamics with respect to the set of primal-dual optimizers X × Λ. Thisproperty can be found in [1, 3] and we include here its proof for completeness.
Lemma 4.1. (Monotonicity of the primal-dual dynamics with respect toprimal-dual optimizers):
Let ( x ∗ , λ ∗ ) ∈ X × Λ and define V : R n × R m → R ≥ , V ( x, λ ) = 12 (cid:0) k x − x ∗ k + k λ − λ ∗ k (cid:1) . (8)10 hen L X p-d V ( x, λ ) ≤ for all ( x, λ ) ∈ R n × R m ≥ . Proof.
By definition of L X p-d V (cf. Section 2.2), we have L X p-d V ( x, λ ) = ( x − x ∗ ) ⊤ ∇ x L ( x, λ ) + ( λ − λ ∗ ) ⊤ [ −∇ λ L ( x, λ )] + λ = ( x − x ∗ ) ⊤ ∇ x L ( x, λ ) − ( λ − λ ∗ ) ⊤ ∇ λ L ( x, λ )+ ( λ − λ ∗ ) ⊤ ([ −∇ λ L ( x, λ )] + λ + ∇ λ L ( x, λ )) . Since L is concave in x and convex in λ , applying the first order condition ofconcavity and convexity for the first two terms of the above expression yieldsthe following bound L X p-d V ( x, λ ) ≤ L ( x, λ ) − L ( x ∗ , λ ) + L ( x, λ ∗ ) − L ( x, λ )+ ( λ − λ ∗ ) ⊤ ([ −∇ λ L ( x, λ )] + λ + ∇ λ L ( x, λ ))= L ( x ∗ , λ ∗ ) − L ( x ∗ , λ ) + L ( x, λ ∗ ) − L ( x ∗ , λ ∗ )+ ( λ − λ ∗ ) ⊤ ([ −∇ λ L ( x, λ )] + λ + ∇ λ L ( x, λ )) . Define the shorthand notation M = L ( x ∗ , λ ∗ ) − L ( x ∗ , λ ), M = L ( x, λ ∗ ) − L ( x ∗ , λ ∗ ), and M = ( λ − λ ∗ ) ⊤ ([ −∇ λ L ( x, λ )] + λ + ∇ λ L ( x, λ )), so that the aboveinequality reads L X p-d V ( x, λ ) ≤ M + M + M . Since λ ∗ is a minimizer of the map λ → L ( x ∗ , λ ) over the domain R m ≥ and x ∗ is a maximizer of the map x → L ( x, λ ∗ ), we obtain M , M ≤
0. Replacing −∇ λ L ( x, λ ) = g ( x ), one can write M = P mi =1 T i , where for each i , T i = ( λ i − ( λ ∗ ) i )([ g i ( x )] + λ i − g i ( x )) . If λ i >
0, then [ g i ( x )] + λ i = g i ( x ) and so T i = 0. If λ i = 0, then λ i − ( λ ∗ ) i ≤ g i ( x )] + λ i − g i ( x ) ≥
0, which implies that T i ≤
0. Therefore, we get M ≤
0, and the result follows. (cid:3)
Next, we show that the primal-dual dynamics can be written as a pro-jected dynamical system.
Lemma 4.2. (Primal-dual dynamics as a projected dynamical system):
Theprimal-dual dynamics can be written as a projected dynamical system. roof. Consider the vector field X : R n × R m → R n × R m defined by X ( x, λ ) = (cid:20) ∇ x L ( x, λ ) −∇ λ L ( x, λ ) (cid:21) . (9)We wish to show that X p-d ( x, λ ) = Π R n × R m ≥ (( x, λ ) , X ( x, λ )) for all ( x, λ ) ∈ R n × R m ≥ . To see this, note that the maps X p-d and X take the samevalues over int( R n × R m ≥ ) = R n × R m> . Now consider any point ( x, λ ) ∈ bd( R n × R m ≥ ). Let I ⊂ { , . . . , m } be the set of indices for which λ i = 0 and( −∇ λ L ( x, λ )) i <
0. Then, there exist ˜ δ > δ ∈ [0 , ˜ δ ) andfor any j ∈ { , . . . , n + m } , we have(proj R n × R m ≥ (( x, λ ) + δX ( x, λ ))) j = ( , if j − n ∈ I, ( x, λ ) j + δ ( X ( x, λ )) j , otherwise . Consequently, using the definition of the projection operator, cf. Section 2.3,we get (Π R n × R m ≥ (( x, λ ) , X ( x, λ ))) j = ( , if j − n ∈ I, ( X ( x, λ )) j , otherwise , which implies X p-d ( x, λ ) = Π R n × R m ≥ (( x, λ ) , X ( x, λ )) for all ( x, λ ) ∈ bd( R n × R m ≥ ). This concludes the proof. (cid:3) Next, we use Lemmas 4.1 and 4.2 to show the existence, uniqueness, andcontinuity of the solutions of X p-d starting from R n × R m ≥ . Our proof strategyconsists of using Lemma 4.2 and Proposition 2.2 to conclude the result. Aminor technical hurdle in this process is ensuring the Lipschitz property ofthe vector field (9), the projection of which on R n × R m ≥ is X p-d . We tacklethis by using the monotonicity property of the primal-dual dynamics statedin Lemma 4.1 implying that a solution of X p-d (if it exists) remains in abounded set, which we know explicitly. This further implies that, given astarting point, there exists a bounded set such that the values of the vectorfield outside this set do not affect the solution starting at that point andhence, the vector field can be modified at the outside points without lossof generality to obtain the Lipschitz property. We make this constructionexplicit in the proof. 12 emma 4.3. (Existence, uniqueness, and continuity of solutions of the primal-dual dynamics): Starting from any point ( x, λ ) ∈ R n × R m ≥ , a unique solution t γ ( t ) of the primal-dual dynamics X p-d exists and remains in ( R n × R m ≥ ) ∩ V − ( ≤ V ( x, λ )) . Moreover, if a sequence of points { ( x k , λ k ) } ∞ k =1 ⊂ R n × R m ≥ converge to ( x, λ ) as k → ∞ , then the sequence of solutions { t γ k ( t ) } ∞ k =1 of X p-d starting at these points converge uniformly to the solution t γ ( t ) on every compact set of [0 , ∞ ) . Proof.
Consider ( x (0) , λ (0)) ∈ R n × R m ≥ and let ǫ >
0. Define V = V ( x (0) , λ (0)), where V is given in (8), and let W ǫ = V − ( ≤ V + ǫ ). Note that W ǫ is convex, compact, and V − ( ≤ V ) ⊂ int( W ǫ ). Let X W ǫ : R n × R m → R n × R m be a vector field defined as follows: equal to X on W ǫ and, for any( x, λ ) ∈ ( R n × R m ) \ W ǫ , X W ǫ ( x, λ ) = X (proj W ǫ ( x, λ )) . The vector field X W ǫ is Lipschitz on the domain R n × R m . To see this, notethat X is Lipschitz on the compact set W ǫ with some Lipschitz constant K > f and g have locally Lipschitz gradients. Let ( x , λ ) , ( x , λ ) ∈ R n × R m . Then, k X W ǫ ( x , λ ) − X W ǫ ( x , λ ) k = k X (proj W ǫ ( x , λ )) − X (proj W ǫ ( x , λ )) k≤ K k proj W ǫ ( x , λ ) − proj W ǫ ( x , λ ) k≤ K k ( x , λ ) − ( x , λ ) k . The last inequality follows from the Lipschitz property of the map proj W ǫ (cf. Section 2.3).Next, we employ Proposition 2.2 to establish the existence, uniqueness,and continuity with respect to the initial condition of the solutions of theprojected dynamical system, X W ǫ p-d , associated with X W ǫ and R n × R m ≥ . Ourproof then concludes by showing that in fact all solutions of the projectedsystem X W ǫ p-d starting in W ǫ ∩ R n × R m ≥ are in one-to-one correspondence withthe solutions of X p-d starting in W ǫ ∩ R n × R m ≥ . Let X W ǫ p-d : R n × R m ≥ → R n × R m be the map obtained by projecting X W ǫ with respect to R n × R m ≥ , X W ǫ p-d ( x, λ ) = Π R n × R m ≥ (( x, λ ) , X W ǫ ( x, λ )) , for all ( x, λ ) ∈ R n × R m ≥ . Since X p-d is the projection of X with respect to13 n × R m ≥ , we deduce that X W ǫ p-d = X p-d over the set W ǫ ∩ R n × R m ≥ . Since X W ǫ is Lipschitz, following Proposition 2.2, we obtain that starting from any pointin R n × R m ≥ , a unique solution of X W ǫ p-d exists over [0 , ∞ ) and is continuouswith respect to the initial condition. Consider any solution t (˜ x ( t ) , ˜ λ ( t )) of X W ǫ p-d that starts in W ǫ ∩ R n × R m ≥ . Note that since the solution is absolutelycontinuous and V is continuously differentiable, the map t V (˜ x ( t ) , ˜ λ ( t ))is differentiable almost everywhere on [0 , ∞ ), and hence ddt V (˜ x ( t ) , ˜ λ ( t )) = L X W ǫ p-d V (˜ x ( t ) , ˜ λ ( t )) , almost everywhere on [0 , ∞ ). From Lemma 4.1 and the fact that L X W ǫ p-d V and L X p-d V are the same over W ǫ ∩ R n × R m ≥ , we conclude that V is non-increasingalong the solution. This means the solution remains in the set W ǫ ∩ R n × R m ≥ .Finally, since X W ǫ p-d and X p-d are same on W ǫ ∩ R n × R m ≥ , we conclude that t (˜ x ( t ) , ˜ λ ( t )) is also a solution of X p-d . Therefore, starting at any point in W ǫ ∩ R n × R m ≥ , a solution of X p-d exists. Using Lemma 4.1, one can showthat, if a solution of X p-d that starts from a point in W ǫ ∩ R n × R m ≥ exists,then it remains in W ǫ ∩ R n × R m ≥ and so is a solution of X W ǫ p-d . This, combinedwith the uniqueness of solutions of X W ǫ p-d , implies that a unique solution of X p-d exists starting from any point in W ǫ ∩ R n × R m ≥ . In particular, thisis true for the point ( x (0) , λ (0)). Finally, from the continuity of solutionsof X W ǫ p-d and the one-to-one correspondence of solutions of X p-d and X W ǫ p-d starting W ǫ ∩ R n × R m ≥ , we conclude the continuity with respect to initialcondition for solutions of X p-d starting in V − ( x (0) , λ (0)). Since ( x (0) , λ (0))is arbitrary, the result follows. (cid:3) The next result states the invariance of the omega-limit set of any solu-tion of the primal-dual dynamics. This ensures that all hypotheses of theinvariance principle for Caratheodory solutions of discontinuous dynamicalsystems, cf. Proposition 2.1, are satisfied.
Lemma 4.4. (Omega-limit set of solution of primal-dual dynamics is in-variant):
The omega-limit set of any solution of the primal-dual dynamicsstarting from any point in R n × R m ≥ is invariant under (6) . The proof of Lemma 4.4 follows the same line of argumentation that theproof of invariance of omega-limit sets of solutions of locally Lipschitz vector14elds, cf. [10, Lemma 4.1]. We are now ready to establish our main result,the asymptotic convergence of the solutions of the primal-dual dynamics toa solution of the constrained optimization problem.
Theorem 4.5. (Convergence of the primal-dual dynamics to a primal-dualoptimizer):
The set of primal-dual solutions of (3) is globally asymptoticallystable on R n × R m ≥ under the primal-dual dynamics (6) , and the convergenceof each solution is to a point. Proof.
Let ( x ∗ , λ ∗ ) ∈ X × Λ and consider the function V defined in (8). For δ >
0, consider the compact set S = V − ( ≤ δ ) ∩ ( R n × R m ≥ ). From Lemma 4.3,we deduce that a unique solution of X p-d exists starting from any point in S ,which remains in S . Moreover, from Lemma 4.4, the omega-limit set of eachsolution starting from any point in S is invariant. Finally, from Lemma 4.1, L X p-d V ( x, λ ) ≤ x, λ ) ∈ S . Therefore, Proposition 2.1 implies thatany solution of X p-d staring in S converges to the largest invariant set M contained in cl( Z ), where Z = { ( x, λ ) ∈ S | L X p-d V ( x, λ ) = 0 } . From theproof of Lemma 4.1, L X p-d V ( x, λ ) = 0 implies L ( x ∗ , λ ∗ ) − L ( x ∗ , λ ) = 0 ,L ( x, λ ∗ ) − L ( x ∗ , λ ∗ ) = 0 , ( λ i − ( λ ∗ ) i )([ g i ( x )] + λ i − g i ( x )) = 0 , for all i ∈ { , . . . , m } . Since f is strictly concave, so is the function x L ( x, λ ∗ ) and thus L ( x, λ ∗ ) = L ( x ∗ , λ ∗ ) implies x = x ∗ . The equality L ( x ∗ , λ ∗ ) − L ( x ∗ , λ ) = 0 implies λ ⊤ g ( x ∗ ) = 0. Therefore Z = { ( x, λ ) ∈ S | x = x ∗ , λ ⊤ g ( x ∗ ) = 0 } is closed.Let ( x ∗ , λ ) ∈ M ⊂ Z . The solution of X p-d starting at ( x ∗ , λ ) remains in M (and hence in Z ) only if ∇ f ( x ∗ ) − P mi =1 λ i ∇ g i ( x ∗ ) = 0. This implies that( x ∗ , λ ) satisfies the KKT conditions (5) and hence, M ⊂ X × Λ. Since theinitial choice δ > X × Λ is globallyasymptotically stable on R n × R m ≥ . Finally, we note that convergence isto a point in X × Λ. This is equivalent to saying that the omega-limit setΩ( x, λ ) ⊂ X × Λ of any solution t ( x ( t ) , λ ( t )) of X p-d is a singleton.This fact follows from the definition of omega-limit set and the fact that, byLemma 4.1, primal-dual optimizers are Lyapunov stable. This concludes theproof. (cid:3) Remark 4.6. (Alternative proof strategy via evolution variational inequal- ties): We briefly describe here an alternative proof strategy to the one wehave used here to establish the asymptotic convergence of the primal-dualdynamics. The Caratheodory solutions of the primal-dual dynamics can alsobe seen as solutions of an evolution variational inequality (EVI) problem [18].Then, one can show that the resulting EVI problem has a unique solutionstarting from each point in R n × R m ≥ , which moreover remains in R n × R m ≥ .With this in place, the LaSalle Invariance Principle [18, Theorem 4] for thesolutions of the EVI problem can be applied to conclude the convergence tothe set of primal-dual optimizers. • Remark 4.7. (Primal-dual dynamics with gains):
In power network opti-mization problems [7, 8, 9] and network congestion control problems [19, 20],it is common to see generalizations of the primal-dual dynamics involvinggain matrices. Formally, these dynamics take the form˙ x = K ∇ x L ( x, λ ) , (10a)˙ λ = K [ −∇ λ L ( x, λ )] + λ , (10b)where K ∈ R n × n and K ∈ R m × m are diagonal, positive definite matrices.In such cases, the analysis performed here can be replicated following thesame steps but using instead the Lyapunov function V ′ ( x, λ ) = 12 (( x − x ∗ ) ⊤ K − ( x − x ∗ ) + ( λ − λ ∗ ) ⊤ K − ( λ − λ ∗ )) , to establish the required monotonicity and convergence properties of (10). •
5. Conclusions
We have considered the primal-dual dynamics for a constrained con-cave optimization problem and established the asymptotic convergence ofits Caratheodory solutions to a primal-dual optimizer using classical notionsfrom stability theory. Our technical approach has employed results from pro-jected dynamical systems to establish existence, uniqueness, and continuityof the solutions, and the invariance principle for discontinuous Caratheodorysystems to characterize their asymptotic convergence. We have also shownby means of a counterexample how a proof strategy based on interpreting theprimal-dual dynamics as a hybrid automaton is not valid in general because16f the lack of continuity (understood in the hybrid sense) of the solutions.The technical approach presented in the paper opens up the possibility ofrigorously characterizing the robustness properties of the primal-dual dy-namics against unmodeled dynamics, disturbances, and noise. Motivated byapplications to power networks, we also plan to explore the design of discon-tinuous dynamics that can find the solutions to semidefinite programs andquadratically constrained quadratic programs.
6. Acknowledgements
The first and the third author wish to thank Dr. Bahman Gharesifard andDr. Dean Richert for fruitful discussions on the primal-dual dynamics. Thisresearch was partially supported by NSF Award ECCS-1307176, Los AlamosNational Lab through a DoE grant, and DTRA under grant 11376437.
References [1] K. Arrow, L. Hurwitz, H. Uzawa, Studies in Linear and Non-Linear Program-ming, Stanford University Press, Stanford, California, 1958.[2] T. Kose, Solutions of saddle value problems by differential equations, Econo-metrica 24 (1) (1956) 59–70.[3] D. Feijer, F. Paganini, Stability of primal-dual gradient dynamics and appli-cations to network optimization, Automatica 46 (2010) 1974–1981.[4] J. Chen, V. K. N. Lau, Convergence analysis of saddle point problems in timevarying wireless systems – control theoretical approach, IEEE Transactionson Signal Processing 60 (1) (2012) 443–452.[5] A. Ferragut, F. Paganini, Network resource allocation for users with multipleconnections: fairness and stability, IEEE/ACM Transactions on Networking22 (2) (2014) 349–362.[6] X. Ma, N. Elia, A distributed continuous-time gradient dynamics approach forthe active power loss minimizations, in: Allerton Conf. on Communications,Control and Computing, Monticello, IL, 2013, pp. 100–106.[7] C. Zhao, U. Topcu, N. Li, S. Low, Design and stability of load-side primaryfrequency control in power systems, IEEE Transactions on Automatic Control59 (5) (2014) 1177–1189.
8] E. Mallada, C. Zhao, S. Low, Optimal load-side control for frequency reg-ulation in smart grids, in: Allerton Conf. on Communications, Control andComputing, Monticello, IL, 2014, pp. 731–738.[9] X. Zhang, A. Papachristodoulou, Distributed dynamic feedback control forsmart power networks with tree topology, in: American Control Conference,Portland, OR, 2014, pp. 1156–1161.[10] H. K. Khalil, Nonlinear Systems, 3rd Edition, Prentice Hall, 2002.[11] J. Lygeros, K. H. Johansson, S. N. Simi´c, J. Zhang, S. S. Sastry, Dynami-cal properties of hybrid automata, IEEE Transactions on Automatic Control48 (1) (2003) 2–17.[12] A. Nagurney, D. Zhang, Projected Dynamical Systems and Variational In-equalities with Applications, Vol. 2 of International Series in Operations Re-search and Management Science, Kluwer Academic Publishers, Dordrecht,The Netherlands, 1996.[13] A. Bacciotti, F. Ceragioli, Nonpathological Lyapunov functions and discon-tinuous Caratheodory systems, Automatica 42 (3) (2006) 453–458.[14] J. Cort´es, Discontinuous dynamical systems - a tutorial on solutions, nons-mooth analysis, and stability, IEEE Control Systems Magazine 28 (3) (2008)36–73.[15] W. Rudin, Principles of Mathematical Analysis, McGraw-Hill, 1953.[16] F. H. Clarke, Optimization and Nonsmooth Analysis, Canadian MathematicalSociety Series of Monographs and Advanced Texts, Wiley, 1983.[17] S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge UniversityPress, 2009.[18] B. Brogliato, D. Goeleven, The Krakovskii-LaSalle invariance principle for aclass of unilateral dynamical systems, Mathematics of Control, Signals andSystems 17 (1) (2005) 57–76.[19] J. T. Wen, M. Arcak, A unifying passivity framework for network flow control,IEEE Transactions on Automatic Control 49 (2) (2004) 162–174.[20] S. H. Low, F. Paganini, J. C. Doyle, Internet congestion control, IEEE ControlSystems Magazine 22 (1) (2002) 28–43.8] E. Mallada, C. Zhao, S. Low, Optimal load-side control for frequency reg-ulation in smart grids, in: Allerton Conf. on Communications, Control andComputing, Monticello, IL, 2014, pp. 731–738.[9] X. Zhang, A. Papachristodoulou, Distributed dynamic feedback control forsmart power networks with tree topology, in: American Control Conference,Portland, OR, 2014, pp. 1156–1161.[10] H. K. Khalil, Nonlinear Systems, 3rd Edition, Prentice Hall, 2002.[11] J. Lygeros, K. H. Johansson, S. N. Simi´c, J. Zhang, S. S. Sastry, Dynami-cal properties of hybrid automata, IEEE Transactions on Automatic Control48 (1) (2003) 2–17.[12] A. Nagurney, D. Zhang, Projected Dynamical Systems and Variational In-equalities with Applications, Vol. 2 of International Series in Operations Re-search and Management Science, Kluwer Academic Publishers, Dordrecht,The Netherlands, 1996.[13] A. Bacciotti, F. Ceragioli, Nonpathological Lyapunov functions and discon-tinuous Caratheodory systems, Automatica 42 (3) (2006) 453–458.[14] J. Cort´es, Discontinuous dynamical systems - a tutorial on solutions, nons-mooth analysis, and stability, IEEE Control Systems Magazine 28 (3) (2008)36–73.[15] W. Rudin, Principles of Mathematical Analysis, McGraw-Hill, 1953.[16] F. H. Clarke, Optimization and Nonsmooth Analysis, Canadian MathematicalSociety Series of Monographs and Advanced Texts, Wiley, 1983.[17] S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge UniversityPress, 2009.[18] B. Brogliato, D. Goeleven, The Krakovskii-LaSalle invariance principle for aclass of unilateral dynamical systems, Mathematics of Control, Signals andSystems 17 (1) (2005) 57–76.[19] J. T. Wen, M. Arcak, A unifying passivity framework for network flow control,IEEE Transactions on Automatic Control 49 (2) (2004) 162–174.[20] S. H. Low, F. Paganini, J. C. Doyle, Internet congestion control, IEEE ControlSystems Magazine 22 (1) (2002) 28–43.