A complete characterization on the robust isolated calmness of the nuclear norm regularized convex optimization problems
AA complete characterization on the robust isolated calmness of thenuclear norm regularized convex optimization problems
Ying Cui ∗ , Defeng Sun † February 19, 2017
Abstract
In this paper, we provide a complete characterization on the robust isolated calmness ofthe Karush-Kuhn-Tucker (KKT) solution mapping for convex constrained optimization problemsregularized by the nuclear norm function. This study is motivated by the recent work in [8], wherethe authors show that under the Robinson constraint qualification at a local optimal solution,the KKT solution mapping for a wide class of conic programming problem is robustly isolatedcalm if and only if both the second order sufficient condition (SOSC) and the strict Robinsonconstraint qualification (SRCQ) are satisfied. Based on the variational properties of the nuclearnorm function and its conjugate, we establish the equivalence between the primal/dual SOSCand the dual/primal SRCQ. The derived results lead to several equivalent characterizations of therobust isolated calmness of the KKT solution mapping and add insights to the existing literatureon the stability of the nuclear norm regularized convex optimization problems.
Keywords. robust isolated calmness, nuclear norm, second order sufficient condition, strict Robinsonconstraint qualification
AMS subject classifications:
Let X and Y be two finite dimensional Euclidean spaces. Let G : X Ñ Y be a set-valued mapping.The graph of G is defined as gph G : “ tp x, y q P X ˆ Y | y P G p x qu . Consider any p ¯ x, ¯ y q P gph G .The mapping G is said to be isolated calm at ¯ x for ¯ y if there exist a constant κ ą X of ¯ x and Y of ¯ y such that G p x q X Y Ă t ¯ y u ` κ } x ´ ¯ x } B Y , @ x P X , (1)where B Y is the unit ball in Y (cf. e.g., [9, 3.9 (3I)]). The mapping G is said to be robustly isolatedcalm at ¯ x for ¯ y if (1) holds and G p x q X Y ‰ H for any x P X [8, Definition 2]. ∗ Department of Mathematics, National University of Singapore, 10 Lower Kent Ridge Road, Singapore( [email protected] ). † Department of Mathematics, National University of Singapore, 10 Lower Kent Ridge Road, Singapore( [email protected] ). This research is supported in part by the Academic Research Fund under Grant R-146-000-207-112. a r X i v : . [ m a t h . O C ] F e b n this paper, we are interested in characterizing the robust isolated calmness of the Karush-Kuhn-Tucker (KKT) solution mapping associated with the following nuclear norm regularized convexoptimization problem: min X h p F X q ` x C, X y ` } X } ˚ s.t. A X ´ b P Q , (2)where the function h : R d Ñ R is twice continuously differentiable on dom h , which is assumed to bea non-empty open convex set, and is also essentially strictly convex (i.e., h is strictly convex on everyconvex subset of dom B h ), F : R m ˆ n Ñ R d and A : R m ˆ n Ñ R e are linear operators, C P R m ˆ n and b P R e are given data, Q Ď R e is a nonempty convex polyhedral cone, } ¨ } ˚ denotes the nuclearnorm function in R m ˆ n , i.e., the sum of all the singular values of a given matrix, and m, n, d, e arenon-negative integers. The nuclear norm regularizer has been extensively used in diverse disciplinesdue to its ability in promoting a low rank solution. See the references [18, 19, 2, 3, 13, 16] for asample of applications.The concept of the isolated calmness is of fundamental importance in variational analysis. Themonograph [9] by Dontchev and Rockafellar contains a comprehensive study on this subject. In theoptimization field, the isolated calmness of the KKT solution mapping, besides its own interest insensitivity analysis and perturbation theory, can be employed to investigate the convergence ratesof primal dual type methods, including the proximal augmented Lagrangian method [15] and thealternating direction method of multipliers [10].Obviously problem (2) can be equivalently formulated as the following conic programming prob-lem min X,t h p F X q ` x C, X y ` t s.t. A X ´ b P Q , p X, t q P epi } ¨ } ˚ , (3)where epi } ¨ } ˚ denotes the epigraph of the function } ¨ } ˚ . Since epi } ¨ } ˚ is not a polyhedral set,the sensitivity results in the conventional nonlinear programming are not applicable for problem (3).Recently, some progress has been achieved in characterizing the isolated calmness of KKT solutionmappings for problems involving non-polyhedral functions. For example, Zhang and Zhang [21] showthat for the nonlinear semidefinite programming, the second order sufficient condition (SOSC) andthe strict Robinson constraint qualification (SRCQ) at a local optimal solution together are sufficientfor the KKT solution mapping to be isolated calm. Adding to this result, Han, Sun and Zhang [10]show that the SRCQ is also necessary to ensure the isolated calmness of the KKT solution mappingfor such problems. In [11], Liu and Pan extend the aforementioned results to problems constrainedby the epigraph of the Ky Fan k -norm function. The most recent work of Ding, Sun and Zhang [8]indicates that under the Robinson constraint qualification (RCQ) at a local optimal solution, theKKT solution mapping for a wide class of conic programming is robustly isolated calm at the originfor a KKT point if and only if both the SOSC and the SRCQ hold at the reference point.The results developed in [8] can be directly applied to problem (3). Thus, by examining therelationships between the SOSCs, the (strict) RCQs as well as the robust isolated calmness of thesolution mappings corresponding to problem (2) and problem (3), we are able to extend the work in[8] to the nuclear norm regularized convex optimization problem (2). Additionally, due to the specialstructure of problem (2) and its dual, we could provide more insightful characterizations about theisolated calmness of the KKT solution mapping. Note that the Lagrangian dual of problem (2) is2iven by max y,w,S ´x b, y y ´ δ Q ˝ p y q ´ h ˚ p w q s.t. A ˚ y ` F ˚ w ` S ` C “ , } S } ď , (4)where A ˚ and F ˚ are the adjoint of A and F , respectively, h ˚ p¨q is the conjugate function of h and } ¨ } denotes the spectral norm in R m ˆ n , i.e., the largest singular value of a given matrix. We shallshow that for problem (2), its SRCQ is equivalent to the SOSC of problem (4), and conversely, itsSOSC is equivalent to the SRCQ of problem (4). Armed with these results, we are led to a deepunderstanding of the robust isolated calmness for the KKT solution mapping for the nuclear normregularized convex optimization problems.The remaining parts of this paper are organized as follows. In the next section, we providesome preliminary results on variational analysis. In Section 3, we demonstrate how to translate theresults of set-constrained problems in [8] into the language of nonsmooth optimization problems.In particular, this translation provides us a characterization of the robust isolated calmness of theKKT solution mapping for the nuclear norm regularized convex optimization problems. Section 4is devoted to the study on the variational properties of the nuclear norm function. The derivedresults play an important role in our subsequent analysis. In Section 5, we establish the equivalencebetween the SOSC for the primal/dual problem and the SRCQ for the dual/primal problem. Thisestablishment enables us to describe the robust isolated calmness of the KKT solution mapping forproblem (2) via several equivalent conditions.The following notation will be used throughout our paper. • For a given positive integer p , we use S p to denote the linear space of all p ˆ p real symmetricmatrices, S p ` the cone of all p ˆ p positive semidefinite matrices and S p ´ the cone of all p ˆ p negative semidefinite matrices. • For a given proper closed convex function θ : X Ñ p´8 , `8s , we use dom θ to denote itseffective domain, epi θ to denote its epigraph, θ ˚ to denote its conjugate, B θ to denote itssubdifferential and Prox θ to denote its proximal mapping, all as in standard convex analysis [14]. • Let D Ď R m ˆ n be a non-empty closed convex set. We write δ D p¨q as the indicator functionover D , i.e., δ D p X q “ X P D , and δ D p X q “ 8 if X R D . We write Π D p¨q as the metricprojection onto D , i.e., Π D p X q : “ arg min Y t} Y ´ X } | Y P D u for X P R m ˆ n . • For any z P R m , we denote Diag p z q as the m ˆ m diagonal matrix whose i -th diagonal entry is z i for i “ , . . . , m . Let α Ď t , ..., m u and β Ď t , ..., n u be two index sets. For any Z P R m ˆ n ,we write Z α as the sub-matrix of Z by removing all the columns of Z not in α , and Z αβ tobe the | α | ˆ | β | sub-matrix of Z obtained by removing all the rows of Z not in α and all thecolumns of Z not in β . • Let O n be the set of all n ˆ n orthogonal matrices. For any X P R m ˆ n , let σ p X q P R m bethe vector of all singular values of X with the entries σ p X q ě σ p X q ě . . . ě σ m p X q , and let O m,n p X q be the set of paired orthogonal matrices satisfying the singular value decompositionof X , i.e., O m,n p X q “ tp U, V q P O m ˆ O n | X “ U r Diag p σ p X qq s V T u . Preliminaries
In this section, we gather some knowledge on variational analysis that will be used in our subsequentdevelopments. One can refer to the monograph [1] of Bonnans and Shapiro for detailed discussionson this subject.A cone Q Ď Y is said to be pointed if y P Q and ´ y P Q implies that y “
0. Let Q Ď Y be apointed convex closed cone. The closed convex set K Ď X is said to be C -cone reducible at x P K tothe cone Q , if there exist an open neighborhood W Ď X of x and a twice continuously differentiablemapping Ξ : W Ñ Y such that: (i) Ξ p x q “ P Y ; (ii) the derivative mapping Ξ p x q : X Ñ Y is onto;(iii) K X W “ t x P W | Ξ p x q P Q u . We say that K is C -cone reducible if K is C -cone reducible atevery x P K . A proper closed convex function θ : X Ñ p´8 , is said to be C -cone reducible at x P dom θ if epi θ is C -cone reducible at p x, θ p x qq . Moreover, θ is said to be C -cone reducible if it is C -cone reducible at every x P dom θ .Given a subset K Ď X and x P K , the contingent cone and the inner tangent cone of K at x aredefined as T K p x q “ lim sup t Ó K ´ xt and T i K p x q “ lim inf t Ó K ´ xt , respectively. If K is convex, K ´ xt is a monotone decreasing function of t such that T K p x q “ T i K p x q for any x P K [1, Proposition 2.55]. In this case, both T K p x q and T i K p x q are called the tangent coneof K at x . Given x P K and a direction d P X , define the inner and outer second order tangent setsat x in the direction d as T i, K p x ; d q : “ lim inf t Ó K ´ x ´ td t and T K p x ; d q : “ lim sup t Ó K ´ x ´ td t , respectively. The inner and outer second order tangent sets not necessarily coincide in general,even if the set K is closed and convex. However, if K is a C -cone reducible convex set, then T i, K p x ; d q “ T K p x ; d q [1, Proposition 3.136].For a given function θ : X Ñ p´8 , `8s , the lower and upper directional epiderivatives of θ at x P dom θ in the direction h P X are defined as θ Ó´ p x ; h q : “ lim inf t Ó h Ñ h θ p x ` th q ´ θ p x q t and θ Ó` p x ; h q : “ sup t t n uP Σ ˜ lim inf n Ñ8 h Ñ h θ p x ` t n h q ´ θ p x q t n ¸ , respectively, where Σ denotes the set of positive real sequences t t n u converging to 0. The contingentand inner tangent cone of epi θ are closely related to the lower and upper directional epiderivative of θ [1, proposition 2.58]. Specifically, for any x P dom θ , T epi θ ` x, θ p x q ˘ “ epi θ Ó´ p x ; ¨q , T i epi θ ` x, θ p x q ˘ “ epi θ Ó` p x ; ¨q . (5)4ne can observe from the above equations that if θ is a closed convex function, then θ Ó´ p x ; ¨q “ θ Ó` p x ; ¨q for any x P dom θ . In this case we say that θ is directionally epidifferentiable at x and write thecommon value as θ Ó p x ; ¨q . If θ Ó` p x ; d q and θ Ó´ p x ; d q are finite for x P dom θ and d P X , we also definethe following lower and upper second order epiderivatives for w P X : θ Ó ´ p x ; d, w q : “ lim inf t Ó w w θ p x ` td ` t w q ´ θ p x q ´ tθ Ó´ p x ; d q t ,θ Ó ` p x ; d, w q : “ sup t n P Σ ˜ lim inf n Ñ8 w w θ p x ` t n d ` t n w q ´ θ p x q ´ t n θ Ó` p x ; d q t n ¸ . Similarly to (5), the inner and outer second order tangent sets of epi θ are closely related to the lowerand upper second order epiderivative of θ [1, proposition 3.41]. Specifically, for any x P dom θ and d P X , if θ Ó` p x ; d q and θ Ó´ p x ; d q are finite, then T i, θ pp x, θ p x qq ; p d, θ Ó` p x ; d qq “ epi θ ÓÓ` p x ; d, ¨q , T θ pp x, θ p x qq ; p d, θ Ó´ p x ; d qq “ epi θ ÓÓ´ p x ; d, ¨q . (6) Consider the following canonical perturbation of a general class of nonsmooth optimization problems(not necessarily convex): min t f p x q ` θ p x q ´ x δ , x y | g p x q ` δ P P u , (7)where f : X Ñ R and g : X Ñ Y are twice continuously differentiable functions, P Ď Y is a closedconvex set, θ : X Ñ p´8 , `8s is a closed proper convex function, and δ P X and δ P Y areperturbation parameters.Note that problem (7) can be equivalently written as the following optimization problem:min t f p x q ` t ´ x δ , x y | g p x q ` δ P P , p x, t q P K u , (8)where K : “ epi θ is a closed convex set. The constraint qualifications and SOSCs for problem (8)have been extensively explored in [1, Section 3], through the study of the (second order) tangent setsof P and K at a stationary point. In the following, by employing the equations in (5) and (6), wereduce these properties to the (second order) directional epiderivatives of θ . This reduction leads toa direct approach to the sensitivity analysis of the nonsmooth optimization problem (7).For notational simplicity, denote Z : “ X ˆ R ˆ Y ˆ X ˆ R . Let p δ , δ q P X ˆ Y be given. We saythat p ¯ x, ¯ t q is a feasible solution to problem (8) if p ¯ x, ¯ t q P p F p δ , δ q : “ tp x, t q P X ˆ R | g p x q ` δ P P , p x, t q P K u . For any p x, t, y, z, τ q P Z , the Lagrangian function of (8) with p δ , δ q “ L p x, t ; y, z, τ q : “ f p x q ` t ` x y, g p x qy ` x z, x y ` tτ. For any given p δ , δ q P X ˆ Y , the KKT optimality condition for problem (8) is $’&’% δ “ ∇ x L p x, t, y, z, τ q “ ∇ f p x q ` ∇ g p x q y ` z, ∇ t L p x, t, y, z, τ q “ ` τ “ ,y P N P p g p x q ` δ q , p z, τ q P N K p x, t q , p x, t, y, z, τ q P Z , (9)5here N C p s q denotes the normal cone of a given convex set C at s P C . Let p S KKT : X ˆ Y Ñ Z bethe following KKT solution mapping: p S KKT p δ , δ q : “ tp x, t, y, z, τ q P Z | p x, t, y, z, τ q satisfies (9) u , p δ , δ q P X ˆ Y . (10)For any given p δ , δ q P X ˆ Y , we call p ¯ x, ¯ t q a stationary point of problem (7) if there existsa Lagrangian multiplier p ¯ y, ¯ z, ¯ τ q P Y ˆ X ˆ R such that p ¯ x, ¯ t, ¯ y, ¯ z, ¯ τ q P p S KKT p δ , δ q . Denote y M p ¯ x, ¯ t, δ , δ q Ď Y ˆ X ˆ R as the set of all such Lagrangian multipliers p ¯ y, ¯ z, ¯ τ q associated with p ¯ x, ¯ t q . Note from (9) that ¯ τ ” ´ p δ , δ q “ p ¯ x, θ p ¯ x qq ofproblem (8) if ˆ p g p ¯ x q , qp I X , q ˙ p X ˆ R q ` ˆ T P p g p ¯ x qq T K p ¯ x, θ p ¯ x qq ˙ “ ˆ YX ˆ R ˙ , (11)where I X is the identity mapping from X to X . It is known that the RCQ (11) holds at a localoptimal solution p ¯ x, θ p ¯ x qq if and only if x M p ¯ x, θ p ¯ x q , , q is a nonempty, convex and compact set (cf.,e.g., [1, Theorem 3.9 and Proposition 3.17]). The SRCQ is said to hold at p ¯ x, θ p ¯ x qq for p ¯ y, ¯ z, ´ q P x M p ¯ x, θ p ¯ x q , , q if ˆ p g p ¯ x q , qp I X , q ˙ p X ˆ R q ` ˆ T P p g p ¯ x qq X ¯ y K T K p ¯ x, θ p ¯ x qq X p ¯ z, ´ q K ˙ “ ˆ YX ˆ R ˙ . (12)Obviously the SRCQ (12) is stronger than the RCQ (11). It follows from [1, Proposition 4.50] that x M p ¯ x, θ p ¯ x q , , q is a singleton if the SRCQ holds at p ¯ x, θ p ¯ x qq . The critical cone at a feasible point p ¯ x, θ p ¯ x qq for problem (8) takes the form of p C p ¯ x, θ p ¯ x qq : “ tp d , d q P X ˆ R | g p ¯ x q d P T P p g p ¯ x qq , p d , d q P T K ` ¯ x, θ p ¯ x q ˘ , f p ¯ x q d ` d ď u . Furthermore, if p ¯ x, θ p ¯ x qq is a stationary point of problem (8) and there exists p ¯ y, ¯ z, ´ q P x M p ¯ x, θ p ¯ x q , , q ,then p C p ¯ x, θ p ¯ x qq “ tp d , d q P X ˆ R | g p ¯ x q d P T P p g p ¯ x qq , p d , d q P T K ` ¯ x, θ p ¯ x q ˘ , f p ¯ x q d ` d “ u“ tp d , d q P X ˆ R | g p ¯ x q d P T P p g p ¯ x qq X ¯ y K , p d , d q P T K ` ¯ x, θ p ¯ x q ˘ X p ¯ z, ´ q K u . Assume that epi θ is second order regular at p ¯ x, θ p ¯ x qq (see [1, Definition 3.85] for the definition ofthe second order regularity). This assumption is particularly satisfied when θ p¨q “ } ¨ } ˚ , since inthis case epi θ is a C -cone reducible set [5, Proposition 4.3]. Then the SOSC at a stationary point p ¯ x, θ p ¯ x qq for problem (8) with p δ , δ q “ d P p C p ¯ x, θ p ¯ x qqzt u ,sup p ¯ y, ¯ z, ´ qP x M p ¯ x,θ p ¯ x q , , q x d, ∇ p x,t qp x,t q L p ¯ x, θ p ¯ x q , ¯ y, ¯ z, ´ q d y´ σ ` p ¯ z, ´ q , T K pp ¯ x, θ p ¯ x qq ; p g p ¯ x q , q d q ˘ ą , (13)where σ p s, C q : “ sup tx s , s y | s P C u denotes the support function of a given set C at s . The aboveSOSC implies the quadratic growth condition at p ¯ x, θ p ¯ x qq (cf. e.g., [1, Theorem 3.86]), that is, thereexist a constant κ ą N of p ¯ x, θ p ¯ x qq such that f p x q ` t ě f p ¯ x q ` θ p ¯ x q ` κ }p x, t q ´ p ¯ x, θ p ¯ x qq} , @ p x, t q P p F p , q X N. The following proposition, which is taken from [8, Theorem 24], characterizes the robust isolatedcalmness of problem (8) via the SOSC (13) and the SRCQ (12).6 roposition 3.1.
Suppose that p ¯ x, θ p ¯ x qq P X ˆ R is a feasible solution of problem (8) with p δ , δ q “ .Suppose that the RCQ (11) holds at p ¯ x, θ p ¯ x qq . Assume that epi θ is C -cone reducible at p ¯ x, θ p ¯ x qq .Let p ¯ y, ¯ z, ´ q P x M p ¯ x, θ p ¯ x q , , q . Then the following two statements are equivalent to each other:(i) The SOSC (13) holds at p ¯ x, θ p ¯ x qq and the SRCQ (12) holds at p ¯ x, θ p ¯ x qq for p ¯ y, ¯ z, ´ q .(ii) The point p ¯ x, θ p ¯ x qq is a local optimal solution of problem (8) and the KKT solution mapping p S KKT is robustly isolated calm at the origin for p ¯ x, θ p ¯ x q , ¯ y, ¯ z, ´ q . Now we return to the nonsmooth optimization problem (7). Let p δ , δ q P X ˆ Y be given. Wesay that ¯ x is a feasible solution to problem (7) if¯ x P F p δ , δ q : “ t x P dom θ | g p x q ` δ P P u . Denote l : X ˆ Y Ñ R by l p x, y q : “ f p x q ` x g p x q , y y , p x, y q P X ˆ Y . Then the KKT optimality condition takes the form of δ P ∇ x l p x, y q ` B θ p x q ,y P N P p g p x q ` δ q , p x, y q P X ˆ Y . (14)Let S KKT : X ˆ Y Ñ X ˆ Y be the following KKT solution mapping: S KKT p δ , δ q : “ tp x, y q P X ˆ Y | p x, y q satisfies (14) u , p δ , δ q P X ˆ Y . (15)For any given p δ , δ q P X ˆ Y , we call ¯ x a stationary point of problem (7) if there exists a Lagrangianmultiplier ¯ y P Y such that p ¯ x, ¯ y q P S KKT p δ , δ q . Denote M p ¯ x, δ , δ q Ď Y as the set of all suchLagrangian multipliers ¯ y associated with ¯ x .The following proposition establishes the equivalence between the robust isolated calmness ofthe KKT solution mappings with respect to problem (7) and problem (8). Proposition 3.2.
Let p ¯ x, θ p ¯ x qq P X ˆ R be a local optimal solution of problem (8) with x M p ¯ x, θ p ¯ x q , , q ‰H . Let p ¯ y, ¯ z, ´ q P x M p ¯ x, θ p ¯ x q , , q . If the KKT solution mapping p S KKT given in (10) is robustlyisolated calm at the origin for p ¯ x, θ p ¯ x q , ¯ y, ¯ z, ´ q , then the KKT solution mapping S KKT given in (15)is robustly isolated calm at the origin for p ¯ x, ¯ y q . The reverse implication is true if the function θ isLipschitz continuous at ¯ x .Proof. Note from [4, Corollary 2.4.9] that p z, ´ q P N epi θ ` x, θ p x q ˘ ðñ z P B θ p x q , @ x, z P X . Then for any p δ , δ q P X ˆ Y and any p x, t, y, z, ´ q P p S KKT p δ , δ q , we know from (9) and (14) that p x, y q P S KKT p δ , δ q . Thus, the first part of this proposition follows easily from the definition of therobust isolated calmness.Conversely, consider any p δ , δ q P X ˆ Y and p x, y q P S KKT p δ , δ q . Let z “ δ ´ ∇ f p x q ´ ∇ g p x q y .By the similar arguments as above, we have p x, θ p x q , y, z, ´ q P p S KKT p δ , δ q . Since f and g areassumed to be twice continuously differentiable, ∇ f p¨q and ∇ g p¨q are locally Lipschitz continuous at7 x . Then there exists a constant κ ą x and ¯ y ) such that for any x sufficientlyclose to ¯ x , } θ p x q ´ θ p ¯ x q} ď κ } x ´ ¯ x } , } z ´ ¯ z } ď } δ } ` } ∇ f p x q ´ ∇ f p ¯ x q} ` } ∇ g p x q y ´ ∇ g p ¯ x q ¯ y } ď k p} δ } ` } x ´ ¯ x } ` } y ´ ¯ y }q . Consequently, the second assertion of this proposition also follows from the definition of the robustisolated calmness.As mentioned in Section 2, the closed convex function θ p¨q is always directional epidifferentiableat x P dom θ . Then by [14, Theorem 23.2], the KKT optimality condition (14) is equivalent to θ Ó p x ; d q ` x ∇ x l p x, y q ´ δ , d y ě ,y P N P p g p x q ` δ q , @ d P X , where p x, y q P X ˆ Y . Define the critical of the function θ by C θ p x, z q : “ t d P T dom θ p x q | θ Ó p x ; d q “ x d, z yu , p x, z q P dom θ ˆ X . (16)Let p δ , δ q “
0. The RCQ is said to hold at a feasible solution ¯ x of problem (7) if ˆ g p ¯ x q I X ˙ X ` ˆ T P p g p ¯ x qq T dom θ p ¯ x q ˙ “ ˆ YX ˙ . (17)By the equations in (5), the SRCQ is said to hold at a stationary point ¯ x for ¯ y P M p ¯ x, , q if ˆ g p ¯ x q I X ˙ X ` ˆ T P p g p ¯ x qq X ¯ y K C θ p ¯ x, ´ ∇ x l p ¯ x, ¯ y qq ˙ “ ˆ YX ˙ . (18)The critical cone at a feasible solution ¯ x of problem (7) is given by C p ¯ x q : “ d P X | g p ¯ x q d P T P p g p ¯ x qq , d P T dom θ p ¯ x q , f p ¯ x q d ` θ Ó p ¯ x ; d q ď ( . If ¯ x is a stationary point of problem (7) and ¯ y P M p ¯ x, , q , then C p ¯ x q “ d P X | g p ¯ x q d P T P p g p ¯ x qq , d P T dom θ p ¯ x q , f p ¯ x q d ` θ Ó p ¯ x ; d q “ ( “ d P X | g p ¯ x q d P T P p g p ¯ x qq X ¯ y K , d P C θ p ¯ x, ´ ∇ f p ¯ x qq ( . Based on the equations in (6), we know that if θ is C -cone reducible at ¯ x , then the SOSC at ¯ x forproblem (7) with p δ , δ q “ ¯ y P M p ¯ x, , q ! x d, ∇ xx l p ¯ x, ¯ y q d y ´ ψ ˚p ¯ x,d q p´ ∇ x l p ¯ x, ¯ y qq ) ą , @ d P C p ¯ x qzt u , (19)where ψ ˚p x,d q p¨q is the conjugate function of ψ p x,d q p¨q “ θ Ó ´ p x ; d, ¨q for any x P dom θ and any d P X .Combining Proposition 3.1 and Proposition 3.2, we are ready to state the main result of thissection. Proposition 3.3.
Let ¯ x P X be a local optimal solution of problem (7) with p δ , δ q “ . Supposethat the RCQ (17) holds at ¯ x . Assume that θ is C -cone reducible and Lipschitz continuous at ¯ x . Let ¯ y P M p ¯ x, , q . Then the following two statements are equivalent to each other:(i) The SOSC (19) holds at ¯ x and the SRCQ (18) holds at ¯ x for ¯ y .(ii) The point ¯ x is a local optimal solution of problem (7) and the KKT solution mapping S KKT isrobustly isolated calm at the origin for p ¯ x, ¯ y q . Variational analysis of the nuclear norm function
Throughout this section, we denote θ : R m ˆ n Ñ R as the nuclear norm function. Obviouslydom θ “ R m ˆ n . Since the nuclear norm function is convex and globally Lipschitz continuous, for any X P R m ˆ n , both θ Ó´ p X ; ¨q and θ Ó` p X ; ¨q defined in Section 2 coincide with θ p X ; ¨q , the conventionaldirectional derivative of θ at X [1, Section 2.2.3].Let X P R m ˆ n be an arbitrary but fixed point. Suppose that X admits the following singular-value decomposition (SVD): X “ U r Diag p σ p X qq s V T “ U Diag p σ p X qq V T , (20)where U P O m and V “ r V V s P O n are the left and right singular vectors of X with V P R n ˆ m and V P R n ˆp n ´ m q . Define the index sets a : “ t ď i ď m | σ i p X q P p , `8qu , b : “ t ď i ď m | σ i p X q P r , su , c “ t m ` , . . . , n u . (21)Denote the distinct singular values of X that are greater than 1 by ν p X q ą . . . ą ν r p X q ą
1, where r is a non-negative integer. We further divide the sets a and b into the following subsets: a l : “ t i P a | σ i p X q “ ν l p X qu , l “ , . . . , r,b : “ t i P b | σ i p X q “ u , b : “ t i P b | ă σ i p X q ă u , b : “ t i P b | σ i p X q “ u . (22)Let us first review the concept of L¨owner’s operator and its differential properties. Suppose that X P R m ˆ n has the SVD (20). For any scalar function g : R Ñ R , define the corresponding matrixvalued function G by G p X q : “ U r Diag p g p σ p X qq , g p σ p X qq , . . . , g p σ m p X qq s V T . Such a function is called L¨owner’s operator associated with the function g , which is first studied byL¨owner in the context of symmetric matrices [12]. In particular, let φ : R Ñ R be the scalar function φ p x q : “ max t x ´ , u , x P R . It is easy to verify that the proximal mapping of θ can be expressed as:Prox θ p X q “ U r Diag p φ p σ p X qq , . . . , φ p σ m p X qqq s V T , X P R m ˆ n . (23)Clearly Prox θ p¨q can be taken as L¨owner’s operator associated with the function φ . The directionalderivate of Prox θ p¨q can thus be obtained via the general formula regarding the directional derivativeof L¨owner’s operator [7]. Obviously φ is directionally differentiable with the directional derivative φ p x ; d q “ $&% d if x ą , max t d, u if x “ , x ă , x P R , d P R . For any positive integer p , define linear matrix operators S : R p ˆ p Ñ S p and T : R p ˆ p Ñ R p ˆ p by S p X q : “ p X ` X T q , T p X q : “ p X ´ X T q , X P R p ˆ p . (24)9enote Ξ aa : R m ˆ n Ñ R | a |ˆ| a | , Ξ ab : R m ˆ n Ñ R | a |ˆ| b | , Ξ ab : R m ˆ n Ñ R | a |ˆ| b | and Ξ : R m ˆ n Ñ R | a |ˆ| c | as $’’’’’’’’’’&’’’’’’’’’’% pp Ξ aa qp X qq ij : “ σ i p X q ` σ j p X q ´ σ i p X q ` σ j p X q , i “ , , . . . , | a | , j “ , , . . . , | a | , pp Ξ ab qp X qq ij : “ σ i p X q ´ σ i p X q ´ σ j `| a | p X q , i “ , , . . . , | a | , j “ , , . . . , | b | , pp Ξ ab qp X qq ij : “ σ i p X q ´ σ i p X q ` σ j `| a | p X q , i “ , , . . . , | a | , j “ , , . . . , | b | , pp Ξ qp X qq ij : “ σ i p X q ´ σ i p X q , i “ , , . . . , | a | , j “ , , . . . , n ´ m, X P R m ˆ n . Denote Γ : R m ˆ n ˆ R m ˆ n Ñ R | a |ˆ| a | , Γ : R m ˆ n ˆ R m ˆ n Ñ R | a |ˆ| b | , Γ : R m ˆ n ˆ R m ˆ n Ñ R | b |ˆ| a | and Γ : R m ˆ n ˆ R m ˆ n Ñ R | a |ˆ| c | as $’’’’’&’’’’’% Γ p X, H q : “ p S p H qq aa ` Ξ aa p X q ˝ p T p H qq aa , Γ p X, H q : “ Ξ ab p X q ˝ p S p H qq ab ` Ξ ab p X q ˝ p T p H qq ab , Γ p X, H q : “ p Ξ ab p X qq T ˝ p S p H qq ba ` p Ξ ab p X qq T ˝ p T p H qq ba , Γ p X, H q : “ Ξ p X q ˝ H ac , p X, H q P R m ˆ n ˆ R m ˆ n , where ˝ denotes the Hadamard product between two matrices and H “ r H H s with H P R m ˆ m and H P R m ˆp n ´ m q . Then by [7, Theorem 3], the directional derivative of Prox θ p¨q at X P R m ˆ n inthe direction H P R m ˆ n takes the form ofProx θ p X ; H q “ U ¨˚˚˚˚˚˚˚˝ Γ p X, r H q Γ p X, r H q Γ p X, r H q Γ p X, r H q Π S | b |` p S p r H b b qq b ˆ b
00 0 0 b ˆ b ˛‹‹‹‹‹‹‹‚ V T , (25)where r H “ U T HV .In [17], Watson shows that the subdifferential of the nuclear norm function takes the followingform: B θ p X q “ ! U a V Ta ` U b W r V b V s T | W P R | b |ˆp n ´| a |q , } W } ď ) , X P R m ˆ n . (26)Therefore, for any H P R m ˆ n , the directional derivative of θ at X in the direction H is given by θ p X ; H q “ sup S PB θ p X q x H, S y “ tr p U Ta HV a q ` } U Tb H r V b V s} ˚ . (27)Let A “ Prox θ p X q and B “ Prox θ ˚ p X q . Define the critical cone of θ at A for B as C θ p A, B q : “ t H P R m ˆ n | θ p A ; H q “ x H, B yu . (28)Similarly, define the critical cone of θ ˚ at B for A as C θ ˚ p B, A q : “ t H P R m ˆ n | p θ ˚ q p B ; H q “ x H, A yu . (29)10s can be seen from (19), in order to analyze the SOSC for problem (2), one needs to computethe conjugate of the second order epiderivative of θ . This has already been done in [6]. Firstly, itfollows from (23) that σ p A q “ max t σ p X q ´ , u . Specifically, σ i p A q “ σ i p X q ´ , if i P a, i P b. (30)Clearly A has r numbers of nonzero distinct singular values. Denote them by ν p A q ą . . . ą ν r p A q .The index sets a , . . . , a r that depending on X in (22) also provides a partition of p σ i p A qq i P a , i.e., σ i p A q “ ν l p A q , @ i P a l , @ l “ , . . . , r. For l “ , . . . , r , denote Ω al : R m ˆ n ˆ R m ˆ n Ñ R a l ˆ a l asΩ a l p A, H q : “ p S p r H qq Ta l p Diag p σ p A qq ´ ν l p A q I m q : p S p r H qq a l ` p ν l p A qq ´ r H a l c r H Ta l c `p T p r H qq Ta l p´ Diag p σ p A qq ´ ν l p A q I m q : p T p r H qq a l , p A, H q P R m ˆ n ˆ R m ˆ n , where I m is the m ˆ m identity matrix and Z : denotes the Moore-Penrose pseudoinverse of a givenmatrix Z . Then for any H P R m ˆ n , the conjugate of θ p A ; H, ¨q is ψ ˚p A,H q p B q : “ p θ p A ; H, ¨qq ˚ p B q “ r ÿ l “ tr ` Ω a l p A, H q ˘ ` x Diag p σ b p B qq , U Tb HA : HV b y , (31)where σ b p B q “ p σ i p B qq i P b and r H “ r r H r H s “ r U T HV U T HV s .In the following, we present several properties regarding the critical cone of θ and the directionalderivative of Prox θ p¨q . Proposition 4.1.
Suppose that X P R m ˆ n has the singular value decomposition (20). Let the indexsets a, b, a , . . . , a l , b , b , b be given by (21) and (22). Given any H P R m ˆ n , denote r H “ U T HV for p U, V q P O m ˆ n p X q . Denote A “ Prox θ p X q and B “ Prox θ ˚ p X q . Then the following conclusionshold:(i) H P C θ p A, B q if and only if H satisfies r H “ ¨˚˚˚˚˚˚˚˝ r H aa r H ab r H ac r H ba Π S | b |` p r H b b q b ˆ b
00 0 0 b ˆ b ˛‹‹‹‹‹‹‹‚ . (32) (ii) For any D P R m ˆ n , H “ Prox θ p X ; H ` D q if and only if H P C θ p A, B q and x H, D y “ ´ ψ ˚p A,H q p B q , where the function ψ ˚p A,H q p¨q is given in (31).Proof. The result of part (i) can be obtained from [6, proposition 10]. Now we derive (iii). Suppose H “ Prox θ p C ; H ` D q . Denote r H “ r r H , r H s with r H P R m ˆ m and r H P R m ˆp n ´ m q . Direct11omputations of ψ ˚p A,H q p B q given in (31) show that ψ ˚p A,H q p B q “ ÿ ď l,t ď r ´ ν t p A q ´ ν l p A q }p T p r H qq a l a t } ` ÿ ď l ď r ´ ν l p A q }p T p r H qq a l b } ` ÿ ď l ď r ď i ´| a |´| b |ď| b | ˆ p ´ σ i p B qq´ ν l p A q }p S p r H qq a l i } ` p σ i p B q ` q´ ν l p A q }p T p r H qq a l i } ˙ ` ÿ ď l ď r ď i ´| a |´| b |´| b |ď| b | ˆ ´ ν l p A q }p S p r H qq a l i } ` ´ ν l p A q }p T p r H qq a l i } ˙ ` ÿ ď l ď r ´ ν l p A q }p r H c q a l c } . (33)Recall the formula of Prox θ p X ; ¨q given in (25). We deduce that r H “ ¨˚˚˚˚˚˝ r H aa r H ab r H ac r H ba r H b b b ˆ b
00 0 0 b ˆ b ˛‹‹‹‹‹‚ and $’’’’’’’’’’’’’&’’’’’’’’’’’’’% r D a l a t “ ν l p X q ` ν t p X q ´ p T p r H qq a l a t , ď l, t ď r, p r D ab q ij “ σ i p X q ´ p r H ab q ij ´ σ j `| a | p X q σ i p X q ´ p r H ab q ji , i “ , , . . . | a | , j “ , , . . . , | b | , p r D ba q ji “ σ i p X q ´ p r H ab q ji ´ σ j `| a | p X q σ i p X q ´ p r H ab q ij , i “ , , . . . | a | , j “ , , . . . , | b | , p r D ac q ij “ σ i p X q ´ p r H ac q ij , i “ , , . . . , | a | , j “ , , . . . , n ´ m, S | b |` Q r H b b “ S p r H b b q K S p r D b b q P S | b |´ , where r D “ U T DV . Consequently, it follows from part (i) of this proposition that H P C θ p A, B q .Moreover, we have x D, H y “ x ˜ d aa , r H aa y ` x ˜ d ab , r H ab y ` x ˜ d ba , r H ba y ` x ˜ d a , r H a y“ ÿ ď t,l ď r ν l p X q ` ν t p X q ´ }p T p r H qq a l a t } ` ÿ ď l ď r ν l p X q ´ }p T p r H qq a l b } ` ÿ ď l ď r ď i ´| a |´| b |ď| b | ˆ p ´ σ i p X qq ν l p X q ´ }p S p r H qq a l i } ` p σ i p X q ` q ν l p X q ´ }p T p r H qq a l i } ˙ ` ÿ ď l ď r ď i ´| a |´| b |´| b |ď| b | ˆ ν l p X q ´ }p S p r H qq a l i } ` ν l p X q ´ }p T p r H qq a l i } ˙ ` ÿ ď l ď r ν l p X q ´ }p r H q a l c } . ν l p A q “ ν l p X q ´ l “ , . . . , r . Hence, x D, H y “ ´ ψ ˚p A,H q p B q by (33)and the above equation. The converse of this statement can be established by reversing the abovearguments. Proposition 4.2.
Suppose that X P R m ˆ n has the singular value decomposition (20). Let the indexsets a, b, a , . . . , a l , b , b , b be given by (21) and (22). Given any H P R m ˆ n , denote r H “ U T HV for p U, V q P O m ˆ n p X q . Denote A “ Prox θ p X q and B “ Prox θ ˚ p X q . Then the following conclusionshold:(i) H P C θ ˚ p B, A q if and only if H satisfies S p r H b b q P S | b |´ and r H “ ¨˚˚˚˚˚˚˚˚˝ T p r H aa q p r H ab ´ r H Tb a q r H ab r H ab p r H b a ´ r H Tab q r H b b r H b b r H b b r H b a r H b b r H b b r H b b r H b a r H b b r H b b r H b b r H c ˛‹‹‹‹‹‹‹‹‚ , (34) where the linear operators S p¨q and T p¨q are defined in (24).(ii) H P p C θ p A, B qq ˝ if and only if φ ˚p B,H q p A q “ and H P C θ ˚ p B, A q , where φ ˚p B,H q p¨q is the conjugatefunction of φ p B,H q p¨q “ p θ ˚ q p B ; H, ¨q .(iii) H P p C θ ˚ p B, A qq ˝ if and only if ψ ˚p A,H q p B q “ and H P C θ p A, B q . Proof.
Part (i) follows from [6, Proposition 12]. To prove part (ii), we use a result from [6, Proposition16] stating that φ ˚p B,H q p A q “ ψ ˚p A,H q p B q “ H P R m ˆ n , which is furtherequivalent to r H aa P S | a | , r H ab “ r H Tb a , r H ab “ r H Tb a “ , r H ab “ r H Tb a “ , r H ac “ . Then by part (i) of this proposition, one can see that φ ˚p B,H q p A q “ H P C θ ˚ p B, A q imply that r H aa “ , r H ab “ , r H ac “ , r H ba “ , S p r H b b q P S | b |´ . In view of Proposition 4.1, this is equivalent to have H P p C θ p A, B qq ˝ . To prove part (iii), it sufficesto note that either H P p C θ ˚ p B, A qq ˝ or ψ ˚p A,H q p B q “ H P C θ p A, B q is equivalent to r H “ ¨˚˚˚˚˚˚˚˝ S p r H aa q p r H ab ` r H Tb a q p r H b a ` r H Tab q Π S | b |` p r H b b q ˛‹‹‹‹‹‹‹‚ . The proof of this proposition is completed. 13
The robust isolated calmness of the KKT solution mapping
The aim of this section is to show that the SOSC for the primal problem (2) (the dual problem(4)) is in fact equivalent to the SRCQ for the dual problem (4) (the primal problem (2)). Beforeproceeding, we mention that a variation of this result regarding the linear semidefinite programminghas been studied in [20].Following the notation in the previous section, we use θ to denote the nuclear norm function in R m ˆ n . Let Ω P Ď X and Ω D Ď R e ˆ R d ˆ R m ˆ n be the optimal solution sets of the primal problem(2) and the dual problem (4), respectively, both being assumed to be non-empty. It follows from(14) that the KKT optimality condition of problem (2) is given by P F ˚ ∇ h p F X q ` C ` B θ p X q ` A ˚ y,y P N Q p A X ´ b q , p X, y q P R m ˆ n ˆ R e . (35)We write M P p X q Ď R e as the set of Lagrangian multipliers ¯ y associated with X P Ω P , i.e., ¯ y P M P p X q if and only if p X, ¯ y q satisfies (35). Let M D p ¯ y, ¯ w, S q Ď R m ˆ n be the set of Lagrangianmultipliers associated with p ¯ y, ¯ w, S q P Ω D for problem (4), i.e., X P M D p ¯ y, ¯ w, S q if and only if p X, ¯ y, ¯ w, S q solves the following KKT system: A X ´ b P N Q ˝ p y q , F X P B h ˚ p w q ,X P B θ ˚ p S q , “ A ˚ y ` F ˚ w ` S ` C, p y, w, S, X q P R e ˆ R d ˆ R m ˆ n ˆ R m ˆ n . (36)Since h is assumed to be essentially strictly convex, h ˚ is essentially smooth [14, Theorem 26.3].Thus, ∇ h ˚ is locally Lipschitz continuous and directionally differentiable on int p dom h ˚ q . Moreover, B h ˚ p w q “ H whenever w R int p dom h ˚ q [14, Theorem 26.1]. Therefore, if (36) admits at least onesolution, this KKT optimality condition can be equivalently written as A X ´ b P N Q ˝ p y q , F X P ∇ h ˚ p w q ,X P B θ ˚ p S q , “ A ˚ y ` F ˚ w ` S ` C, p y, w, S, X q P R e ˆ R d ˆ R m ˆ n ˆ R m ˆ n . As in Section 3, we consider the canonical perturbation of problem (2) for the sake of subsequentsensitivity analysis: min X h p F X q ` x C, X y ` } X } ˚ ´ x X, δ y s.t. A X ´ b ` δ P Q , where δ P R m ˆ n and δ P R e are perturbation parameters. For any given p δ , δ q P R m ˆ n ˆ R e , theKKT optimality condition then takes the form of δ P F ˚ ∇ h p F X q ` C ` B θ p X q ` A ˚ y,y P N Q p A X ´ b ` δ q , p X, y q P R m ˆ n ˆ R e . (37)Let S KKT : R m ˆ n ˆ R e Ñ R m ˆ n ˆ R e be the following KKT solution mapping: S KKT p δ , δ q : “ tp x, y q P R m ˆ n ˆ R e | p x, y q satisfies (37) u , p δ , δ q P R m ˆ n ˆ R e . (38)The RCQ of problem (2) at a feasible solution X P R m ˆ n is given by A R m ˆ n ` T Q p A X ´ b q “ R e . (39)14et p y, q q P R e ˆ R e satisfy y P N Q p q q . We denote the critical cone of Q at q for y and the criticalcone of Q ˝ at y for q as C Q p q, y q : “ T Q p q q X y K , C Q ˝ p y, q q : “ T Q ˝ p y q X q K . It is easy to verify that p C Q p q, y qq ˝ “ C Q ˝ p y, q q . (40)The following theorem, which is the main result of our paper, demonstrates the equivalencebetween the primal SOSC and the dual SRCQ. Theorem 5.1.
Let X P R m ˆ n be an optimal solution of problem (2) with M P p X q ‰ H . Let ¯ y P M P p X q . Denote S : “ ´ A ˚ ¯ y ´ F ˚ ∇ h p F X q´ C . Then the following two statements are equivalentto each other:(i) The SOSC holds at X for ¯ y with respect to the primal problem (2), i.e., x F H, ∇ h p F X q F H y ´ ψ ˚p X,H q p S q ą , @ H P C p ¯ x qzt u , (41) where C p X q : “ C Q p A X ´ b, ¯ y q X C θ p X, S q . (ii) The SRCQ holds at ¯ y for X with respect to the dual problem (4), i.e., F ˚ R d ` A ˚ C Q ˝ p ¯ y, A X ´ b q ` C θ ˚ p S, X q “ R m ˆ n (42) Proof.
Firstly, let us assume that the statement (i) holds. Denote E : “ F ˚ R d ` A ˚ C Q ˝ p ¯ y, A X ´ b q ` C θ ˚ p S, X q . Suppose on the contrary that E ‰ R m ˆ n . Then cl p E q ‰ R m ˆ n [14, Theorem 6.3]. Hence, there exists D P R m ˆ n but D R cl p E q . Note that cl p E q is a closed convex cone. Let D : “ D ´ Π cl p E q p D q “ Π p cl p E qq ˝ p D q ‰ . Obviously, x H, D y ď H P cl p E q . This implies that F D “ , A D P p C Q ˝ p ¯ y, A X ´ b qq ˝ , D P p C θ ˚ p S, X qq ˝ . Thus, it follows from (40) that A D P C Q p A X ´ b, ¯ y q . From Proposition 4.2, we also have that ψ ˚p X,D q p S q “ , D P C θ p X, S q . Therefore, D P C p X qzt u and x F D, ∇ h p F X q F D y ´ ψ ˚p X,D q p S q “ , which contradicts the assumedSOSC (41) at X .The reverse implication can be proved similarly. Suppose that the SOSC (41) fails to hold at X for ¯ y . Then there exists H P C p X qzt u such that x F H, ∇ h p F X q F H y ´ ψ ˚p X,H q p S q “ . Since h is assumed to be essentially strictly convex, x F H, ∇ h p F X q F H y ą H P R m ˆ n such that F H ‰
0. It also follows from [6, Proposition 16] that ψ ˚p X,H q p S q ď H P R m ˆ n .Consequently, F H “ , ψ ˚p X,H q p S q “ .
15e have from H P C p X qzt u that A H P C Q p A X ´ b, ¯ y q , H P C θ p X, S q . Hence, we deduce from (40) and Proposition 4.2 that H P p A ˚ C Q ˝ p ¯ y, A X ´ b qq ˝ X p C θ ˚ p S, X qq ˝ “ p A ˚ C Q ˝ p ¯ y, A X ´ b q ` C θ ˚ p S, X qq ˝ . By the assumed SRCQ (42) at ¯ y for X , there exist ˜ w P R d and r H P A ˚ C Q ˝ p ¯ y, A X ´ b q ` C θ ˚ p S, X q such that H “ F ˚ ˜ w ` r H . Then x H, H y “ x H, F ˚ ˜ w ` r H y “ x H, r H y ď , which implies H “
0. This contradicts the previous assumption that H ‰
0. The proof is thuscompleted.One can also establish an analogous result by swapping the roles of the primal and dual problemsin Theorem 5.1.
Theorem 5.2.
Let p ¯ y, ¯ w, S q P R e ˆ R d ˆ R m ˆ n be an optimal solution of problem (4) with M D p ¯ y, ¯ w, S q ‰H . Let X P M D p ¯ y, ¯ w, S q . Then the following two statements are equivalent to each other:(i) The SOSC holds at p ¯ y, ¯ w, S q for X with respect to the dual problem (4), i.e., x H w , p ∇ h ˚ q p ¯ w ; H w qy ´ φ ˚p S,H S q p X q ą , @ p H y , H w , H s q P C p ¯ y, ¯ w, S qzt u , (43) where the critical cone C p ¯ y, ¯ w, S q is defined as C p ¯ y, ¯ w, S q : “ " p H y , H w , H S q P R e ˆ R d ˆ R m ˆ n ˇˇˇˇ A ˚ H y ` F ˚ H w ` H S “ ,H y P C Q ˝ p ¯ y, A X ´ b q , H S P C θ ˚ p S, X q * . (ii) The SRCQ holds at X for p ¯ y, ¯ w, S q with respect to the primal problem (2), i.e., ˆ AI R m ˆ n ˙ R m ˆ n ` ˆ C Q p A X ´ b, ¯ y q C θ p X, S q ˙ “ ˆ R e R m ˆ n ˙ . (44) Proof.
With the help of Proposition 4.2, one can establish the assertion in the same fashion as inTheorem 5.1. We omit the details here.Finally, by combining Proposition 3.3, Theorem 5.1 and Theorem 5.2, we are ready to provideseveral equivalent characterizations of the robust isolated calmness of the KKT solution mapping atthe origin for the unique KKT point of problem (2).
Theorem 5.3.
Let X P R m ˆ n be an optimal solution of problem (2) and p ¯ y, ¯ w, S q P R e ˆ R d ˆ R m ˆ n be an optimal solution of problem (4). Assume that the RCQ (39) holds at X . Then the followingstatements are equivalent to each other:(i) The KKT solution mapping S KKT in (38) is robustly isolated calm at the origin for p X, ¯ y q .(ii) The SOSC (41) holds at X for ¯ y with respect to the primal problem (2) and the SRCQ (44) holdsat X for p ¯ y, ¯ w, S q with respect to the primal problem (2). iii) The SOSC (41) holds at X for ¯ y with respect to the primal problem (2) and the SOSC (43)holds at p ¯ y, ¯ w, S q for X with respect to the dual problem (4).(iv) The SRCQ (42) holds at ¯ y for X with respect to the dual problem (4) and the SRCQ (44) holdsat X for p ¯ y, ¯ w, S q with respect to the primal problem (2).(v) The SOSC (43) holds at p ¯ y, ¯ w, S q for X with respect to the dual problem (4) and the SRCQ (42)holds at ¯ y for X with respect to the dual problem (4). It is worth mentioning that in this paper, we focus on the characterizations of the robust isolatedcalmness of the KKT solution mapping for problem (2) when it admits a unique KKT point. It wouldbe certainly interesting to know to what extent our results can be extended to the case when problem(2) admits non-unique solutions. We shall leave this as a future research topic.
References [1]
J. F. Bonnans and A. Shapiro , Perturbation Analysis of Optimization Problems , Springer,New York, 2000.[2]
E. J. Cand`es and B. Recht , Exact matrix completion via convex optimization , Foundationsof Computational Mathematics, 9 (2008), pp. 717–772.[3]
E. J. Cand`es and T. Tao , The power of convex relaxation: near-optimal matrix completion ,IEEE Transactions on Information Theory, 56 (2009), pp. 2053–2080.[4]
F. H. Clarke , Optimization and Nonsmooth Analysis , Second edition, Classics in AppliedMathematics, 5, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1990.[5]
C. Ding , An Introduction to a Class of Matrix Optimization Problems , Ph.D Thesis, Depart-ment of Mathematics, National University of Singapore, 2012.[6]
C. Ding , Variational analysis of the Ky Fan k -norm , Set-Valued and Variational Analysis,(2017), pp. 1–34, DOI:10.1007/s11228-016-0378-3.[7] C. Ding, D. F. Sun, J. Sun, and K.-C. Toh , Spectral operators of matrices ,arXiv:1401.2269, 2014.[8]
C. Ding, D. F. Sun, and L. W. Zhang , Characterization of the robust isolated calmness fora class of conic programming problems , SIAM Journal on Optimization, 27 (2017), pp. 67–90.[9]
A. L. Dontchev and R. T. Rockafellar , Implicit Functions and Solution Mappings ,Springer, New York, 2009.[10]
D. R. Han, D. F. Sun, and L. W. Zhang , Linear rate convergence of the alternatingdirection method of multipliers for convex composite programming , arXiv:1508.02134, 2015;revised in 2016.[11]
Y. L. Liu and S. H. Pan , Locally upper Lipschitz of the perturbed KKT system of Ky Fan k -norm matrix conic optimization problems , arXiv:1509.00681, 2015.1712] K. L¨owner , ¨Uber monotone matrixfunktionen , Mathematische Zeitschrift, 38 (1934), pp. 177–216.[13] B. Recht, M. Fazel, and P. A. Parrilo , Guaranteed minimum rank solutions to linearmatrix equations via nuclear norm minimization , SIAM Review, 52 (2010), pp. 471–501.[14]
R. T. Rockafellar , Convex Analysis , Princeton University Press, 1970.[15]
R. T. Rockafellar , Augmented Lagrangians and applications of the proximal point algorithmin convex programming , Mathematics of Operations Research, 1 (1976), pp. 97–116.[16]
T. K. Pong, P. Tseng, S. W. Ji, and J. P. Ye , Trace norm regularization: reformulations,algorithms, and multi-task learning , SIAM Journal on Optimization, 20 (2010), pp. 3465–3489.[17]
G. A. Watson , Characterization of the subdifferential of some matrix norms , Linear Algebraand its Applications, 170 (1992), pp. 33–45.[18]
G. A. Watson , On matrix approximation problems with Ky Fan k norms , Numerical Algo-rithms, 5 (1993), pp. 263–272.[19] M. Yuan, A. Ekici, Z. S. Lu, and R. D. C. Monteiro , Dimension reduction and coefficientestimation in multivariate linear regression , Journal of the Royal Statistical Society: Series B69 (2007), pp. 329–346.[20]
X. Y. Zhao, D. F. Sun, and K.-C. Toh , A Newton-CG augmented Lagrangian method forsemidefinite programming , SIAM Journal on Optimization, 20 (2010), pp. 1737–1765.[21]