On the complexity of the projective splitting and Spingarn's methods for the sum of two maximal monotone operators
aa r X i v : . [ m a t h . O C ] M a y On the Complexity of the Projective Splitting and Spingarn’sMethods for the Sum of Two Maximal Monotone Operators
Majela Pent´on Machado ∗ Abstract
In this work, we study the pointwise and ergodic iteration-complexity of a family of projectivesplitting methods proposed by Eckstein and Svaiter, for finding a zero of the sum of two maximalmonotone operators. As a consequence of the complexity analysis of the projective splitting meth-ods, we obtain complexity bounds for the two-operator case of Spingarn’s partial inverse method.We also present inexact variants of two specific instances of this family of algorithms and derivecorresponding convergence rate results.
Keywords. splitting algorithms; maximal monotone operators; complexity; Spingarn method.
AMS Classification:
A wide variety of problems, such as optimization and min-max problems, complementarityproblems and variational inequalities, can be posed as the monotone inclusion problem (MIP)associated to a maximal monotone point-to-set operator. An important tool for the design andanalysis of several implementable methods for solving MIPs is the proximal point algorithm (PPA),proposed by Martinet [1] and generalized by Rockafellar [2]. Even though the PPA has good globaland local convergence properties [2], its major drawback is that it requires the evaluation of the resolvent mappings (or proximal mappings ) associated with the operator. The difficulty lies in thefact that evaluating a resolvent mapping, which is equivalent to solving a proximal subproblem , canbe as complicated as finding a root of the operator.One alternative to surmount this difficulty is to decompose the operator as the sum of twomaximal monotone operators such that their resolvents are considerably easier to evaluate. Then,one can devise methods that use independently these proximal mappings.In this work, we are concerned with MIPs defined by the sum of two maximal monotone op-erators. We are also interested in the case where the problems of finding zeros of these operatorsseparately are easier than solving the MIP for the sum. A typical instance of this situation is the variational inequality problem associated with a maximal monotone operator A and a closed convexsubset C , whose solutions are precisely the zeros of the sum of A and the normal operator associatedwith C , known to be maximal monotone. Splitting methods (or decomposition methods ) for problems of the above-mentioned type attemptto converge to a solution of the MIP by solving, in each iteration, subproblems involving one ofthe operators, but not both.
Peaceman-Rachford and
Douglas-Rachford methods are examples ofthis type of algorithms. These were first introduced in [3] and [4] for the particular case of linearmappings, and then generalized in [5] by Lions and Mercier to address MIPs.
Forward-backward methods [5–7], which generalize standard gradient projection methods for variational inequalitiesand optimization problems, are also examples of splitting algorithms.Recently, a new family of splitting methods for solving MIPs given by the sum of two maximalmonotone operators was introduced in [8] by Eckstein and Svaiter. Through a generalized solutionset in a product space, whose projection onto the first space is indeed the solution set of the problem,the authors constructed a class of decomposition methods with quite solid convergence properties.These algorithms are essentially projection methods, in the sense that in each iteration a hyperplane ∗ IMPA, Estr. Dona Castorina 110, 22460-320 Rio de Janeiro, Brazil, [email protected] . s constructed separating the current iterate from the generalized solution set, and then the nextiterate is taken as a relaxed projection of the current one onto this separating hyperplane. In orderto construct such hyperplanes, two proximal subproblems are solved, each of which involves onlyone of the two maximal monotone operators, which ensures the splitting nature of the methods.In this work we study the iteration-complexity of the family of methods proposed in [8], to bereferred as projective splitting methods (PSM) in the sequel. We start our analysis by introducinga projective algorithm that generalizes the PSM. We then consider a termination criterion forthis general algorithm in terms of the ǫ -enlargements of the operators, which allows us to obtainconvergence rates for the PSM measured by the pointwise and ergodic iteration-complexities.Using the complexity analysis developed for the PSM, we also study the complexity of Spin-garn’s splitting method for solving inclusion problems given by the sum of two maximal monotoneoperators. In [9], Spingarn introduced a splitting method for finding a zero of the sum of m maximalmonotone operators using the concept of partial inverses . For the two-operator case, Eckstein andSvaiter proved in [8] that Spingarn’s method is a special case of a scaled variant of the PSM. Thiswill allow us to establish iteration-complexity results for Spingarn’s method for the case of the sumof two maximal monotone operators.The general projective method that we introduce in this work is also used to construct inexactvariants of two special cases of the family of PSM. For two specific instances of the PSM, weconsider a relative error condition for approximately evaluating the resolvents. The error criterionconsidered in this work is different from the one used in [10], where was generalized the projectivesplitting framework for MIPs given by the sum of m maximal monotone operators. Indeed, we willuse the notion of approximate solutions of a proximal subproblem presented in [11], which yields amore flexible error tolerance criterion and allows evaluation of the ǫ -enlargement. We also deriveconvergence rate results for these two novel algorithms.The remainder of this paper is organized as follows. Section 2 reviews the definitions and somebasic properties of a point-to-set maximal monotone operator and its ǫ -enlargements. Section 3presents a relaxed projection method that extends the framework introduced in [8]. It also provessome properties regarding this general scheme and establishes the stopping criterion that will beconsidered for such method and its instances. Section 4 presents the PSM introduced in [8] andderives global convergence rate results for these methods. Subsection 4.1 specializes these generalcomplexity bounds for the case where global convergence for the family of PSM was obtained in [8].Section 5 studies the iteration complexity of the two-operator case of Spingarn’s method of partialinverses [9]. Finally, sections 6 and 7 propose inexact versions of two special cases of the PSM andestablish iteration-complexity results for them. Throughout this paper, we let R n denote an n -dimensional space with inner product andinduced norm denoted by h· , ·i and k·k , respectively. We also define the spaces R + and E as R + := { x ∈ R : x ≥ } and E := R n × R n × R + .In what follows in this section, we will review some material related to a point-to-set maximalmonotone operator and its ǫ -enlargements that will be needed along this work.A point-to-set operator T : R n ⇒ R n is a relation T ⊆ R n × R n and T ( z ) := { v ∈ R n : ( z, v ) ∈ T } z ∈ R n . Given T : R n ⇒ R n its graph is the setGr ( T ) := { ( z, v ) ∈ R n × R n : v ∈ T ( z ) } . An operator T : R n ⇒ R n is monotone , if (cid:10) z − z ′ , v − v ′ (cid:11) ≥ ∀ ( z, v ) , ( z ′ , v ′ ) ∈ Gr ( T ) , and it is maximal monotone if it is monotone and maximal in the family of monotone operators of R n into R n , with respect to the partial order of inclusion. This is, if S : R n ⇒ R n is a monotoneoperator such that Gr ( T ) ⊆ Gr ( S ), then S = T .The resolvent mapping of a maximal monotone operator T with parameter λ > I + λT ) − ,where I is the identity mapping. It follows directly from the definition that z ′ = ( I + λT ) − ( z ), if nd only if z ′ is the solution of the proximal subproblem ∈ λT ( z ′ ) + ( z ′ − z ) . (1)The ǫ - enlargement of a maximal monotone operator was introduced in [12] by Burachik, Iusemand Svaiter. In [13], Monteiro and Svaiter extended this notion to a generic point-to-set operatoras follows. Given T : R n ⇒ R n and ǫ ∈ R , define the operator ǫ -enlargement of T , T ǫ : R n ⇒ R n ,by T ǫ ( z ) := { v ∈ R n : (cid:10) z ′ − z, v ′ − v (cid:11) ≥ − ǫ, ∀ ( z ′ , v ′ ) ∈ Gr ( T ) } , ∀ z ∈ R n . The following proposition presents some important properties of T ǫ . Its proof can be found in [13]. Proposition 2.1.
Let T : R n ⇒ R n . Then,(a) if ǫ ′ ≤ ǫ , we have T ǫ ′ ( z ) ⊆ T ǫ ( z ) for all z ∈ R n ;(b) T is monotone if and only if T ⊆ T ;(c) T is maximal monotone if and only if T = T . Observe that items (a) and (c) above imply that, if T : R n ⇒ R n is maximal monotone, then T ( z ) ⊆ T ǫ ( z ) for all z ∈ R n and ǫ ≥
0. Hence, T ǫ ( z ) is indeed an enlargement of T ( z ).We now state the weak transportation formula [14] for computing points in the graph of T ǫ . Thisformula will be used in the complexity analysis of some ergodic iterates generated by the algorithmsstudied in this work (see subsection 3.2). Theorem 2.1.
Assume that T : R n ⇒ R n is a maximal monotone operator. Let z i , v i ∈ R n and ǫ i , α i ∈ R + , for i = 1 , . . . , k , be such that v i ∈ T ǫ i ( z i ) , i = 1 , . . . , k, k X i =1 α i = 1 , and define z := k X i =1 α i z i , v := k X i =1 α i v i , ǫ := k X i =1 α i ( ǫ i + h z i − z, v i i ) . Then, ǫ ≥ and v ∈ T ǫ ( z ) . The monotone inclusion problem (MIP) of interest in this work consists of finding z ∈ R n suchthat 0 ∈ A ( z ) + B ( z ) , (2)where A, B : R n ⇒ R n are maximal monotone operators.The framework presented in [8] reformulates problem (2) in terms of a convex feasibility prob-lem, which is defined by a certain closed convex extended solution set . To solve the feasibilityproblem, the authors introduced successive projection algorithms that use, at each iteration, inde-pendent calculations involving each operator. Our goals in this section are to present a scheme thatgeneralizes the methods in [8], and to study its properties. This general framework will allow usto derive convergence rates for the family of PSM and Spingarn’s method. In addition, using thisgeneral method, we construct inexact versions of two specific instances of the PSM and study theircomplexities.Consider S e ( A, B ) ⊂ R n × R n the extended solution set of (2) defined in [8] as S e ( A, B ) := { ( z, w ) ∈ R n × R n : w ∈ B ( z ) , − w ∈ A ( z ) } . The following result establishes two important properties of S e ( A, B ). Its proof can be foundin [8, Lemma 1].
Lemma 3.1. If A, B : R n ⇒ R n are maximal monotone operators, then the following statementshold.(a) A point z ∈ R n is a solution of (2) , if and only if there is w ∈ R n such that ( z, w ) ∈ S e ( A, B ) . b) S e ( A, B ) is a closed and convex subset of R n × R n . According to the above lemma, problem (2) is equivalent to the convex feasibility problem offinding a point in S e ( A, B ). In order to solve this feasibility problem by successive orthogonalprojection methods, we need to construct hyperplanes separating points ( z, w ) / ∈ S e ( A, B ) from S e ( A, B ). For this purpose, in [8] it was used points in the graph of A and B to define affinefunctions, which were called decomposable separators , such that S e ( A, B ) was contained in thenon-positive half-spaces determined by them. Here, we generalize this concept using points in the ǫ -enlargements of A and B . Definition 3.1.
Given two triplets ( x, b, ǫ x ), ( y, a, ǫ y ) ∈ E such that b ∈ B ǫ x ( x ) and a ∈ A ǫ y ( y ),the decomposable separator associated with ( x, b, ǫ x ) and ( y, a, ǫ y ) is the affine function φ : R n × R n → R φ ( z, w ) := h z − x, b − w i + h z − y, a + w i − ǫ x − ǫ y . The non-positive level set of φ is H φ := { ( z, w ) ∈ R n × R n : φ ( z, w ) ≤ } . Lemma 3.2. If φ is the decomposable separator associated with ( x, b, ǫ x ) and ( y, a, ǫ y ) ∈ E , where b ∈ B ǫ x ( x ) and a ∈ A ǫ y ( y ) , and H φ is its non-positive level set, then(a) S e ( A, B ) ⊆ H φ ;(b) either ∇ φ = 0 or φ ≤ in R n × R n ;(c) either H φ is a closed half-space or H φ = R n × R n .Proof. Item (a) is a direct consequence of the definitions of the ǫ -enlargement of a point-to-setoperator and the set S e ( A, B ). Rewriting φ ( z, w ) as φ ( z, w ) = h z − y, a + b i + h w − b, x − y i − ǫ x − ǫ y ∀ ( z, w ) ∈ R n × R n , (3)and noting that ∇ φ = ( a + b, x − y ) and ǫ x , ǫ y ≥
0, then (b) and (c) follow immediately.We now present the general projection scheme for finding a point in S e ( A, B ) that will be studiedin this work. Algorithm 1 below generalizes the framework introduced in [8], since we use the notionof decomposable separator introduced in Definition 3.1.
Algorithm 1.
Choose ( z , w ) ∈ R n × R n . For k = 1 , , . . .
1. Choose ( x k , b k , ǫ xk ) and ( y k , a k , ǫ yk ) ∈ E such that b k ∈ B ǫ xk ( x k ) and a k ∈ A ǫ yk ( y k ) .
2. Define φ k : R n × R n → R as the decomposable separator associated with ( x k , b k , ǫ xk ) and ( y k , a k , ǫ yk ) , and compute P H φk ( z k − , w k − ) , the orthogonal projection of ( z k − , w k − ) ontothe set H φ k , i.e. define γ k := , if φ k ( z k − , w k − ) ≤ ,φ k ( z k − , w k − ) k∇ φ k k , otherwise ; and set P H φ ( z k − , w k − ) = ( z k − , w k − ) − γ k ∇ φ k .
3. Choose ρ k ∈ ]0 , and set ( z k , w k ) = ( z k − , w k − ) + ρ k h P H φk ( z k − , w k − ) − ( z k − , w k − ) i = ( z k − , w k − ) − ρ k γ k ∇ φ k . Note that the general form of Algorithm 1 is not sufficient to guarantee convergence of thesequence { ( z k , w k ) } to a point in S e ( A, B ). For example, if the separation between the point( z k − , w k − ) / ∈ S e ( A, B ) and S e ( A, B ) by φ k is not strict, then the next iterate is in fact ( z k − , w k − )itself, which might lead to a constant sequence. Hence, to ensure convergence it is necessary toimpose additional conditions on the decomposable separators, see [8] and sections 4, 6 and 7 below.However, since Algorithm 1 is a relaxed projection type method, it is possible to establish Fej´ermonotone convergence to S e ( A, B ) and boundedness of its generated sequence, as well as otherclassical properties of this kind of algorithms (see for example [8], [15]). .1 The Generated SequencesWe will now analyze some properties of the sequences { ( z k , w k ) } , { φ k } , { γ k } and { ρ k } generatedby Algorithm 1, which will be needed in our complexity study. To this end, let us first prove thefollowing technical result. Lemma 3.3.
For any ( z, w ) ∈ R n × R n and k ≥ we have k ( z, w ) − ( z k , w k ) k + 12 k X j =1 ρ j (2 − ρ j ) γ j k∇ φ j k = 12 k ( z, w ) − ( z , w ) k + k X j =1 ρ j γ j φ j ( z, w ) . (4) Proof.
First we observe that for j = 1 , , . . . , and any ( z, w ) ∈ R n × R n it holds that12 k ( z, w ) − ( z j , w j ) k = 12 k ( z, w ) − ( z j − , w j − ) + ρ j γ j ∇ φ j k = 12 k ( z, w ) − ( z j − , w j − ) k + h ( z, w ) − ( z j − , w j − ) , ρ j γ j ∇ φ j i + 12 ρ j γ j k∇ φ j k = 12 k ( z, w ) − ( z j − , w j − ) k + ρ j γ j h ( z, w ) − ( y j , b j ) , ∇ φ j i + ρ j γ j h ( y j , b j ) − ( z j − , w j − ) , ∇ φ j i + 12 ρ j γ j k∇ φ j k , (5)where the first equality above follows from the update rule in step 3 of Algorithm 1.Equation (3) with φ = φ j implies that φ j ( z, w ) = h ( z, w ) − ( y j , b j ) , ∇ φ j i − ǫ xj − ǫ yj ∀ ( z, w ) ∈ R n × R n . Therefore, adding and subtracting ρ j γ j ( ǫ xj + ǫ yj ) on the right-hand side of the last equality in (5)and combining with the identity above, we obtain12 k ( z, w ) − ( z j , w j ) k = 12 k ( z, w ) − ( z j − , w j − ) k + ρ j γ j φ j ( z, w ) − ρ j γ j φ j ( z j − , w j − ) + 12 ρ j γ j k∇ φ j k . If we assume that γ j >
0, then the definition of γ j in step 2 of Algorithm 1 yields that φ j ( z j − , w j − ) = γ j k∇ φ j k . Hence, substituting this expression into the equality above and rear-ranging, we have12 k ( z, w ) − ( z j , w j ) k + 12 ρ j (2 − ρ j ) γ j k∇ φ j k = 12 k ( z, w ) − ( z j − , w j − ) k + ρ j γ j φ j ( z, w ) . It is clear that this latter equality also holds if γ j = 0. Thus, adding equation above from j = 1 to k we obtain (4).In what follows we assume that problem (2) has at least one solution, which implies that S e ( A, B )is a non-empty set in view of Lemma 3.1.Next theorem, which follows directly from Lemma 3.3, establishes boundedness of the sequence { ( z k , w k ) } calculated by Algorithm 1, and it also shows that the sum appearing on the left-handside of (4) is bounded by the distance of the initial point to the set S e ( A, B ). Theorem 3.1.
Take ( z , w ) ∈ R n × R n and let { ( z k , w k ) } , { φ k } , { γ k } and { ρ k } be the sequencesgenerated by Algorithm . Then, for every integer k ≥ , we have k X j =1 ρ j (2 − ρ j ) γ j k∇ φ j k ≤ d and k ( z k , w k ) − ( z , w ) k ≤ d , (6) where d is the distance of ( z , w ) to S e ( A, B ) .Proof. Take ( z ∗ , w ∗ ) the orthogonal projection of ( z , w ) onto S e ( A, B ). From Lemma 3.2(a) itfollows that φ j ( z ∗ , w ∗ ) ≤ j ≥
1. Hence, specializing equality (4) with ( z ∗ , w ∗ ) weobtain the first bound in (6) and the following inequality k ( z k , w k ) − ( z ∗ , w ∗ ) k ≤ d . Since k ( z , w ) − ( z ∗ , w ∗ ) k = d , the second estimate in (6) follows from the latter two relationsand the triangle inequality for norms. t is important to say that Theorem 3.1 can be proven using standard arguments of relaxedprojection algorithms, see for instance [15]. We have chosen the above approach since it will bemore convenient for our subsequent analysis.3.2 The Ergodic SequencesBesides the pointwise complexity of specific instances of Algorithm 1, we are also interestedin deriving their convergence rates measured by the iteration complexity in an ergodic sense. Todo this, we consider sequences obtained by weighted averages of the sequences { x k } and { y k } ,generated by Algorithm 1, and study their properties.Let { x k } , { y k } , { γ k } and { ρ k } be the sequences computed with Algorithm 1, for every integer k ≥ γ k > x k and y k as x k := 1Γ k k X j =1 ρ j γ j x j , y k := 1Γ k k X j =1 ρ j γ j y j , where Γ k := k X j =1 ρ j γ j . (7)The following lemma is a direct consequence of the weak transportation formula, Theorem 2.1. Lemma 3.4.
Let { ( x k , b k , ǫ xk ) } , { ( y k , a k , ǫ yk ) } , { γ k } and { ρ k } be the sequences generated by Algo-rithm . For every integer k ≥ , suppose that γ k > and consider x k , y k and Γ k given as in (7) .Define also b k := 1Γ k k X j =1 ρ j γ j b j , ǫ xk := 1Γ k k X j =1 ρ j γ j ( ǫ xj + h x j − x k , b j i ) , (8) a k := 1Γ k k X j =1 ρ j γ j a j , ǫ yk := 1Γ k k X j =1 ρ j γ j ( ǫ yj + h y j − y k , a j i ) . (9) Then, we have ǫ xk ≥ , b k ∈ B ǫ xk ( x k ) ,ǫ yk ≥ , a k ∈ A ǫ yk ( y k ) . Proof.
The lemma follows from Theorem 2.1 and the inclusions b j ∈ B ǫ xj ( x j ) and a j ∈ A ǫ yj ( y j ).We will refer to the sequences { ( x k , b k , ǫ xk ) } and { ( y k , a k , ǫ yk ) } , defined in (7)-(9), as the ergodicsequences associated with Algorithm 1.Next lemma presents alternative expressions for a k + b k , x k − y k and ǫ xk + ǫ yk , which will be usedfor obtaining bounds on their sizes. Lemma 3.5.
Let { ( x k , b k , ǫ xk ) } , { ( y k , a k , ǫ yk ) } , { γ k } and { ρ k } be the sequences generated by Algo-rithm . Assume that γ k > for all k ≥ , and define the sequences { x k } , { y k } , { Γ k } , { b k } , { a k } , { ǫ xk } and { ǫ yk } as in (7) , (8) and (9) . Then, for every integer k ≥ , we have a k + b k = 1Γ k ( z − z k ) , x k − y k = 1Γ k ( w − w k ) , ǫ xk + ǫ yk = − k k X j =1 ρ j γ j φ j ( y k , b k ) . (10) Proof.
Direct use of the definitions of x k , y k , b k and a k yields( a k + b k , x k − y k ) = 1Γ k k X j =1 ρ j γ j ( a j + b j , x j − y j ) . Since ∇ φ j = ( a j + b j , x j − y j ) for all integer j ≥
1, in view of the update rule in step 3 of Algorithm1, the definition of Γ k and the equation above, we have( z k , w k ) = ( z , w ) − k X j =1 ρ j γ j ( a j + b j , x j − y j )= ( z , w ) − Γ k ( a k + b k , x k − y k ) . The relation above clearly implies the first two identities in (10). o prove the last equality in (10) we first note that k X j =1 ρ j γ j φ j ( y k , b k ) = k X j =1 ρ j γ j (cid:0)(cid:10) y k − x j , b j − b k (cid:11) + (cid:10) y k − y j , a j + b k (cid:11) − ǫ xj − ǫ yj (cid:1) = k X j =1 ρ j γ j (cid:0) h y k , b j i + (cid:10) x j , b k − b j (cid:11) + h y k − y j , a j i − (cid:10) y j , b k (cid:11) − ǫ xj − ǫ yj (cid:1) . Next, we multiply the equation above by 1 / Γ k and use the definitions of y k and b k to obtain1Γ k k X j =1 ρ j γ j φ j ( y k , b k ) = 1Γ k k X j =1 ρ j γ j (cid:0)(cid:10) x j , b k − b j (cid:11) + h y k − y j , a j i − ǫ xj − ǫ yj (cid:1) . (11)Now, we observe that ǫ xk can be rewritten as ǫ xk = 1Γ k k X j =1 ρ j γ j (cid:0) ǫ xj + (cid:10) x j , b j − b k (cid:11)(cid:1) . Thus, adding ǫ xk and ǫ yk and combining with (11) and the equation above, we deduce the thirdequality in (10).The following result establishes bounds for the quantities a k + b k , x k − y k and ǫ xk + ǫ yk . Theorem 3.2.
Assume the hypotheses of Lemma . and let d be the distance of ( z , w ) to S e ( A, B ) . Then, for all integer k ≥ , we have (cid:13)(cid:13) a k + b k (cid:13)(cid:13) ≤ d Γ k , k x k − y k k ≤ d Γ k , (12) ǫ xk + ǫ yk ≤ k " k k X j =1 ρ j γ j k ( y j , b j ) − ( z j − , w j − ) k + 4 d . (13) Proof.
We combine the first two identities in (10) with the second inequality in (6) to obtain (cid:13)(cid:13) ( a k + b k , x k − y k ) (cid:13)(cid:13) = 1Γ k k ( z , w ) − ( z k , w k ) k ≤ d Γ k . Thus, the bounds in (12) follow immediately from the equation above.Now, taking ( z, w ) = ( y k , b k ) in equation (4) and rearranging the terms we have − k X j =1 ρ j γ j φ j ( y k , b k ) = 12 (cid:13)(cid:13) ( y k , b k ) − ( z , w ) (cid:13)(cid:13) − (cid:13)(cid:13) ( y k , b k ) − ( z k , w k ) (cid:13)(cid:13) − k X j =1 ρ j (2 − ρ j ) γ j k∇ φ j k . Since ρ j ∈ ]0 ,
2[ for all integer j ≥
1, the equation above implies − k X j =1 ρ j γ j φ j ( y k , b k ) ≤ (cid:13)(cid:13) ( y k , b k ) − ( z , w ) (cid:13)(cid:13) . (14)Next, we define ( z k , w k ) := 1Γ k k P j =1 ρ j γ j ( z j − , w j − ) and use the triangle inequality for norms toobtain 12 (cid:13)(cid:13) ( y k , b k ) − ( z , w ) (cid:13)(cid:13) ≤ (cid:13)(cid:13) ( y k , b k ) − ( z k , w k ) (cid:13)(cid:13) + k ( z k , w k ) − ( z , w ) k ≤ k k X j =1 ρ j γ j k ( y j , b j ) − ( z j − , w j − ) k + 1Γ k k X j =1 ρ j γ j k ( z j − , w j − ) − ( z , w ) k ≤ k k X j =1 ρ j γ j k ( y j , b j ) − ( z j − , w j − ) k + 4 d , (15) here the second and the third inequalities above are due to the convexity of k·k and the secondbound in (6), respectively.Combining (14) with (15) we deduce that − k X j =1 ρ j γ j φ j ( y k , b k ) ≤ k k X j =1 ρ j γ j k ( y j , b j ) − ( z j − , w j − ) k + 4 d . Relation above, together with the last equality in (10), now yields (13).3.3 Stopping CriterionIn order to analyze the complexity properties of instances of Algorithm 1, we define a terminationcondition for this general method in terms of the ǫ -enlargements of operators A and B . This criterionwill enable the obtention of complexity bounds, proportional to the distance of the initial iterateto the extended solution set S e ( A, B ), for all the schemes presented in this work.We consider the following stopping criterion for Algorithm 1. Given an arbitrary pair of scalars δ , ǫ >
0, Algorithm 1 will stop when it finds a pair of points ( x, b, ǫ x ), ( y, a, ǫ y ) ∈ E such that b ∈ B ǫ x ( x ) , a ∈ A ǫ y ( y ) , max {k a + b k , k x − y k} ≤ δ, max { ǫ x , ǫ y } ≤ ǫ. (16)We observe that if δ = ǫ = 0, in view of Proposition 2.1, the above termination criterion is reducedto b ∈ B ( x ), a ∈ A ( y ), x = y and b = − a , in which case ( x, b ) ∈ S e ( A, B ).Based on the termination condition (16) we can define the following notion of approximatesolutions of problem (2).
Definition 3.2.
For a given pair of positive scalars ( δ, ǫ ), a pair ( x, y ) ∈ R n × R n is called a ( δ, ǫ ) -approximate solution (or ( δ, ǫ ) -solution ) of problem (2), if there exist b, a ∈ R n and ǫ x , ǫ y ∈ R + suchthat the relations in (16) hold. Our goal in this section is to establish the complexity analysis of the family of PSM developedin [8] for solving (2). First, we will observe that the PSM is an instance of the general Algorithm 1with the feature of solving two proximal subproblems exactly, one involving only A and the otherone only B , for obtaining the decomposable separator in step 2 of Algorithm 1. This will allow us touse the results of section 3 to derive general iteration-complexity bounds for the PSM. Such boundswill be expressed in terms of the parameter sequences { λ k } , { µ k } , { ρ k } and { α k } , calculated ateach iteration of the method (see the PSM below). In subsection 4.1, we will specialize these resultsfor the case where global convergence was obtained in [8].We start by stating the family of projective splitting methods (PSM). Algorithm ( PSM ) . Choose ( z , w ) ∈ R n × R n . For k = 1 , , . . .
1. Choose λ k , µ k > and α k ∈ R such that µ k λ k − (cid:16) α k (cid:17) > , (17) and find ( x k , b k ) ∈ Gr ( B ) and ( y k , a k ) ∈ Gr ( A ) such that λ k b k + x k = z k − + λ k w k − , (18) µ k a k + y k = (1 − α k ) z k − + α k x k − µ k w k − . (19)
2. If k a k + b k k + k x k − y k k = 0 stop. Otherwise, set γ k = h z k − − x k , b k − w k − i + h z k − − y k , a k + w k − ik a k + b k k + k x k − y k k . (20)
3. Choose a parameter ρ k ∈ ]0 , and set z k = z k − − ρ k γ k ( a k + b k ) ,w k = w k − − ρ k γ k ( x k − y k ) . (21) everal remarks are in order. The PSM is the same as [8, Algorithm 2], except for the stoppingcriterion in step 2 above and boundedness conditions imposed on the parameters ρ k , λ k and µ k in [8].Note that if k a k + b k k + k x k − y k k = 0 for some k , then x k = y k , b k = − a k and, since the points( x k , b k ) and ( y k , a k ) are chosen in the graph of B and A , respectively, we have ( x k , b k ) ∈ S e ( A, B ).Therefore, when the PSM stops in step 2, it has found a point in the extended solution set.Observe also that, since B is maximal monotone, Minty’s theorem [16] implies that the resolventmappings ( I + λ k B ) − are everywhere defined and single valued for all integer k ≥
1. Hence, by(18) we have that the points x k = ( I + λ k B ) − ( z k − + λ k w k − ) and b k = (1 /λ k )( z k − − x k ) + w k − exist and are unique. Similarly, the maximal monotonicity of A , together with (19), guaranteesthe existence and uniqueness of y k = ( I + µ k A ) − ((1 − α k ) z k − + α k x k − µ k w k − ) and a k = (1 /µ k )((1 − α k ) z k − + α k x k ) − w k − .Moreover, if for k = 1 , , . . . , we denote by φ k the decomposable separator (see Definition 3.1)associated with the triplets ( x k , b k ,
0) and ( y k , a k , z k , w k ) = ( z k − , w k − ) − ρ k γ k ∇ φ k . Consequently, the family of PSM falls within the general framework of Algorithm 1, and the resultsof section 3 apply.Finally, note that the PSM generates, on each iteration, a pair ( x k , y k ) and vectors b k , a k ∈ R n such that the inclusions in (16) hold with ( x, b, ǫ x ) = ( x k , b k ,
0) and ( y, a, ǫ y ) = ( y k , a k , k a k + b k k and k x k − y k k to estimate when an iterate( x k , y k ) is bound to satisfy the termination criterion (16).Before establishing the iteration-complexity results for the PSM, we need the following technicalresult. It presents two lower bounds for φ k ( z k − , w k − ). Lemma 4.1.
Let { ( x k , b k ) } , { ( y k , a k ) } , { ( z k , w k ) } , { λ k } , { µ k } , { α k } and { ρ k } be the sequencesgenerated by the PSM, and { φ k } be the sequence of decomposable separators associated with thePSM. Then, for all integer k ≥ , the following inequalities hold φ k ( z k − , w k − ) ≥ θ k δ k (cid:0) k a k + b k k + k x k − y k k (cid:1) , (22) φ k ( z k − , w k − ) ≥ θ k µ k (cid:0) k z k − − y k k + k w k − − b k k (cid:1) , (23) where δ k := µ k + (1 − α k ) λ k > and θ k > is the smallest eigenvalue of the matrix − λ k | α k | − λ k | α k | λ k µ k ! . Proof.
Inequality (22) was obtained in [8, Proposition 3] as part of the convergence proof of Algo-rithm 2 in [8], as were the assertions that θ k , δ k > y k from both sides of (18) and rearrange the terms we obtain x k − y k = z k − − y k + λ k ( w k − − b k ) . (24)Now, adding µ k b k to both sides of (19) and rearranging we have µ k ( a k + b k ) = (1 − α k ) z k − + α k x k − y k − µ k ( w k − − b k )= α k ( x k − y k ) + (1 − α k )( z k − − y k ) − µ k ( w k − − b k ) . (25)Next, we substitute (24) into (25) and divide by µ k to obtain a k + b k = α k µ k ( z k − − y k + λ k ( w k − − b k )) + (1 − α k ) µ k ( z k − − y k ) − ( w k − − b k )= 1 µ k ( z k − − y k ) + (cid:18) α k λ k µ k − (cid:19) ( w k − − b k ) . (26)Since φ k ( z k − , w k − ) = h a k + b k , z k − − y k i + h x k − y k , w k − − b k i , quation above, together with (24) and (26), yields φ k ( z k − , w k − ) = 1 µ k k z k − − y k k + α k λ k µ k h z k − − y k , w k − − b k i + λ k k w k − − b k k ≥ µ k k z k − − y k k − | α k | λ k µ k k z k − − y k k k w k − − b k k + λ k k w k − − b k k = 1 µ k (cid:18) k z k − − y k kk w k − − b k k (cid:19) T − λ k | α k | − λ k | α k | λ k µ k ! (cid:18) k z k − − y k kk w k − − b k k (cid:19) , where the inequality in the above relation follows from the Cauchy-Schwartz inequality. Finally,(23) follows from the expression above and the definition of θ k .For simplicity, the convergence rate results presented below suppose that the PSM never stopsin step 2, i.e. they are assuming that k∇ φ k k > k ≥
1. However, they can easily berestated without assuming such condition by saying that either the conclusion stated below holdsor ( x k , b k ) is a point in S e ( A, B ).Next result estimates the quality of the best iterate among ( x , y ) , . . . , ( x k , y k ) in terms of thestopping criterion (16). We refer to these estimates as pointwise complexity bounds for the PSM. Theorem 4.1.
Let { ( x k , b k ) } , { ( y k , a k ) } , { λ k } , { µ k } , { α k } , { γ k } and { ρ k } be the sequences gen-erated by the PSM. Then, for every integer k ≥ , we have b k ∈ B ( x k ) , a k ∈ A ( y k ) , (27) and there exists an index ≤ i ≤ k such that k a i + b i k + k x i − y i k ≤ d k P j =1 ρ j (2 − ρ j ) (cid:16) θ j δ j (cid:17) , (28) where d is the distance of the first iterate ( z , w ) to S e ( A, B ) , and θ k and δ k are defined in Lemma . .Proof. The assertions that b k ∈ B ( x k ) and a k ∈ A ( y k ) are direct consequences of step 1 in thePSM. The definition of γ j in step 2 of the method, together with the definition of φ j and inequality(22), yields γ j = φ j ( z j − , w j − ) k∇ φ j k ≥ θ j δ j for j = 1 , , . . . . (29)Therefore, γ j ≥ (cid:18) θ j δ j (cid:19) for j = 1 , , . . . . Multiplying both sides of the inequality above by ρ j (2 − ρ j ) k∇ φ j k , adding from j = 1 to k andusing the first bound in (6), we have d ≥ k X j =1 k∇ φ j k ρ j (2 − ρ j ) (cid:18) θ j δ j (cid:19) . Taking i such that i ∈ arg min j =1 ,...,k (cid:0) k∇ φ j k (cid:1) , and using the previous inequality we obtain d ≥ k X j =1 ρ j (2 − ρ j ) (cid:18) θ j δ j (cid:19) k∇ φ i k . Bound (28) now follows from the above relation and noting that ∇ φ i = ( a i + b i , x i − y i ). e will now derive alternative complexity bounds for the PSM. Using the sequences of ergodiciterates associated with the PSM, defined as in subsection 3.2, we will obtain convergence rates forthe methods in the ergodic sense. We refer to these kind of estimates as ergodic complexity bounds.Define the sequences of ergodic means { ( x k , b k , ǫ xk ) } and { ( y k , a k , ǫ yk ) } , associated with the se-quences { ( x k , b k , } , { ( y k , a k , } , { γ k } and { ρ k } generated by the PSM, as in (7), (8) and (9).According to Lemma 3.4, we can attempt to bound the size of (cid:13)(cid:13) a k + b k (cid:13)(cid:13) , k x k − y k k , ǫ xk and ǫ yk , inorder to know when the ergodic iterates { x k } and { y k } will meet the stopping condition (16). Theorem 4.2.
Assume the hypotheses of Theorem . . In addition, consider the sequences ofergodic iterates { ( x k , b k , ǫ xk ) } and { ( y k , a k , ǫ yk ) } associated with the sequences generated by the PSM,defined as in (7) , (8) and (9) . Then, for every integer k ≥ , we have b k ∈ B ǫ xk ( x k ) , a k ∈ A ǫ yk ( y k ) , (30) and (cid:13)(cid:13) a k + b k (cid:13)(cid:13) ≤ d Γ k , k x k − y k k ≤ d Γ k , ǫ xk + ǫ yk ≤ d ( ς k + 4)Γ k , (31) where ς k := max j =1 ,...,k (cid:26) µ j θ j (2 − ρ j )Γ k (cid:27) . Proof.
Inclusions in (30) are a consequence of Lemma 3.4. The first two inequalities in (31) areobtained by applying Theorem 3.2.Now, we observe that relation (23), together with the equality in (29), implies µ j θ j k∇ φ j k γ j ≥ k ( y j , b j ) − ( z j − , w j − ) k for j = 1 , , . . . . Multiplying the inequality above by 1Γ k ρ j γ j and adding from j = 1 to k , we obtain1Γ k k X j =1 ρ j γ j k ( y j , b j ) − ( z j − , w j − ) k ≤ k k X j =1 µ j θ j ρ j γ j k∇ φ j k = 1Γ k k X j =1 µ j θ j (2 − ρ j ) ρ j (2 − ρ j ) γ j k∇ φ j k ≤ (cid:18) max j =1 ,...,k (cid:26) µ j θ j (2 − ρ j )Γ k (cid:27)(cid:19) k X j =1 ρ j (2 − ρ j ) γ j k∇ φ j k ≤ (cid:18) max j =1 ,...,k (cid:26) µ j θ j (2 − ρ j )Γ k (cid:27)(cid:19) d , where the last inequality above follows from the first estimate in (6). We combine the relationabove with (13) and the definition of ς k to deduce the last bound in (31).4.1 Specialized Complexity ResultsIn this subsection, we will specialize the general pointwise and ergodic complexity bounds derivedfor the PSM in Theorems 4.1 and 4.2, respectively, for the case where global convergence wasobtained in [8].In [8], it was proven convergence of the sequences { ( x k , b k ) } , { ( y k , − a k ) } and { ( z k , w k ) } , calcu-lated by the PSM, to a point of S e ( A, B ) under the following assumptions:(A.1) there exist λ and λ such that, λ ≥ λ > λ k , µ k ∈ [ λ, λ ] for all integer k ≥ ρ ∈ [0 ,
1[ such that ρ k ∈ [1 − ρ, ρ ] for all integer k ≥ ν := inf k (cid:26) µ k λ k − (cid:16) α k (cid:17) (cid:27) > O (1 / √ k ) pointwise convergencerate, while the rate in the ergodic sense is O (1 /k ). heorem 4.3. Let { ( x k , b k ) } , { ( y k , a k ) } , { λ k } , { µ k } , { α k } , { γ k } and { ρ k } be the sequences gen-erated by the PSM under assumptions (A.1)-(A.3) . If d denote the distance of ( z , w ) to theextended solution set S e ( A, B ) . Then, for all integer k ≥ , we have b k ∈ B ( x k ) , a k ∈ A ( y k ) , and there exists an index ≤ i ≤ k such that k a i + b i k ≤ d υ √ k (1 − ρ ) and k x i − y i k ≤ d υ √ k (1 − ρ ) , where υ := 2 λ (cid:16) λ (cid:17) (cid:18) q λ/λ (cid:19) λ ν . Proof.
The inclusions in the statement of the theorem follow from (27). Now, we note that condition(A.2) implies ρ j (2 − ρ j ) ≥ (1 − ρ ) for j = 1 , , . . . . (32)Next, we observe that relation (17) in step 1 of the PSM yields | α j | ≤ p µ j /λ j . Hence, assumption(A.1) implies | α j | ≤ q λ/λ for j = 1 , , . . . . The inequality above, together with the definition of δ j in Lemma 4.1 and assumption (A.1), yields δ j ≤ λ (cid:18) q λ/λ (cid:19) . (33)Moreover, in [8, Proposition 3] it was proven that θ j := 12 (cid:16) λ j µ j − p (1 + λ j µ j ) − λ j µ j − ( λ j α j / ) (cid:17) ≥ λ j ( µ j /λ j − ( α j / )1 + λ j µ j . Thus, under hypotheses (A.1)-(A.3) we have θ j ≥ λ ν λ , (34)and combining (33) with (34) we obtain θ j δ j ≥ λ ν (cid:16) λ (cid:17) λ (cid:18) q λ/λ (cid:19) = 1 υ for j = 1 , , . . . . (35)Now, from inequalities (35) and (32) we deduce that ρ j (2 − ρ j ) (cid:18) θ j δ j (cid:19) ≥ (1 − ρ ) υ for j = 1 , . . . , k . Hence, adding equation above from j = 1 to k we have k X j =1 ρ j (2 − ρ j ) (cid:18) θ j δ j (cid:19) ≥ k (1 − ρ ) υ . The theorem follows combining the above expression with inequality (28).
Theorem 4.4.
Assume the hypotheses of Theorem . . Consider the sequences of ergodic iterates { ( x k , b k , ǫ xk ) } and { ( y k , a k , ǫ yk ) } associated with the sequences generated by the PSM, defined as in (7) , (8) and (9) . Then, for every integer k ≥ , we have b k ∈ B ǫ xk ( x k ) , a k ∈ A ǫ yk ( y k ) , (36) nd (cid:13)(cid:13) a k + b k (cid:13)(cid:13) ≤ d υk (1 − ρ ) , k x k − y k k ≤ d υk (1 − ρ ) , ǫ xk + ǫ yk ≤ d υ ( υ ′ k + 4) k (1 − ρ ) , (37) where υ ′ k := λ (cid:16) λ (cid:17) υλ ν (1 − ρ ) k . Proof.
The inclusions in (36) follow from Lemma 3.4. The definition of Γ k , together with assumption(A.2) and equation (29), yields Γ k ≥ (1 − ρ ) k X j =1 θ j δ j . Therefore, by (35) and the inequality above we haveΓ k ≥ (1 − ρ ) kυ . (38)The first two bounds in (37) now follow from (38) and the first two inequalities in (31).To conclude the proof we observe that the definition of ς k , hypothesis (A.1), (34) and (38) imply ς k ≤ λ (1 + λ ) υλ ν (1 − ρ ) k . Thus, combining the above relation with the last inequality in (31), the definition of υ ′ k and (38),we obtain the last bound in (37).We emphasize here that the derived bounds obtained in Theorem 4.4 are asymptotically betterthan the ones obtained in Theorem 4.3. Indeed, the bounds for x k , y k , a k and b k are O (1 / √ k ),whereas for x k , y k , b k , a k , ǫ xk and ǫ yk the bounds are O (1 /k ). However, the iterates calculated by thePSM are points in the graph of A and B , while the ergodic iterates are in some outer approximationof the operators, namely they are points in an ǫ -enlargement of A and B .The following result, which is an immediate consequence of Theorems 4.3 and 4.4, presentscomplexity bounds for the PSM to obtain ( δ, ǫ )-approximate solutions of problem (2). Corollary 4.1.
Assume the hypotheses of Theorem . . Then, the following statements hold.(a) For every δ > there exists an index i = O (cid:18) d υ δ (cid:19) such that the iterate ( x i , y i ) is a ( δ, -solution of problem (2) .(b) For every δ , ǫ > there exists an index k = O (cid:18) max (cid:26) d υδ , d υǫ (cid:27)(cid:19) such that, for any k ≥ k , the ergodic iterate ( x k , y k ) is a ( δ, ǫ ) -solution of problem (2) . In this section, we study the iteration-complexity of the two-operator case of Spingarn’s splittingalgorithm. In [9], Spingarn describes a partial inverse method for solving MIPs given by the sumof m maximal monotone operators. Spingarn’s method computes, at each iteration, independentproximal subproblems on each of the m operators involved in the problem and then finds the nextiterate by essentially averaging the results. This algorithm is actually a special case of the Douglas-Rachford splitting method [17], and it is also a particular instance of the general projective splittingmethods for sums of m maximal monotone operators, which were introduced in [10].Eckstein and Svaiter proved in [8] that the m = 2 case of the Spingarn splitting method is aspecial case of a scaled variant of the PSM. Interpreting Spingarn’s algorithm as an instance of the SM allows us to use the analysis developed in the previous section for obtaining its complexitybounds.For this purpose, let us begin with a brief discussion of the reformulation of problem (2) studiedin [8], obtained via including a scale factor. If η > η gives the problem 0 ∈ ηA ( z ) + ηB ( z ) . This simple reformulation leaves the solution set unchanged, but it transforms the set S e ( A, B ).Indeed, the extended solution set associated with operators ηA and ηB has the form S e ( ηA, ηB ) = { ( z, ηw ) : ( z, w ) ∈ S e ( A, B ) } . If we apply the PSM to ηA and ηB , and consider ηa k , ηb k and ηw k , respectively, in place of a k , b k and w k , after some algebraic manipulations we obtain a scheme identical to the PSM, exceptthat (18)-(21) are modified to λ k ηb k + x k = z k − + λ k ηw k − , b k ∈ B ( x k ) , (39) µ k ηa k + y k = (1 − α k ) z k − + α k x k − µ k ηw k − , a k ∈ A ( y k ) , (40) γ k = h z k − − x k , b k − w k − i + h z k − − y k , a k + w k − i η k a k + b k k + η k x k − y k k , (41) z k = z k − − ρ k γ k η ( a k + b k ) , (42) w k = w k − − ρ k γ k η ( x k − y k ) . (43)The general pointwise and ergodic complexity bounds for the method above are obtained as a directconsequence of Theorems 4.1 and 4.2, replacing a i , b i , a k , b k , ǫ xk and ǫ yk by ηa i , ηb i , ηa k , ηb k , ηǫ xk and ηǫ yk , respectively.If η >
0, in our notation, the Spingarn splitting method is reduced to the following set ofrecursions: ηb k + x k = z k − + ηw k − , b k ∈ B ( x k ) , (44) ηa k + y k = z k − − ηw k − , a k ∈ A ( y k ) , (45) z k = (1 − ρ k ) z k − + ρ k x k + y k ) , (46) w k = (1 − ρ k ) w k − + ρ k b k − a k ) . (47)Note that if we take λ k = µ k = 1 and α k = 0 in (39)-(43) for all integer k ≥
1, then (39)-(40) and(44)-(45) are identical. Moreover, the remaining calculations, (41), (42) and (43), can be rewritteninto the form (46)-(47), as it is established in the next result.
Theorem 5.1.
Assume that the following condition is satisfied: (B) λ k = µ k = 1 and α k = 0 in (39) - (43) for every integer k ≥ .Then, the recursions (39) - (43) and (44) - (47) are identical. Hence, Spingarn’s method is a specialcase of the PSM.Proof. This result was proven in [8, Subsection 4.2].The following theorem derives the global convergence rate of Spingarn’s splitting method interms of the termination criterion (16).
Theorem 5.2.
Let η > and let { ( x k , b k ) } , { ( y k , a k ) } and { ρ k } be the sequences generated bySpingarn’s splitting method (44) - (47) . For every k ≥ , define P k := k X j =1 ρ j , (48) and x k := 1 P k k X j =1 ρ j x j , b k := 1 P k k X j =1 ρ j b j , ǫ xk := 1 P k k X j =1 ρ j h x j − x k , b j i , (49) y k := 1 P k k X j =1 ρ j y j , a k := 1 P k k X j =1 ρ j a j , ǫ yk := 1 P k k X j =1 ρ j h y j − y k , a j i . (50) ssume hypothesis (A.2) and set d := dist (( z , ηw ) , S e ( ηA, ηB )) . Then, the following statementshold.(a) For every integer k ≥ we have b k ∈ B ( x k ) , a k ∈ A ( y k ) , and there exists an index ≤ i ≤ k such that k a i + b i k ≤ d η √ k (1 − ρ ) , k x i − y i k ≤ d √ k (1 − ρ ) . (b) For every integer k ≥ we have b k ∈ B ǫ xk ( x k ) , a k ∈ A ǫ yk ( y k ) , and (cid:13)(cid:13) a k + b k (cid:13)(cid:13) ≤ d ηk (1 − ρ ) , k x k − y k k ≤ d k (1 − ρ ) ,ǫ xk + ǫ yk ≤ d ηk (1 − ρ ) (cid:18) − ρ ) + 4 (cid:19) . Proof. (a) By the definitions of δ k and θ k in the statement of Lemma 4.1 and hypothesis (B), we have that δ k = 2 and θ k = 1 for every integer k ≥ α k = 0 and λ k = µ k = 1.(b) The first assertions in (b) follow from the definitions of P k , x k , b k , ǫ xk , y k , a k , ǫ yk in (48), (49)and (50), the inclusions in (44), (45) and Theorem 2.1.Now, we observe that Theorem 5.1 implies that the sequences { ( x k , ηb k ) } and { ( y k , ηa k ) } canbe viewed as generated by the PSM applied to the operators ηA and ηB , with λ k = µ k = 1 and α k = 0 for k = 1 , , . . . . Moreover, in [8, Subsection 4.2] it was proven under assumption (B) that γ k , given in (41), is equal to 1 / k ≥
1. Therefore, the sequences of ergodic iteratesassociated with { ( x k , ηb k ) } , { ( y k , ηa k ) } , { ρ k } and { γ k } , which are obtained by equations (7), (8)and (9) with Γ k = (1 / P k , are exactly as defined in (49) and (50), but with ηb k , ηǫ xk , ηa k and ηǫ yk instead of b k , ǫ xk , a k and ǫ yk , respectively.Hence, applying Theorem 4.2 we have (cid:13)(cid:13) η ( a k + b k ) (cid:13)(cid:13) ≤ d (1 / P k , k x k − y k k ≤ d (1 / P k , η ( ǫ xk + ǫ yk ) ≤ d ( ς k + 4)(1 / P k , (51)where d is the distance of ( z , ηw ) to S e ( ηA, ηB ) and ς k = max j =1 ,...,k (cid:26) µ j θ j (2 − ρ j )(1 / P k (cid:27) .Next, we note that condition (A.2) yields ρ j ≥ − ρ for every j , therefore by the definition of P k we have P k ≥ k (1 − ρ ) . (52)Furthermore, since in this case µ j = 1, θ j = 1 and 2 − ρ j ≥ − ρ for all integer j ≥
1, the definitionof ς k and (52) imply ς k ≤ − ρ ) k ≤ − ρ ) , for k = 1 , , . . . . Hence, the remaining claims in (b) follow combining the bounds in (51) with the inequality aboveand (52).
Corollary 5.1.
Consider the sequences { x k } and { y k } generated by Spingarn’s method and thesequences { x k } and { y k } defined in (49) and (50) , respectively. Then, the following statementshold. a) For every δ > there exists an index i = O (cid:18) max (cid:26) d η δ , d δ (cid:27)(cid:19) such that the pair ( x i , y i ) is a ( δ, -solution of problem (2) .(b) For every δ , ǫ > there exists an index k = O (cid:18) max (cid:26) d ηδ , d δ , d ηǫ (cid:27)(cid:19) such that, for any k ≥ k , the pair ( x k , y k ) is a ( δ, ǫ ) -solution of problem (2) . The PSM has to solve two proximal subproblems on each iteration, in order to construct de-composable separators. Since finding the exact solution of subproblems (18) and (19) could be achallenging task, one might wish to allow approximate evaluations of the resolvent mappings, whilemaintaining convergence of the method.Our main goal in this section is to propose an inexact version of the PSM in the special case oftaking α k = 0 for all iteration k , which possibly allows the subproblems to be performed in parallel.It is customary to appeal to the theory of approximation criteria for the PPA and relatedmethods, when attempting to approximate solutions of proximal subproblems. The first inexactversions of the PPA were introduced in [2] by Rockafellar and are based on absolute summable errorcriteria. For instance, one of the approximation conditions proposed in [2] is (cid:13)(cid:13) z k +1 − ( I + λ k T ) − ( z k ) (cid:13)(cid:13) ≤ s k , ∞ X k =1 s k < ∞ . This kind of approximation criteria, which involves a theoretical sequence { s k } ⊂ [0 , ∞ [ suchthat P ∞ k =1 s k < ∞ , has as a practical disadvantage that there is no direct guidance as to howto select it when solving a specific problem. Therefore, it is useful to construct error conditionsfor approximating proximal subproblems that could be computable during the progress of thealgorithm. Inexact versions of the PPA, which use relative error tolerance criteria of this kind, weredeveloped in [11, 18, 19].To solve subproblems (18) and (19) inexactly, we will use the notion of approximate solutionsof a proximal subproblem proposed in [11] by Solodov and Svaiter.The general projective splitting framework for the sum of m ≥ m maximal monotone operators of the relative errortolerance of the hybrid proximal extragradient method [19]. We have preferred the frameworkdeveloped in [11] since it yields a more flexible error tolerance criterion and evaluation of the ǫ -enlargements of the operators.We now present the notion of inexact solutions of a proximal subproblem introduced in [11]. Let T : R n ⇒ R n be a maximal monotone operator, λ > z ∈ R n . Consider the proximal system (cid:26) w ∈ T ( z ′ ) ,λw + z ′ − z = 0 , (53)which is clearly equivalent to the proximal subproblem (1). Definition 6.1.
Given σ ∈ [0 , z ′ , w, ǫ ) ∈ E is called a σ -approximate solution of (53)at ( λ, z ), if w ∈ T ǫ ( z ′ ) , (cid:13)(cid:13) λw + z ′ − z (cid:13)(cid:13) + 2 λǫ ≤ σ (cid:16) k λw k + (cid:13)(cid:13) z ′ − z (cid:13)(cid:13) (cid:17) . (54)We observe that if ( z ′ , w ) is the exact solution of (53) then, taking ǫ = 0, the triplet ( z ′ , w, ǫ )satisfies the approximation criterion (54) for all σ ∈ [0 , σ = 0, only the exactsolution of (53), with ǫ = 0, will satisfy (54).The method that will be studied in this section is as follows. lgorithm 2. Choose ( z , w ) ∈ R n × R n , σ ∈ [0 , and ρ ∈ [0 , . Then, for k = 1 , , . . .
1. Choose λ k , µ k > and calculate ( x k , b k , ǫ xk ) and ( y k , a k , ǫ yk ) ∈ E such that b k ∈ B ǫ xk ( x k ) , λ k ( b k − w k − ) = z k − − x k + r xk , k r xk k + 2 λ k ǫ xk ≤ σ (cid:0) k x k − z k − k + k λ k ( b k − w k − ) k (cid:1) , (55) and a k ∈ A ǫ yk ( y k ) , µ k ( a k + w k − ) = z k − − y k + r yk , k r yk k + 2 µ k ǫ yk ≤ σ (cid:0) k y k − z k − k + k µ k ( a k + w k − ) k (cid:1) . (56)
2. If k a k + b k k + k x k − y k k = 0 stop. Otherwise, set γ k = h z k − − x k , b k − w k − i + h z k − − y k , a k + w k − i − ǫ xk − ǫ yk k a k + b k k + k x k − y k k .
3. Choose a parameter ρ k ∈ [1 − ρ, ρ ] and set z k = z k − − ρ k γ k ( a k + b k ) ,w k = w k − − ρ k γ k ( x k − y k ) . Note that for all iteration k , the triplet ( x k , b k , ǫ xk ) calculated in step 1 of Algorithm 2 is a σ -approximate solution of (53) at ( λ k , z k − ), where T = B − w k − . Similarly, ( y k , a k , ǫ yk ) is a σ -approximate solution of (53) (with T = A + w k − ) at point ( µ k , z k − ). Observe also that taking σ = 0 in Algorithm 2 yields exactly the PSM with α k = 0 for all integer k ≥
1, since condition (17)is satisfied.Let us denote by φ k the decomposable separator associated with the pair ( x k , b k , ǫ xk ) and( y k , a k , ǫ yk ), for every integer k ≥ k , then it has found a point in S e ( A, B ). Otherwisewe will have k∇ φ k k >
0, which gives φ k ( z k − , w k − ) >
0. This clearly implies that Algorithm 2falls within the general framework of Algorithm 1.
Lemma 6.1.
Let { ( x k , b k , ǫ xk ) } , { ( y k , a k , ǫ yk ) } , { ( z k , w k ) } , { λ k } , { µ k } and { ρ k } be the sequencesgenerated by Algorithm , and { φ k } be the sequence of decomposable separators associated withAlgorithm . Then, for every integer k ≥ , we have φ k ( z k − , w k − ) ≥ − σ ξ k (cid:0) k a k + b k k + k x k − y k k (cid:1) ≥ , (57) where ξ k := min (cid:26) λ k , λ k , µ k , µ k (cid:27) . (58) If k∇ φ k k > , then it follows that φ k ( z k − , w k − ) > . Furthermore, k∇ φ k k = 0 if and only if ( x k , b k ) = ( y k , − a k ) ∈ S e ( A, B ) .Proof. From the definition of φ k and direct calculations it follows that φ k ( z k − , w k − ) = h z k − − x k , b k − w k − i + h z k − − y k , a k + w k − i − ǫ xk − ǫ yk = 12 λ k (cid:16) k z k − − x k k + k λ k ( b k − w k − ) k − k r xk k − λ k ǫ xk (cid:17) + 12 µ k (cid:16) k z k − − y k k + k µ k ( a k + w k − ) k − k r yk k − µ k ǫ yk (cid:17) . The identity above, together with the error criteria (55) and (56), implies φ k ( z k − , w k − ) ≥ − σ λ k (cid:0) k z k − − x k k + k λ k ( b k − w k − ) k (cid:1) + 1 − σ µ k (cid:0) k z k − − y k k + k µ k ( a k + w k − ) k (cid:1) . (59) f we interpret this last expression as a quadratic form applied to the R vector( k z k − − x k k , k b k − w k − k , k z k − − y k k , k a k + w k − k ) T , we obtain φ k ( z k − , w k − ) ≥ − σ k z k − − x k kk w k − − b k kk z k − − y k kk w k − + a k k T λ k λ k µ k
00 0 0 µ k k z k − − x k kk w k − − b k kk z k − − y k kk w k − + a k k ≥ − σ ξ k (cid:0) k z k − − x k k + k b k − w k − k + k z k − − y k k + k a k + w k − k (cid:1) , (60)where ξ k , defined in (58), is the smallest eigenvalue of the matrix in (60).Now, we combine the second inequality in (60) with relations k z k − − x k k + k z k − − y k k ≥ k x k − y k k , k b k − w k − k + k a k + w k − k ≥ k a k + b k k ;to obtain the first inequality in (57). Since ξ k > σ ∈ [0 , φ k ( z k − , w k − ) > k∇ φ k k > φ k ( z k − , w k − ) as φ k ( z k − , w k − ) = h a k + b k , z k − − y k i + h x k − y k , w k − − b k i − ǫ xk − ǫ yk . Then, if k∇ φ k k = 0, it follows that x k = y k , b k = − a k and φ k ( z k − , w k − ) = − ǫ xk − ǫ yk . From equation (57), the equality above and the fact that ǫ xk , ǫ yk ≥
0, we obtain ǫ xk = ǫ yk = 0. Hence, b k ∈ B ( x k ), a k ∈ A ( y k ) and we conclude that ( x k , b k ) = ( y k , − a k ) ∈ S e ( A, B ).For deriving complexity bounds for Algorithm 2 we will assume, as was done in the precedingsection, that the method does not stop in a finite number of iterations. Thus, from now on wesuppose that k∇ φ k k > k ≥ Theorem 6.1.
Take ( z , w ) ∈ R n × R n and let { ( x k , b k , ǫ xk ) } , { ( y k , a k , ǫ yk ) } , { λ k } , { µ k } , { γ k } and { ρ k } be the sequences generated by Algorithm . Let d be the distance of ( z , w ) to the set S e ( A, B ) and, for all integer k ≥ , define ξ k by (58) . Then, for every integer k ≥ , we have b k ∈ B ǫ xk ( x k ) , a k ∈ A ǫ yk ( y k ) , (61) and there exists an index ≤ i ≤ k such that k a i + b i k + k x i − y i k ≤ d (1 − σ ) (1 − ρ ) ξ i k P j =1 ξ j ,ǫ xi + ǫ yi ≤ σd (1 − σ ) (1 − ρ ) k P j =1 ξ j . Proof.
The inclusions in (61) are due to step 1 of Algorithm 2. Since γ k = φ k ( z k − , w k − ) k∇ φ k k , using(57) we have γ k ≥ − σ ξ k for k = 1 , , . . . . (62)Thus, squaring both sides of (62) and multiplying by k∇ φ k k we obtain γ k k∇ φ k k ≥ (cid:18) − σ (cid:19) ξ k k∇ φ k k . (63) ow, we observe that the error criteria (55) and (56) imply ǫ xk ≤ σ λ k (cid:0) k z k − − x k k + k λ k ( b k − w k − ) k (cid:1) and ǫ yk ≤ σ µ k (cid:0) k z k − − y k k + k µ k ( a k + w k − ) k (cid:1) , respectively. Adding these two inequalities and combining with relation (59) we obtain ǫ xk + ǫ yk ≤ σ − σ φ k ( z k − , w k − ) = σ − σ γ k k∇ φ k k . Multiplying the latter inequality by γ k , using (62) and multiplying both sides of the resultingexpression by 1 − σσ , we have(1 − σ ) σ ξ k ( ǫ xk + ǫ yk ) ≤ γ k k∇ φ k k for k = 1 , , . . . . (64)Now, we define ψ k := max ((cid:18) − σ (cid:19) ξ k k∇ φ k k , (1 − σ ) σ ( ǫ xk + ǫ yk ) ) , and combine (63) with (64) to obtain ξ k ψ k ≤ γ k k∇ φ k k for k = 1 , , . . . . Next, adding the inequality above from j = 1 to k , using the assumption that ρ k ∈ [1 − ρ, ρ ] forall integer k ≥ k X j =1 ξ j ψ j ≤ d (1 − ρ ) , and consequently (cid:18) min j =1 ,...,k { ψ j } (cid:19) k X j =1 ξ j ≤ d (1 − ρ ) . The theorem now follows from this last inequality and the definition of ψ k .If { ( x k , b k , ǫ xk ) } , { ( y k , a k , ǫ yk ) } , { γ k } and { ρ k } are the sequences generated by Algorithm 2, weconsider their associated sequences of ergodic iterates { ( x k , b k , ǫ xk ) } and { ( y k , a k , ǫ yk ) } , defined as in(7), (8) and (9). Since Algorithm 2 is a special instance of Algorithm 1, the results of subsection3.2 hold for its ergodic sequences. Thus, combining Theorem 3.2 and Lemma 6.1 we can deriveergodic complexity estimates for the method. Theorem 6.2.
Let { ( x k , b k , ǫ xk ) } , { ( y k , a k , ǫ yk ) } , { γ k } and { ρ k } be the sequences generated by Al-gorithm . Let { ( x k , b k , ǫ xk ) } and { ( y k , a k , ǫ yk ) } be the sequences of ergodic iterates associated withAlgorithm , defined as in (7) - (9) , and consider ξ k given by (58) . Then, for all integer k ≥ , wehave b k ∈ B ǫ xk ( x k ) , a k ∈ A ǫ yk ( y k ) (65) and (cid:13)(cid:13) a k + b k (cid:13)(cid:13) ≤ d Γ k , k x k − y k k ≤ d Γ k , ǫ xk + ǫ yk ≤ d ( ϕ k + 4)Γ k , (66) where d is the distance of ( z , w ) to S e ( A, B ) and ϕ k := (cid:18) − σ (cid:19) max j =1 ,...,k (cid:26) ξ j (2 − ρ j )Γ k (cid:27) . roof. The inclusions in (65) follow from Lemma 3.4. The first two bounds in (66) are due to (12)in Theorem 3.2.Now, we note that the second inequality in (60) implies (cid:0) k z j − − y j k + k b j − w j − k (cid:1) − σ ξ j ≤ φ j ( z j − , w j − ) for j = 1 , , . . . . The relation above, together with the definition of γ j , yields k z j − − y j k + k b j − w j − k ≤ − σ ) ξ j γ j k∇ φ j k for j = 1 , , . . . . Multiplying the above inequality by 1Γ k ρ j γ j and adding from j = 1 to k , we obtain1Γ k k X j =1 ρ j γ j k ( y j , b j ) − ( z j − , w j − ) k ≤ k k X j =1 − σ ) ξ j ρ j γ j k∇ φ j k = 1Γ k k X j =1 − σ ) ξ j (2 − ρ j ) ρ j (2 − ρ j ) γ j k∇ φ j k ≤ ϕ k k X j =1 ρ j (2 − ρ j ) γ j k∇ φ j k ≤ ϕ k d , where the second and the third inequalities are due to the definition of ϕ k and the first bound in(6), respectively. Substituting equation above into (13) we obtain the last bound in (66).Theorems 6.1 and 6.2 provide general complexity results for Algorithm 2. Observe that thederived bounds are expressed in terms of ξ k and Γ k . Next result, which is a direct consequence ofthese theorems, presents iteration-complexity bounds for Algorithm 2 to obtain ( δ, ǫ )-approximatesolutions of problem (2). Theorem 6.3.
Assume the hypotheses of Theorem . . Assume also condition (A.1) and define ξ := min (cid:26) λ, λ (cid:27) . Then, for all δ, ǫ > , the following statements hold.(a) There exists an index i = O (cid:18) max (cid:26) d ξ δ , d ξǫ (cid:27)(cid:19) such that the iterate ( x i , y i ) is a ( δ, ǫ ) -solution of problem (2) .(b) There exists an index k = O (cid:18) max (cid:26) d ξδ , d ξǫ (cid:27)(cid:19) such that, for any k ≥ k , the ergodic iterate ( x k , y k ) is a ( δ, ǫ ) -solution of problem (2) .Proof. We first note that assumption (A.1) implies ξ j ≥ ξ for j = 1 , , . . . . (67)Now, we combine the definition of Γ k in (7) with (62) and (67) to obtainΓ k ≥ k (1 − ρ ) ξ (1 − σ )4 . Furthermore, the inequality above, together with (67) and the definition of ϕ k , yields ϕ k ≤ − ρ ) (1 − σ ) ξ . We conclude the proof combining Theorems 6.1 and 6.2 with these three relations above. A Sequential Inexact Case
In this section, we propose an inexact variant of a sequential case of the PSM and study itsiteration-complexity. We observe that, unless α k = 0, subproblems (18) and (19) cannot be solvedin parallel. For example, if we specialize the PSM by setting α k = 1 for all k , we have to performon each iteration the following steps λ k b k + x k = z k − + λ k w k − , b k ∈ B ( x k ) ,µ k a k + y k = x k − µ k w k − , a k ∈ A ( y k ) . Therefore, the first problem above must be solved after the second one; these steps cannot beperformed simultaneously like the proximal subproblems of Algorithm 2. However, this choice of α k could be an advantage since the second subproblem uses more recent information, that is x k instead of z k − .In this section, we are assuming that the resolvent mappings of operator B are easy to evaluate,but this is not the case for the proximal mappings associated with operator A . Such situationsare typical in practice even in the case of convex optimization. Indeed, if A = ∂f and B = ∂g are the subdifferential operators of functions f and g , where f and g are proper, convex and lowersemicontinuous, then the solutions of the MIP (2) are minimizers of the sum f + g . In this case,in order to evaluate the resolvent mapping ( I + λA ) − = ( I + λ∂f ) − , it is necessary to solvea strongly convex minimization problem and, if f has a complicated algebraic expression, suchproblem could be hard to solve exactly. Therefore, it is desirable to admit inexact solutions of theproximal subproblems associated with this operator.With these assumptions, we propose the following modification of the specific case of the PSMwhere α k = 1 for all iteration k . Specifically, in Algorithm 3 below we allow the solution of thesecond proximal subproblem to be approximated, provided that the approximate solution satisfiesthe relative error condition of Definition 6.1. Algorithm 3.
Choose ( z , w ) ∈ R n × R n , σ ∈ [0 , / and ρ ∈ [0 , . Then, for k = 1 , , . . .
1. Choose λ k > and calculate ( x k , b k ) ∈ R n × R n and ( y k , a k , ǫ yk ) ∈ E such that λ k b k + x k = z k − + λ k w k − , b k ∈ B ( x k ) , (68) and λ k a k + y k = x k − λ k w k − + r k , a k ∈ A ǫ yk ( y k ) , (69) k r k k + 2 λ k ǫ yk ≤ σ (cid:0) k y k − x k k + k λ k ( a k + w k − ) k (cid:1) . (70)
2. If k a k + b k k + k x k − y k k = 0 stop. Otherwise, set γ k = h z k − − x k , b k − w k − i + h z k − − y k , a k + w k − i − ǫ yk k a k + b k k + k x k − y k k .
3. Choose a parameter ρ k ∈ [1 − ρ, ρ ] and set z k = z k − − ρ k γ k ( a k + b k ) ,w k = w k − − ρ k γ k ( x k − y k ) . We note that the maximum tolerance for the relative error in the resolution of (69)-(70) is 1 / k ≥ φ k the decomposable separator associated with the triplets( x k , b k ,
0) and ( y k , a k , ǫ yk ), calculated in step 1 of Algorithm 3 (see Definition 3.1). It is thus clearthat if φ k ( z k − , w k − ) > k ≥
1, then Algorithm 3 is an instance of the generalscheme presented in section 3.The following lemma implies that Algorithm 3 stops in step 2 when it has found a point in theextended solution set S e ( A, B ). emma 7.1. Let { ( x k , b k ) } , { ( y k , a k , ǫ yk ) } , { ( z k , w k ) } , { λ k } and { ρ k } be the sequences generatedby Algorithm , and { φ k } be the sequence of decomposable separators associated with Algorithm .Then, for all integer k ≥ , we have φ k ( z k − , w k − ) ≥ − σ τ k (cid:0) k a k + b k k + k x k − y k k (cid:1) ≥ , (71) where τ k := min (cid:26) λ k , λ k (cid:27) . (72) If k∇ φ k k > , then it follows that φ k ( z k − , w k − ) > . Furthermore, k∇ φ k k = 0 if and only if ( x k , b k ) = ( y k , − a k ) ∈ S e ( A, B ) .Proof. Since φ k ( z k − , w k − ) = h z k − − x k , b k − w k − i + h z k − − y k , a k + w k − i − ǫ yk , adding and subtracting h x k , a k + w k − i on the right-hand side of this equation and regrouping theterms, we obtain φ k ( z k − , w k − ) = h z k − − x k , b k + a k i + h x k − y k , a k + w k − i − ǫ yk = λ k h b k − w k − , b k + a k i + 12 λ k (cid:2) k x k − y k k + k λ k ( a k + w k − ) k (cid:3) − λ k (cid:2) k r k k + 2 λ k ǫ yk (cid:3) , (73)where we have used in the last equality the identity in (68) and r k is given in (69). We observe that λ k h b k − w k − , b k + a k i = λ k (cid:2) k b k − w k − k + k b k + a k k − k a k + w k − k (cid:3) . Hence, combining equality above with (73) and the error criterion (70) we have φ k ( z k − , w k − ) ≥ λ k k b k − w k − k + λ k k a k + b k k + 1 − σ λ k k x k − y k k − σλ k k a k + w k − k . Since k a k + w k − k ≤ k a k + b k k + 2 k b k − w k − k , we deduce that φ k ( z k − , w k − ) ≥ λ k (1 − σ )2 k b k − w k − k + λ k (1 − σ )2 k a k + b k k + 1 − σ λ k k x k − y k k . (74)The inequalities in (71) now follow from the relation above, the definition of τ k and noting that1 − σ ≥ − σ > k∇ φ k k > φ k ( z k − , w k − ) > k∇ φ k k = 0, then x k = y k , b k = − a k and it follows from (71), the first equality in (73) and the fact that ǫ yk ∈ R + , that ǫ yk = 0.Thus, we have ( x k , b k ) ∈ S e ( A, B ).From now on we assume that Algorithm 3 generates infinite sequences { x k } and { y k } , which isequivalent to k∇ φ k k > k ≥ Theorem 7.1.
Take ( z , w ) ∈ R n × R n and let { ( x k , b k ) } , { ( y k , a k , ǫ yk ) } , { λ k } , { γ k } and { ρ k } bethe sequences generated by Algorithm . Let d be the distance of ( z , w ) to S e ( A, B ) and, for allinteger k ≥ , let τ k be given by (72) . Then, for every integer k ≥ , we have b k ∈ B ( x k ) , a k ∈ A ǫ yk ( y k ) , (75) and there exists an index ≤ i ≤ k such that k a i + b i k + k x i − y i k ≤ d (1 − σ ) (1 − ρ ) τ i k P j =1 τ j ,ǫ yi ≤ σd (1 − σ ) (1 − ρ ) k P j =1 τ j . roof. The inclusions in (75) are due to step 1 of Algorithm 3. It follows from the definition of γ k and inequality (71) that γ k ≥ (cid:18) − σ (cid:19) τ k for k = 1 , , . . . . (76)Squaring both sides of the above inequality and multiplying by k∇ φ k k we obtain γ k k∇ φ k k ≥ (cid:18) − σ (cid:19) τ k k∇ φ k k , for k = 1 , , . . . . (77)Now, we note that the error criterion (70) implies ǫ yk ≤ σ λ k (cid:2) k x k − y k k + k λ k ( a k + b k ) k (cid:3) . Consequently, we have ǫ yk ≤ σ λ k k x k − y k k + σλ k k a k + w k − k + σλ k k b k − w k − k . The above inequality, together with (74), yields ǫ yk ≤ σ − σ φ k ( z k − , w k − ) . Next, multiplying the above relation by γ k and combining with (76), after some manipulations, weobtain (1 − σ ) σ τ k ǫ yk ≤ γ k k∇ φ k k . (78)Finally, defining ψ k := max (cid:26) (1 − σ ) τ k k∇ φ k k , (1 − σ ) σ ǫ yk (cid:27) and using (77) and (78), we can conclude the proof proceeding analogously to the proof of Theorem6.1.The following theorem presents complexity estimates in the ergodic sense for Algorithm 3. Theorem 7.2.
Let { ( x k , b k ) } , { ( y k , a k , ǫ yk ) } , { γ k } and { ρ k } be the sequences generated by Algorithm . Let { ( x k , b k , ǫ xk ) } and { ( y k , a k , ǫ yk ) } be the associated sequences of ergodic iterates, defined as in (7) - (9) , and consider τ k given by (72) . Then, for all integer k ≥ , we have b k ∈ B ǫ xk ( x k ) , a k ∈ A ǫ yk ( y k ) , (79) and (cid:13)(cid:13) a k + b k (cid:13)(cid:13) ≤ d Γ k , k x k − y k k ≤ d Γ k , ǫ xk + ǫ yk ≤ d ( ϑ k + 4)Γ k , (80) where d is the distance of ( z , w ) to S e ( A, B ) and ϑ k := max j =1 ,...,k (cid:26) τ j (1 − σ )(2 − ρ j )Γ k (cid:27) . Proof.
Since Algorithm 3 is an instance of Algorithm 1, Lemma 3.4 and Theorem 3.2 apply, thereforethe inclusions in (79) and the first two inequalities in (80) follow.We derive now an estimate for the sum on the right-hand side of (13). We note that (74) implies φ j ( z j − , w j − ) ≥ λ j (cid:18) − σ (cid:19) k b j − w j − k (81)for all integer j ≥
1. We also note that z j − − y j = z j − − x j + x j − y j = λ j ( b j − w j − ) + x j − y j , here the last identity is due to the equality in (68). This last expression and the triangle inequalityfor norms yield k z j − − y j k ≤ λ j k b j − w j − k + k x j − y j k . Moreover, squaring both sides of the inequality above and making some manipulations, we obtain12 λ j k z j − − y j k ≤ λ j k b j − w j − k + 1 λ j k x j − y j k ≤ − σ φ j ( z j − , w j − ) , (82)where the last inequality follows from (74). Now, adding (81) and (82) we have12 λ j k z j − − y j k + λ j k b j − w j − k ≤ − σ φ j ( z j − , w j − ) . The above relation, together with the definitions of γ j and τ j , implies k b j − w j − k + k z j − − y j k ≤ − σ ) τ j γ j k∇ φ j k . Multiplying both sides of the above inequality by 1Γ k ρ j γ j and adding from j = 1 to k , we obtainthe desired estimate, i.e.1Γ k k X j =1 ρ j γ j (cid:2) k b j − w j − k + k z j − − y j k (cid:3) ≤ k k X j =1 − σ ) τ j ρ j γ j k∇ φ j k = 1Γ k k X j =1 − σ ) τ j (2 − ρ j ) ρ j (2 − ρ j ) γ j k∇ φ j k ≤ ϑ k k X j =1 ρ j (2 − ρ j ) γ j k∇ φ j k ≤ ϑ k d , where the second and the third inequalities above follow from the definition of ϑ k and (6), re-spectively. The proof of the last bound in (80) now follows combining the above relation with(13).Next result provides complexity bounds for Algorithm 3 to find a ( δ, ǫ )-approximate solution ofproblem (2). It may be proven in much the same way as Theorem 6.3 and for the sake of brevitywe omit the proof here. Theorem 7.3.
Assume the hypotheses of Theorem . . Suppose also that there exist λ and λ suchthat λ ≥ λ > and λ k ∈ [ λ, λ ] , for all integer k ≥ , and define τ := min (cid:26) λ, λ (cid:27) . Then, for every δ, ǫ > , the following claims hold.(a) There exists an index i = O (cid:18) max (cid:26) d τ δ , d τ ǫ (cid:27)(cid:19) such that the point ( x i , y i ) calculated by Algorithm is a ( δ, ǫ ) -approximate solution of problem (2) .(b) There exists an index k = O (cid:18) max (cid:26) d τ δ , d τ ǫ (cid:27)(cid:19) such that, for any k ≥ k , the ergodic iterate ( x k , y k ) is a ( δ, ǫ ) -approximate solution of problem (2) . Conclusions
We introduced a general projective splitting scheme for solving monotone inclusion problemsgiven by the sum of two maximal monotone operators, which generalizes the family of projectivesplitting methods (PSM) proposed by Eckstein and Svaiter. Using this general framework weanalyzed the iteration-complexity of the family of PSM and, as a consequence, we obtained theiteration-complexity of the two-operator case of the Spingarn partial inverse method. We introducedtwo inexact variants of two special cases of the family of PSM, which allow the resolvent mappingsto be solved inexactly. We also proved the iteration-complexity for the above-mentioned methods.
Acknowledgments
This work is part of the author’s Ph.D. thesis, written under the supervisionof Benar Fux Svaiter at IMPA, and supported by CAPES and FAPERJ.
References
1. Martinet, B.: R´egularisation d’in´equations variationnelles par approximations successives. Rev.Fran¸caise Informat. Recherche Op´erationnelle (Ser. R-3), 154–158 (1970)2. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. ControlOptimization (5), 877–898 (1976)3. Peaceman, D.W., Rachford Jr., H.H.: The numerical solution of parabolic and elliptic differen-tial equations. J. Soc. Indust. Appl. Math. , 28–41 (1955)4. Douglas Jr., J., Rachford Jr., H.H.: On the numerical solution of heat conduction problems intwo and three space variables. Trans. Amer. Math. Soc. , 421–439 (1956)5. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear opera-tors. SIAM J. Numer. Anal. (6), 964–979 (1979). DOI 10.1137/0716071. URL http://dx.doi.org/10.1137/0716071
6. Passty, G.B.: Ergodic convergence to a zero of the sum of monotone operators in Hilbertspace. J. Math. Anal. Appl. (2), 383–390 (1979). DOI 10.1016/0022-247X(79)90234-8. URL http://dx.doi.org/10.1016/0022-247X(79)90234-8
7. Tseng, P.: A modified forward-backward splitting method for maximal monotone mappings.SIAM J. Control Optim. (2), 431–446 (2000). DOI 10.1137/S0363012998338806. URL http://dx.doi.org/10.1137/S0363012998338806
8. Eckstein, J., Svaiter, B.F.: A family of projective splitting methods for the sum of twomaximal monotone operators. Math. Program. (1-2, Ser. B), 173–199 (2008). DOI10.1007/s10107-006-0070-8. URL http://dx.doi.org/10.1007/s10107-006-0070-8
9. Spingarn, J.E.: Partial inverse of a monotone operator. Appl. Math. Optim. (3), 247–265(1983). DOI 10.1007/BF01448388. URL http://dx.doi.org/10.1007/BF01448388
10. Eckstein, J., Svaiter, B.F.: General projective splitting methods for sums of maximal monotoneoperators. SIAM J. Control Optim. (2), 787–811 (2009). DOI 10.1137/070698816. URL http://dx.doi.org/10.1137/070698816
11. Solodov, M.V., Svaiter, B.F.: A unified framework for some inexact proximal point algorithms.Numer. Funct. Anal. Optim. (7-8), 1013–1035 (2001). DOI 10.1081/NFA-100108320. URL http://dx.doi.org/10.1081/NFA-100108320
12. Burachik, R.S., Iusem, A.N., Svaiter, B.F.: Enlargement of monotone operators with appli-cations to variational inequalities. Set-Valued Anal. (2), 159–180 (1997). DOI 10.1023/A:1008615624787. URL http://dx.doi.org/10.1023/A:1008615624787
13. Monteiro, R.D.C., Svaiter, B.F.: On the complexity of the hybrid proximal extragradientmethod for the iterates and the ergodic mean. SIAM J. Optim. (6), 2755–2787 (2010).DOI 10.1137/090753127. URL http://dx.doi.org/10.1137/090753127
14. Burachik, R.S., Sagastiz´abal, C.A., Svaiter, B.F.: ǫ -enlargements of maximal monotoneoperators: theory and applications. In: Reformulation: nonsmooth, piecewise smooth,semismooth and smoothing methods (Lausanne, 1997), Appl. Optim. , vol. 22, pp. 25–43. Kluwer Acad. Publ., Dordrecht (1999). DOI 10.1007/978-1-4757-6388-1 2. URL http://dx.doi.org/10.1007/978-1-4757-6388-1_2
5. Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibilityproblems. SIAM Rev. (3), 367–426 (1996). DOI 10.1137/S0036144593251710. URL http://dx.doi.org/10.1137/S0036144593251710
16. Minty, G.J.: Monotone (nonlinear) operators in Hilbert space. Duke Math. J. , 341–346(1962)17. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximalpoint algorithm for maximal monotone operators. Math. Programming (3, Ser. A), 293–318(1992). DOI 10.1007/BF01581204. URL http://dx.doi.org/10.1007/BF01581204
18. Solodov, M.V., Svaiter, B.F.: A hybrid projection-proximal point algorithm. J. Convex Anal. (1), 59–70 (1999)19. Solodov, M.V., Svaiter, B.F.: A hybrid approximate extragradient-proximal point algorithmusing the enlargement of a maximal monotone operator. Set-Valued Anal. (4), 323–345 (1999).DOI 10.1023/A:1008777829180. URL http://dx.doi.org/10.1023/A:1008777829180http://dx.doi.org/10.1023/A:1008777829180