Attouch-Théra duality revisited: paramonotonicity and operator splitting
Heinz H. Bauschke, Radu I. Bot, Warren L. Hare, Walaa M. Moursi
aa r X i v : . [ m a t h . O C ] O c t A TTOUCH -T H ´ ERA DUALITY R EVISITED : PAR AMONOTONICITY AND OPER ATOR SPLITTING
Heinz H. Bauschke ∗ , Radu I. Bot¸ † , Warren L. Hare ‡ , and Walaa M. Moursi § October 21, 2011
Abstract
The problem of finding the zeros of the sum of two maximally monotone operators is of fundamentalimportance in optimization and variational analysis. In this paper, we systematically study Attouch-Th´era duality for this problem. We provide new results related to Passty’s parallel sum, to Eckstein andSvaiter’s extended solution set, and to Combettes’ fixed point description of the set of primal solutions.Furthermore, paramonotonicity is revealed to be a key property because it allows for the recovery of all primal solutions given just one arbitrary dual solution. As an application, we generalize the best approx-imation results by Bauschke, Combettes and Luke [J. Approx. Theory 141 (2006), 63–69] from normalcone operators to paramonotone operators. Our results are illustrated through numerous examples.
Primary 41A50, 47H05, 90C25; Secondary 47J05, 47H09, 47N10,49M27, 49N15, 65K05, 65K10, 90C46.
Keywords:
Attouch-Th´era duality, Douglas-Rachford splitting, Eckstein-Ferris-Robinson duality, Fenchel duality,Fenchel-Rockafellar duality, firmly nonexpansive mapping, fixed point, Hilbert space, maximal monotone operator,nonexpansive mapping, paramonotonicity, resolvent, subdifferential operator, total duality.
Throughout this paper,(1) X is a real Hilbert space with inner product h· , ·i ∗ Mathematics, University of British Columbia, Kelowna, B.C. V1V 1V7, Canada. E-mail: [email protected] . † Technische Universit¨at Chemnitz, Fakult¨at f¨ur Mathematik, 09107 Chemnitz, Germany. E-mail: [email protected] . ‡ Mathematics, University of British Columbia, Kelowna, B.C. V1V 1V7, Canada. E-mail: [email protected] . § Mathematics, University of British Columbia, Kelowna, B.C. V1V 1V7, Canada. E-mail: [email protected] . k · k .Let A : X ⇒ X be a set-valued operator, i.e., ( ∀ x ∈ X ) Ax ⊆ X . Recall that A is monotone if(2) ( ∀ ( x , x ∗ ) ∈ gr A )( ∀ ( y , y ∗ ) ∈ gr A ) h x − y , x ∗ − y ∗ i ≥ A is maximally monotone if it is impossible to properly enlarge the graph of A while keepingmonotonicity. Monotone operators continue to play an important role in modern optimization and variationalanalysis; see, e.g., [5], [11], [14], [16], [36], [37], [38], [39], [42], [44], [45], and [46]. This is due to the factthat subdifferential operators of proper lower semicontinuous convex functions are maximally monotone,as are continuous linear operators with a monotone symmetric part. The sum of two maximally monotoneoperators is monotone, and often maximally monotone if an appropriate constraint qualification is imposed.Finding the zeros of two maximally monotone operators A and B , i.e., determining(3) ( A + B ) − = (cid:8) x ∈ X (cid:12)(cid:12) ∈ Ax + Bx (cid:9) , is a problem of great interest because it covers constrained convex optimization, convex feasibility, and manyothers. Attouch and Th´era provided a comprehensive study of this (primal) problem in terms of duality .Specifically, they associated with the primal problem a dual problem. We set B − > = ( − Id ) ◦ B − ◦ ( − Id ) where Id : X → X : x x is the identity operator. The Attouch-Th´era dual problem is then to determine(4) (cid:0) A − + B − > (cid:1) − = (cid:8) x ∗ ∈ X (cid:12)(cid:12) ∈ A − x ∗ + B − > x ∗ (cid:9) . This duality is very beautiful; e.g., the dual of the dual problem is the primal problem, and the primalproblem possesses at least one solution if and only if the same is true for the dual problem.
Our goal in this paper is to systematically study Attouch-Th´era duality, to derive new results, and toexpose new applications.
Let us now summarize our main results. • We observe a curious convexity property of the intersection of two sets involving the graphs of A and B (see Theorem 3.3). This relates to Passty’s work on the parallel sum as well as to Eckstein andSvaiter’s work on the extended solution set. • We provide a new description of the fixed point set of the Douglas-Rachford splitting operator (seeTheorem 4.5); this refines Combettes’ description of ( A + B ) − • We reveal the importance of paramonotonicity: in this case, the fixed point set of the Douglas-Rachford splitting operator is a rectangle (see Corollary 5.6) and it is possible to recover all primalsolutions from one dual solution (see Theorem 5.3). • We generalize the best approximation results by Bauschke-Combettes-Luke from normal cone opera-tors to paramonotone operators with a common zero (see Corollary 6.8 and Theorem 8.1).The remainder of this paper is organized as follows. In Section 2, we review and slightly refine the basicresults on Attouch-Th´era duality. The solution mappings between primal and dual solutions are studied2n Section 3. Section 4 deals with the Douglas-Rachford splitting operator. The results in Section 5 andSection 6 underline the importance of paramonotonicity in the understanding of the zeros of the sum. Ap-plications to best approximation as well as comments on other duality framework are the topic of the finalSection 8.We conclude this introductory section with some notational comments. The set of zeros of A is written aszer A = A −
0. The resolvent and reflected resolvent is defined by(5) J A = ( Id + A ) − and R A = J A − Id , respectively. It is well known that zer A = Fix J A : = (cid:8) x ∈ X (cid:12)(cid:12) J A x = x (cid:9) . Moreover, J A is firmly nonexpansiveif and only if R A is nonexpansive; see, e.g., [20], [28], or [30]. We also have the inverse resolvent identity (6) J A + J A − = Idand the following very useful Minty parametrization.
Fact 1.1 (Minty parametrization)
Let A : X ⇒ X be maximally monotone. Then gr A → X : ( a , a ∗ ) a + a ∗ is a continuous bijection with continuous inverse x ( J A x , x − J A x ) ; thus, (7) gr A = (cid:8) ( J A x , x − J A x ) (cid:12)(cid:12) x ∈ X (cid:9) . Without explicit mentioning it, we employ standard notation from convex analysis (see [36], [37], or [42]).Most importantly, f ∗ denotes the Fenchel conjugate of a function f , and ¶ f its subdifferential operator . Theset of all convex lower semicontinuous proper functions on X is denoted by G (or G X if we need to emphasizethe space). Finally, we set f ∨ : = f ◦ ( − Id ) , which yields ¶ ( f ∨ ) = ( ¶ f ) > . In this paper, we study the problem of finding zeros of the sum of maximally monotone operators. Morespecifically, we assume that(8) A and B are maximally monotone operators on X . Definition 2.1 (primal problem)
The primal problem , for the ordered pair ( A , B ) , is to find the zeros ofA + B. At first, it looks strange to define the primal problem with respect to the (ordered) pair ( A , B ) . Thereason we must do this is to associate a unique dual problem. (The ambiguity arises because addition iscommutative.) It will be quite convenient to set(9) A > = ( − Id ) ◦ A ◦ ( − Id ) .
3n easy calculation shows that ( A − ) > = ( A > ) − , which motivates the notation(10) A − > : = (cid:0) A − (cid:1) > = (cid:0) A > (cid:1) − . (This is similar to the linear-algebraic notation A − T for invertible square matrices.)Now since A and B form a pair of maximally monotone operators, so do A − and B − > : We thus definethe dual pair (11) ( A , B ) ∗ : = ( A − , B − > ) . The biduality(12) ( A , B ) ∗∗ = ( A , B ) holds, since ( A − ) − = A , ( B > ) > = B , and ( B > ) − = ( B − ) > .We are now in a position to formulate the dual problem. Definition 2.2 ((Attouch-Th´era) dual problem)
The (Attouch-Th´era) dual problem , for the ordered pair ( A , B ) , is to find the zeros of A − + B − > . Put differently: the dual problem for ( A , B ) is precisely the primalproblem for ( A , B ) ∗ . In view of (12), it is the clear that the primal problem is precisely the dual of the dual problem, asexpected. One central aim of this paper is to understand the interplay between the primal and dual solutionsthat we formally define next.
Definition 2.3 (primal and dual solutions)
The primal solutions are the solutions to the primal problemand analogously for the dual solutions . We shall abbreviate these sets by (13) Z : = ( A + B ) − ( ) and K : = (cid:0) A − + B − > (cid:1) − ( ) , respectively. As observed by Attouch and Th´era in [1, Corollary 3.2], one has:(14) Z = ∅ ⇔ K = ∅ . Let us make this simple but important equivalence a little more precise. In order to do so, we define(15) ( ∀ z ∈ X ) K z : = ( Az ) ∩ ( − Bz ) ( ∀ k ∈ X ) Z k : = ( A − k ) ∩ ( − B − > k ) = ( A − k ) ∩ (cid:0) B − ( − k ) (cid:1) . As the next proposition illustrates, these objects are intimately tied to primal and dual solutions definedin Definition 2.3.
Proposition 2.4
Let z ∈ X and let k ∈ X . Then the following hold. (i) K z and Z k are closed convex (possibly empty) subsets of X . (ii) k ∈ K z ⇔ z ∈ Z k ; (iii) z ∈ Z ⇔ K z = ∅ . (iv) S z ∈ Z K z = K. (v) Z = ∅ ⇔ K = ∅ . (vi) k ∈ K ⇔ Z k = ∅ . (vii) S k ∈ K Z k = Z.Proof . (i): Because A and B are maximally monotone, the sets Az and Bz are closed and convex. Hence K z is also closed and convex. We see analogously that Z k is closed and convex as well.(ii): This is easily verified from the definitions.(iii): Indeed, z ∈ Z ⇔ ∈ ( A + B ) z ⇔ ( ∃ a ∗ ∈ Az ∩ ( − Bz )) ⇔ ( ∃ a ∗ ∈ K z ) ⇔ K z = ∅ .(iv): Take k ∈ S z ∈ Z K z . Then there exists z ∈ Z such that k ∈ K z = Az ∩ ( − Bz ) . Hence z ∈ A − k and z ∈ ( − B ) − k = B − ( − Id ) − k = B − ( − k ) . Thus z ∈ A − k and − z ∈ B − > k . Hence 0 ∈ ( A − + B − > ) k and so k ∈ K . The reverse inclusion is proved analogously.(v): Combine (iii) and (iv).(vi)&(vii): The proofs are analogous to the ones of (iii)&(iv). (cid:4) Let us provide some examples illustrating these notions.
Example 2.5
Suppose that X = R , and that we consider the rotators by ∓ p /
2, i.e.,(17) A : R → R : ( x , x ) ( x , − x ) and B : R → R : ( x , x ) ( − x , x ) . Note that B = − A = A − = A ∗ , where A ∗ denote the adjoint operator. Hence A + B ≡ Z = X , and ( ∀ z ∈ Z ) K z = { Az } = {− Bz } . Furthermore, A − = B while the linearity of B implies that B − > = B − = − B = A .Therefore, ( A , B ) ∗ = ( B , A ) . Hence K = Z , while ( ∀ k ∈ K ) Z k = { A − k } = { Bk } .5 xample 2.6 Suppose that X = R , that A is the normal cone operator of R + , and that B : X → X : ( x , x ) ( − x , x ) is the rotator by p /
2. As already observed in Example 2.5, we have B − = − B and B − > = B − = − B . A routine calculation yields(18) Z = R + × { } ;thus, since B is single-valued,(19) ( ∀ z = ( z , ) ∈ Z ) K z = (cid:8) − Bz (cid:9) = (cid:8) ( , − z ) (cid:9) . Thus,(20) K = [ z ∈ Z K z = { } × R − and so(21) ( ∀ k = ( , k ) ∈ K ) Z k = (cid:8) − B − > k (cid:9) = (cid:8) Bk (cid:9) = (cid:8) ( − k , ) (cid:9) . The dual problem is to find the zeros of A − + B − > , i.e., the zeros of the sum of the normal cone operatorof the negative orthant and the rotator by − p / Example 2.7 (convex feasibility)
Suppose that A = N U and B = N V , where U and V are closed convexsubsets of X such that U ∩ V = ∅ . Then clearly Z = U ∩ V . Using [7, Proposition 2.4.(i)], we deducethat ( ∀ z ∈ Z ) K z = N U − V ( ) = K . Note that we do know at least one dual solution: 0 ∈ K . Thus, byProposition 2.4(ii)&(vii), ( ∀ k ∈ K ) Z k = Z . Remark 2.8
The preceding examples give some credence to the conjecture that(22) z ∈ Zz ∈ Zz = z ⇒ either K z = K z or K z ∩ K z = ∅ .Note that (22) is trivially true whenever A or B is at most single-valued. While this conjecture fails ingeneral (see Example 2.9 below), it does, however, hold true for the large class of paramonotone operators(see Theorem 5.3). Example 2.9
Suppose that X = R , and set U : = R × R + , V = R × { } , and R : X → X : ( x , x ) ( − x , x ) . Now suppose that A = N U + R and that B = N V . Then dom A = U and dom B = V ; hence,dom ( A + B ) = U ∩ V = V . Let x = ( x , ) ∈ V . Then Ax = { } × ] − ¥ , x ] and Bx = { } × R . Hence Ax ⊂ ± Bx , ( A + B ) x = { } × R and therefore Z = V . Furthermore, K x = Ax ∩ ( − Bx ) = Ax . Now take y = ( h , ) ∈ V = Z with x < h . Then K x = Ax $ Ay = K y and thus (22) fails. Proposition 2.10 (common zeros) zer A ∩ zer B = ∅ ⇔ ∈ K.Proof . Suppose first that z ∈ zer A ∩ zer B . Then 0 ∈ Az and 0 ∈ Bz , so 0 ∈ Az ∩ ( − Bz ) = K z ⊆ K . Nowassume that 0 ∈ K . Then 0 ∈ K z , for some z ∈ Z and so 0 ∈ Az ∩ ( − Bz ) . Therefore, 0 ∈ zer A ∩ zer B . (cid:4) xample 2.11 Suppose that B = A . Then Z = zer A , and zer A = ∅ ⇔ ∈ K . Proof . Since 2 A is maximally monotone and A + A is a monotone extension of 2 A , we deduce that that A + A = A . Hence Z = zer ( A ) = zer A and the result follows from Proposition 2.10. (cid:4) The following result, observed first by Passty, is very useful. For the sake of completeness, we include itsshort proof.
Proposition 2.12 (Passty)
Suppose that, for every i ∈ { , } , w i ∈ Ay i ∩ B ( x − y i ) . Then h y − y , w − w i = .Proof . (See [33, Lemma 14].) Since A is monotone, 0 ≤ h y − y , w − w i . On the other hand, since B ismonotone, 0 ≤ h ( x − y ) − ( x − y ) , w − w i = h y − y , w − w i . Altogether, h y − y , w − w i = (cid:4) Corollary 2.13
Suppose that z and z belong to Z, that k ∈ K z , and that k ∈ K z . Then h k − k , z − z i = .Proof . Apply Proposition 2.12 (with B replaced by B > and at x = (cid:4) We now interpret the families of sets ( K z ) z ∈ X and ( Z k ) k ∈ X as set-valued operators by setting(23) K : X ⇒ X : z K z and Z : X ⇒ X : k Z k . Let us record some basic properties of these fundamental operators.
Proposition 3.1
The following hold. (i) gr K = gr A ∩ gr ( − B ) and gr Z = gr A − ∩ gr ( − B − > ) . (ii) dom K = Z, ran K = K, dom Z = K, and ran Z = Z. (iii) gr K and gr Z are closed sets. (iv) The operators K , − K , Z , − Z are monotone. (v) K − = Z . roof . (i): This is clear from the definitions.(ii): This follows from Proposition 2.4.(iii): Since A and B are maximally monotone, the sets gr A and gr B are closed. Hence, by (i), gr K isclosed and similarly for gr Z .(iv): Since gr K ⊆ gr A and A is monotone, we see that K is monotone. Similarly, since B is monotoneand gr ( − K ) ⊆ gr B , we obtain the monotonicity of − K . The proofs for ± Z are analogous.(v): Clear from Proposition 2.4(ii). (cid:4) In Proposition 2.4(iii) we observed the closedness and convexity of K z and Z k . In view of Proposi-tion 2.4(iii)&(vii), the sets of primal and dual solutions are both unions of closed convex sets. It would seemthat we cannot a priori deduce convexity of these solution sets because unions of convex sets need not beconvex. However, not only are Z and K indeed convex, but so are gr Z and gr K . This surprising result,which is basically contained in works by Passty [33] and by Eckstein and Svaiter [22, 23], is best stated byusing the parallel sum, a notion systematically explored by Passty in [33]. Definition 3.2 (parallel sum)
The parallel sum of A and B is (24) A (cid:3) B : = ( A − + B − ) − . The notation we use for the parallel sum (see [5, Section 24.4]) is nonstandard but highly convenient: indeed,for sufficiently nice convex functions f and g , one has ¶ ( f (cid:3) g ) = ( ¶ f ) (cid:3) ( ¶ g ) (see [33, Theorem 28], [31,Proposition 4.2.2], or [5, Proposition 24.27]).The proof of the following result is contained in the proof of [33, Theorem 21], although Passty stated amuch weaker conclusion. For the sake of completeness, we present his proof. Theorem 3.3
For every x ∈ X , the set (25) (cid:0) gr A (cid:1) ∩ (cid:0) ( x , ) − gr ( − B ) (cid:1) = (cid:8) ( y , w ) ∈ gr A (cid:12)(cid:12) ( x − y , w ) ∈ gr B (cid:9) is convex.Proof . (See also [33, Proof of Theorem 21].) The identity (25) is easily verified. To tackle convexity, forevery i ∈ { , } take ( y i , w i ) from the intersection (25); equivalently,(26) ( ∀ i ∈ { , } ) w i ∈ Ay i ∩ B ( x − y i ) . By Proposition 2.12,(27) h y − y , w − w i = . Now let t ∈ [ , ] , set ( y t , w t ) = ( − t )( y , w ) + t ( y , w ) , and take ( a , a ∗ ) ∈ gr A . Using (26) and themonotonicity of A in (28d), we obtain h y t − a , w t − a ∗ i = h ( − t )( y − a ) + t ( y − a ) , ( − t )( w − a ∗ ) + t ( w − a ∗ ) i (28a) 8 ( − t ) h y − a , w − a ∗ i + t h y − a , w − a ∗ i (28b) + ( − t ) t (cid:0) h y − a , w − a ∗ i + h y − a , w − a ∗ i (cid:1) (28c) ≥ ( − t ) t (cid:0) h y − a , w − a ∗ i + h y − a , w − a ∗ i (cid:1) . (28d)Thus, using again monotonicity of A and recalling (27), we obtain h y − a , w − a ∗ i + h y − a , w − a ∗ i = h y − a , w − w i + h y − a , w − a ∗ i (29a) + h y − a , w − w i + h y − a , w − a ∗ i (29b) = h y − y , w − w i (29c) + h y − a , w − a ∗ i + h y − a , w − a ∗ i (29d) ≥ h y − y , w − w i (29e) = . (29f)Combining (28) and (29), we obtain h y t − a , w t − a ∗ i ≥
0. Since ( a , a ∗ ) is an arbitrary element of gr A and A is maximally monotone, we deduce that ( y t , w t ) ∈ gr A . A similar argument yields ( x − y t , w t ) ∈ gr B .Therefore, ( y t , w t ) is an element of the intersection (25). (cid:4) Before returning to the objects of interest, we record Passty’s [33, Theorem 21] as a simple corollary.
Corollary 3.4 (Passty)
For every x ∈ X , the set ( A (cid:3) B ) x is convex.Proof . Let x ∈ X . Since ( y , w ) → w is linear and (cid:8) ( y , w ) ∈ gr A (cid:12)(cid:12) ( x − y , w ) ∈ gr B (cid:9) is convex (Theorem 3.3),we deduce that(30) (cid:8) w ∈ X (cid:12)(cid:12) ( ∃ y ∈ X ) w ∈ Ay ∩ B ( x − y ) (cid:9) is convex.On the other hand, a direct computation or [33, Lemma 2] implies that ( A (cid:3) B ) x = S y ∈ X (cid:0) Ay ∩ B ( x − y ) (cid:1) .Altogether, ( A (cid:3) B ) x is convex. (cid:4) Corollary 3.5
For every x ∈ X , the set ( gr A ) ∩ (( x , ) + gr ( − B )) is convex.Proof . On the one hand, − gr ( − B > ) = gr ( − B ) . On the other hand, B > is maximally monotone. Altogether,Theorem 3.3 (applied with B > instead of B ) implies that ( gr A ) ∩ (( x , ) − gr ( − B > )) = ( gr A ) ∩ (( x , ) + gr ( − B )) is convex. (cid:4) Remark 3.6
Theorem 3.3 and Corollary 3.5 imply that the intersections ( gr A ) ∩ ± ( gr − B ) are convex.This somewhat resembles works by Mart´ınez-Legaz (see [29, Theorem 2.1]) and by Z˘alinescu [43], whoencountered convexity when studying the Minkowski sum/difference ( gr A ) ± ( gr − B ) . Corollary 3.7 (convexity)
The sets gr Z and gr K are convex; consequently, Z and K are convex.Proof . Combining Proposition 3.1(i) and Corollary 3.5 (with x = K . Hencegr Z is convex by Proposition 3.1(v). It thus follows that Z and K are convex as images of convex sets underlinear transformations. (cid:4) emark 3.8 Since Z = ( A − (cid:3) B − )( ) and K = ( A (cid:3) B > )( ) , the convexity of Z and K also follows fromCorollary 3.4. Remark 3.9 (connection to Eckstein and Svaiter’s “extended solution set”)
In [22, Section 2.1], Eck-stein and Svaiter defined in 2008 the extended solution set (for the primal problem) by(31) S e ( A , B ) : = (cid:8) ( z , w ) ∈ X × X (cid:12)(cid:12) w ∈ Bz , − w ∈ Az (cid:9) . It is clear that gr Z − = gr K = S e ( B , A ) = − S e ( A , B ) . Unaware of Passty’s work, they proved in [22,Lemma 1 and Lemma 2] (in the present notation) that Z = ran Z , and that gr Z is closed and convex. Theirproof is very elegant and completely different from the above Passty-like proof. In their 2009 follow-up pa-per [23], Eckstein and Svaiter generalize the notion of the extended solution set to three or more operators;their corresponding proof of convexity in [23, Proposition 2.2] is more direct and along Passty’s lines. Remark 3.10 (convexity of Z and K ) If Z is nonempty and a constraint qualification holds, then A + B ismaximally monotone (see, e.g., [5, Section 24.1]) and therefore Z = zer ( A + B ) is convex. It is somewhatsurprising that Z is always convex even without the maximal monotonicity of A + B .One may inquire whether or not Z is also closed, which is another standard property of zeros of maximallymonotone operators. The next example illustrates that Z may fail to be closed. Example 3.11 ( Z need not be closed!) Suppose that X = ℓ , the real Hilbert space of square-summablesequences. In [10, Example 3.17], the authors provide a monotone discontinuous linear at most single-valued operator S on X such that S is maximally monotone and its adjoint S ∗ is a maximally monotonesingle-valued extension of − S . Hence dom S is not closed. Now assume that A = S and B = S ∗ . Then A + B is operator that is zero on the dense proper subspace Z = dom ( A + B ) = dom S of X . Thus Z fails to beclosed. Furthermore, in the language of Passty’s parallel sums (see Remark 3.8), this also illustrates that theparallel sum need not map a point to a closed set. Remark 3.12
We do not know whether or not such counterexamples can reside in finite-dimensional Hilbertspaces when dom A ∩ dom B = ∅ . On the one hand, in view of the forthcoming Corollary 5.5(i), any coun-terexample must feature at least one operator that is not paramonotone, which means that the operators cannot be simultaneously subdifferential operators of functions in G . On the other hand, one has to avoidthe situation when A + B is maximally monotone, which happens when ri dom A ∩ ri dom B = ∅ . This meansthat neither is one of the operators allowed to have full domain, nor can they simultaneously have relativelyopen domains, which excludes the situation when both operators are maximally monotone linear relations(i.e., maximally monotone operators with graphs that are linear subspaces, see [9]). Remark 3.13
We note that K and Z are in general not maximally monotone. Indeed if Z , say, is maximallymonotone, then Corollary 3.7 and [9, Theorem 4.2] imply that gr Z is actually affine (i.e., a translate of asubspace) and so are Z and K (as range and domain of Z ). However, the set Z of Example 2.7 need not bean affine subspace (e.g., when U , V and Z coincide with the closed unit ball in X ).10 Reflected Resolvents and Splitting Operators
We start with some useful identities involving resolvents and reflected resolvents (recall (5)).
Proposition 4.1
Let C : X ⇒ X be maximally monotone. Then the following hold. (i) R C − = − R C . (ii) J C > = J > C . (iii) R C − > = Id − J > C .Proof . (i): By (6), we have R C − = J C − − Id = ( Id − J C ) − Id = Id − J C = − ( J C − Id ) = − R C .(ii): Indeed, J C > = (cid:0) Id +( − Id ) ◦ C ◦ ( − Id ) (cid:1) − (32a) = (cid:0) ( − Id ) ◦ ( Id + C ) ◦ ( − Id ) (cid:1) − (32b) = ( − Id ) − ◦ ( Id + C ) − ◦ ( − Id ) − (32c) = ( − Id ) ◦ J C ◦ ( − Id ) (32d) = J > C . (32e)(iii): Using (6) and (ii), we have that R C − > = J C − > − Id = ( Id − J C > ) − Id = Id − J > C . (cid:4) Corollary 4.2 (Peaceman-Rachford operator is self-dual) (See
Eckstein ’s [19, Lemma 3.5 on page 125].)
The Peaceman-Rachford operators for ( A , B ) and ( A , B ) ∗ = ( A − , B − > ) coincide, i.e., we have self-duality in the sense that (33) R B R A = R B − > R A − . Consequently, (34) ( ∀ l ∈ [ , ]) ( − l ) Id + l R B R A = ( − l ) Id + l R B − > R A − . Proof . Using Proposition 4.1(i)&(iii), we obtain (33) R B − > R A − = ( Id − J > B )( − R A ) = − R A + J B R A =( J B − Id ) R A = R B R A . Now (34) follows immediately from (33). (cid:4) Corollary 4.3 (Douglas-Rachford operator is self-dual) (See
Eckstein ’s [19, Lemma 3.6 on page 133].)
For the
Douglas-Rachford operator(35) T ( A , B ) : = Id + R B R A we have (36) T ( A , B ) = J B R A + Id − J A = T ( A − , B − > ) . roof . The left equality is a simple expansion while self-duality is (34) with l = . (cid:4) Remark 4.4 (backward-backward operator is not self-dual)
In contrast to Corollary 4.2, the backward-backward operator is not self-dual: indeed, using (6) and Proposition 4.1(iii), we deduce that(37) J B − > J A − = ( Id − J > B )( Id − J A ) = Id − J A + J B ( J A − Id ) = ( J B − Id )( J A − Id ) . Thus if A ≡ B is not a singleton (equivalently, J A = Id and ran J B is not a singleton), then J B − > J A − ≡ ( J B − Id ) ≡ J B = J B = J B J A .For the rest of this paper, we set(38) T = Id + R B R A = J B R A + Id − J A . Clearly,(39) Fix T = Fix R B R A . Theorem 4.5
The mapping (40) Y : gr K → Fix T : ( z , k ) z + kis a well defined bijection that is continuous in both directions, with Y − : x ( J A x , x − J A x ) .Proof . Take ( z , k ) ∈ gr K . Then k ∈ K z = ( Az ) ∩ ( − Bz ) . Now k ∈ Az ⇔ z + k ∈ ( Id + A ) z ⇔ z = J A ( z + k ) ,and k ∈ ( − Bz ) ⇔ − k ∈ Bz ⇔ z − k ∈ ( Id + B ) z ⇔ z = J B ( z − k ) . Set x : = z + k . Then J A x = J A ( z + k ) = z and hence R A x = J A x − x = z − ( z + k ) = z − k . Thus,(41) T x = x − J A x + J B R A x = z + k − z + J B ( z − k ) = k + z = x , i.e., x ∈ Fix T . It follows that Y is well defined .Let us now show that Y is surjective . To this end, take x ∈ Fix T . Set z : = J A x as well as k : = ( Id − J A ) x = x − z . Clearly,(42) x = z + k . Now z = J A x ⇔ x ∈ ( Id + A ) z = z + Az ⇔ k = x − z ∈ Az . Thus,(43) k ∈ Az . We also have R A x = J A x − x = z − ( z + k ) = z − k ; hence, x = T x = x − J A x + J B R A x ⇔ J A x = J B R A x ⇔ z = J B ( z − k ) ⇔ z − k ∈ ( Id + B ) z = z + Bz ⇔ (44) k ∈ − Bz . k ∈ ( Az ) ∩ ( − Bz ) = K z ⇔ ( z , k ) ∈ gr K . Hence Y is surjective.In view of Fact 1.1 and since gr K ⊆ gr A , it is clear that Y is injective with the announced inverse. (cid:4) The following result is a straight-forward consequence of Theorem 4.5.
Corollary 4.6
We have (45) ( ∀ z ∈ Z ) K z = J A − (cid:0) J − A z ∩ Fix T (cid:1) and (46) ( ∀ k ∈ K ) Z k = J A ( J − A − k ∩ Fix T (cid:1) . Corollary 4.7 (Combettes) (see [18, Lemma 2.6(iii)]) J A ( Fix T ) = Z.Proof . Set Q : X × X → X : ( x , x ) x . By Theorem 4.5 and Proposition 3.1(ii), J A ( Fix T ) = Q ran Y − = Q dom Y = Q ( gr K ) = dom K = Z . (cid:4) Example 4.8 (see also [6, Fact A1]) Suppose that A = N U and B = N V , where U and V are closed convexsubsets of X such that U ∩ V = ∅ . Then P U ( Fix T ) = U ∩ V . Corollary 4.9 ( Id − J A )( Fix T ) = K.Proof . Either argue similarly to the proof of Corollary 4.7, or apply Corollary 4.7 to the dual and recall that T is self-dual by Corollary 4.3. (cid:4) Definition 5.1
A monotone operator C : X ⇒ X is paramonotone , if (47) x ∗ ∈ Cxy ∗ ∈ Cy h x − y , x ∗ − y ∗ i = ⇒ x ∗ ∈ Cy and y ∗ ∈ Cx . Remark 5.2
Paramonotonicity has proven to be a very useful property for finding solution of variationalinequalities by iteration; see, e.g., [27], [17], [15], [32], and [24]. Examples of paramonotone operatorsabound: indeed, each of the following is paramonotone.(i) ¶ f , where f ∈ G [27, Proposition 2.2].(ii) C : X ⇒ X , where C is strictly monotone.(iii) R n → R n : x Cx + b , where C ∈ R n × n , b ∈ R n , C + = C + C T , ker C + ⊆ ker C , and C + is positivesemidefinite [27, Proposition 3.1]. 13or further examples, see [27]. When C is a continuous linear monotone operator, then C is paramonotoneif and only if C is rectangular (a.k.a. 3* monotone); see [3, Section 4]. It is straight-forward to check thatfor C : X ⇒ X , we have C is paramonotone ⇔ C − is paramonotone ⇔ C > is paramonotone(48) ⇔ C − > is paramonotone. Theorem 5.3
Suppose that A and B are paramonotone. Then ( ∀ z ∈ Z ) K z = K and ( ∀ k ∈ K ) Z k = Z.Proof . Suppose that z and z belong to Z and that z = z . Take k ∈ K z = Az ∩ ( − Bz ) and k ∈ K z = Az ∩ ( − Bz ) . By Corollary 2.13,(49) h k − k , z − z i = . Since A and B are paramonotone, we have k ∈ Az and − k ∈ Bz ; equivalently, k ∈ K z . It follows that K z ⊆ K z . Since the reverse inclusion follows in the same fashion, we see that K z = K z . In view ofProposition 2.4(iv), K z = K , which proves the first conclusion. Since A and B are paramonotone so are A − and B − > by (48). Therefore, the second conclusion follows from what we already proved (applied to A − and B − > ). (cid:4) Remark 5.4 (recovering all primal solutions from one dual solution)
Suppose that A and B are para-monotone and we know one (arbitrary) dual solution, say k ∈ K . Then(50) Z k = A − k ∩ (cid:0) B − ( − k ) (cid:1) recovers the set Z of all primal solutions, by Theorem 5.3. If A = ¶ f and B = ¶ g , where f and g belong to G , then, since ( ¶ f ) − = ¶ f ∗ and ( ¶ g ) − = ¶ g ∗ , we obtain a formula well known in Fenchel duality , namely,(51) Z = ¶ f ∗ ( k ) ∩ ¶ g ∗ ( − k ) . We shall revisit this setting in more detail in Section 7. In striking contrast, the complete recovery of allprimal solutions from one dual solution is generally impossible when at least one of the operators is nolonger paramonotone — see, e.g., Example 2.6 where one of the operators is even a normal cone operator.
Corollary 5.5
Suppose A and B are paramonotone. Then the following hold. (i)
Z and K are closed. (ii) gr K and gr Z are the “rectangles” Z × K and K × Z, respectively. (iii) Fix T = Z + K. (iv) ( Z − Z ) ⊥ ( K − K ) . (v) span ( K − K ) = X ⇒ Z is a singleton. ( Z − Z ) = X ⇒ K is a singleton.Proof . (i): Combine Theorem 5.3 and Proposition 2.4.(ii): Clear from Theorem 5.3.(iii): Combine (ii) with Theorem 4.5.(iv): Combine Corollary 2.13 with Theorem 5.3.(v): In view of (iv), we have that 0 = h Z − Z , K − K i = h Z − Z , span ( K − K ) i = h Z − Z , X i ⇒ Z − Z = { }⇔ Z is a singleton.(vi): This is verified analogously to the proof of (v). (cid:4) Corollary 5.6
Suppose that A and B are paramonotone. Then
Fix T = Z + K, Z = J A ( Z + K ) and K = J A − ( Z + K ) = ( Id − J A )( Z + K ) .Proof . Combine Corollary 5.5(ii) with Theorem 4.5. (cid:4) Remark 5.7 (paramonotonicity is critical)
Various results in this section — e.g., Theorem 5.3, Re-mark 5.4, Corollary 5.5(ii)–(vi)— fail if the assumption of paramonotonicity is omitted. To generate thesecounterexamples, assume that A and B are as in Example 2.5 or Example 2.6. The following two facts regarding projection operators will be used in the sequel.
Fact 6.1 (See, e.g., [8, Proposition 2.6].)
Let U and V be nonempty closed convex subsets of X such thatU ⊥ V . Then U + V is convex and closed, and P U + V = P U + P V . Fact 6.2
Let S be a nonempty subset of X , and let y ∈ X . Then ( ∀ x ∈ X ) P y + S ( x ) = y + P S ( x − y ) . Theorem 6.3
Suppose that A and B are paramonotone, that ( z , k ) ∈ Z × K, and that x ∈ X . Then thefollowing hold. (i) Z + K is convex and closed. (ii) P Z + K ( x ) = P Z ( x − k ) + P K ( x − z ) . (iii) If ( Z − Z ) ⊥ K, then P Z + K ( x ) = P Z ( x ) + P K ( x − z ) . (iv) If Z ⊥ ( K − K ) , then P Z + K ( x ) = P Z ( x − k ) + P K ( x ) . roof . (i): The convexity and closedness of Z and K follows from Corollary 3.7 and Corollary 5.5(i). ByCorollary 5.5(iv),(52) ( Z − z ) ⊥ ( K − k ) . Using Fact 6.1,(53) Z + K − z − k is convex and closed, and P Z + K − z − k = P Z − z + P K − k . Hence Z + K is convex and closed. (ii): Using (53), Fact 6.1, and Fact 6.2, we obtain P Z + K x = P ( z + k )+( Z + K − z − k ) x (54a) = z + k + P ( Z − z )+( K − k ) (cid:0) x − ( z + k ) (cid:1) (54b) = z + P Z − z (cid:0) ( x − k ) − z (cid:1) + k + P K − k (cid:0) ( x − z ) − k (cid:1) (54c) = P Z ( x − k ) + P K ( x − z ) . (54d)(iii): Using Fact 6.1 and Fact 6.2, we have P Z + K x = P z +( Z + K − z ) x (55a) = z + P ( Z − z )+ K ( x − z ) (55b) = z + P Z − z ( x − z ) + P K ( x − z ) (55c) = P Z x + P K ( x − z ) . (55d)(iv): Argue analogously to the proof of (iii). (cid:4) Remark 6.4
Suppose that A and B are paramonotone and that 0 ∈ K . Then Corollary 5.5(iv) implies that ( Z − Z ) ⊥ K − { } = K and we thus may employ either item (ii) (with k =
0) or item (iii) to obtain theformula for P Z + K .However, if ( Z − Z ) ⊥ K , then the next two examples show—in strikingly different ways since Z is eitherlarge or small—that we cannot conclude that 0 ∈ K : Example 6.5
Fix u ∈ X and suppose that ( ∀ x ∈ X ) Ax = u and B = − A . Then A and B are paramonotone, A + B ≡
0, and hence Z = X . Furthermore, K = { u } . Thus if u =
0, then K X = ( Z − Z ) . Example 6.6
Let U and V be closed convex subsets of X such that(56) 0 / ∈ U ∩ V and U − V = X . (For example, suppose that X = R and set U = V = [ , + ¥ [ .) Now assume that ( A , B ) = ( N U , N V ) ∗ . Inview of Example 2.7, K = U ∩ V and Z = N U − V ( ) = N X ( ) = { } . Hence Z is a singleton and thus Z − Z = { } ⊥ K while 0 / ∈ K . Theorem 6.7
Suppose that A and B are paramonotone, let k ∈ K, and let x ∈ X . Then the following hold. J A P Z + K ( x ) = P Z ( x − k ) . (ii) If ( Z − Z ) ⊥ K, then J A ◦ P Z + K = P Z .Proof . Take an arbitrary z ∈ Z . (i): Set z : = P Z ( x − k ) . Using Theorem 6.3(ii) and Theorem 5.3, we have(57) P Z + K x − z = P Z + K x − P Z ( x − k ) = P K ( x − z ) ∈ K = K z ⊆ Az . Hence P Z + K x ∈ ( Id + A ) z ⇔ z = J A P Z + K x ⇔ P Z ( x − k ) = J A P Z + K x .(ii): This time, let us set z : = P Z x . Using Theorem 6.3(iii) and Theorem 5.3, we have(58) P Z + K x − z = P Z + K x − P Z x = P K ( x − z ) ∈ K = K z ⊆ Az . Hence P Z + K x ∈ ( Id + A ) z ⇔ z = J A P Z + K x ⇔ P Z x = J A P Z + K x . (cid:4) Corollary 6.8
Suppose that A and B are paramonotone, and that ∈ K. Then (59) P Z = J A P Z + K . Specializing the previous result to normal cone operators, we recover the consistent case of [7, Corol-lary 3.9].
Example 6.9
Suppose that A = N U and B = N V , where U and V are closed convex subsets of X such that U ∩ V = ∅ . Then Z = U ∩ V , K = N U − V ( ) , and(60) P Z = P U P Z + K = P U P Fix T . Proof . This follows from Example 2.7, Corollary 5.5(iii), and Corollary 6.8. (cid:4)
In this section, we assume that(61) A = ¶ f and B = ¶ g , where f and g belong to G . We consider the primal problem (62) minimize x ∈ X f ( x ) + g ( x ) the associated Fenchel dual problem (63) minimize x ∗ ∈ X f ∗ ( x ∗ ) + g ∗ ( − x ∗ ) , primal and dual optimal values (64) m = inf ( f + g )( X ) and m ∗ = inf ( f ∗ + g ∗ ∨ )( X ) . Note that(65) m ≥ − m ∗ . Following [12] and [13], we say that total duality holds if m = − m ∗ ∈ R , the primal problem (62) has asolution, and the dual problem (63) has a solution. Theorem 7.1 (total duality)
Suppose that A = ¶ f and B = ¶ g, where f and g belong to G . Then Z = ∅ ⇔ total duality holds, in which case Z coincides with the set of solutions to the primal problem (62) .Proof . Observe that ( ¶ f ) − = ¶ f ∗ and that ( ¶ g ) − > = ( ¶ g ∗ ) > = ¶ ( g ∗ ∨ ) .“ ⇒ ”: Suppose that Z = ∅ , and let z ∈ Z . Then 0 ∈ ¶ f ( z ) + ¶ g ( z ) ⊆ ¶ ( f + g )( z ) . Hence z solves theprimal problem (62), and(66) m = f ( z ) + g ( z ) . Take k ∈ K = K z = ( ¶ f )( z ) ∩ ( − ¶ g )( z ) . First, we note that 0 ∈ ( ¶ f ) − ( k ) + ( ¶ g ) − > ( k ) = ¶ f ∗ ( k ) + ¶ g ∗ ∨ ( k ) ⊆ ¶ ( f ∗ + g ∗ ∨ )( k ) and so k solves the Fenchel dual problem (63). Thus,(67) m ∗ = f ∗ ( k ) + g ∗ ∨ ( k ) . Moreover, k ∈ ¶ f ( z ) and − k ∈ ¶ g ( z ) , i.e., f ( z ) + f ∗ ( k ) = h z , k i and g ( z ) + g ∗ ( − k ) = h z , − k i . Adding theseequations gives 0 = f ( z ) + f ∗ ( k ) + g ( z ) + g ∗ ∨ ( k ) = m + m ∗ . This verifies total duality.“ ⇐ ”: Suppose we have total duality. Then there exists x ∈ dom f ∩ dom g and x ∗ ∈ dom f ∗ ∩ dom g ∗ ∨ suchthat(68) f ( x ) + g ( x ) = m = − m ∗ = − f ∗ ( x ∗ ) − g ∗ ∨ ( x ∗ ) ∈ R . Hence 0 = ( f ( x ) + f ∗ ( x ∗ )) + ( g ( x ) + g ∗ ( − x ∗ )) ≥ h x , x ∗ i + h x , − x ∗ i =
0. Therefore, using convex analysisand Proposition 2.4,(69) (cid:0) x ∗ ∈ ¶ f ( x ) and − x ∗ ∈ ¶ g ( x ) (cid:1) ⇔ x ∗ ∈ K x ⇔ x ∈ Z x ∗ . Hence x ∈ Z .Note that Z = zer ( ¶ f + ¶ g ) ⊆ zer ¶ ( f + g ) since gr ( ¶ f + ¶ g ) ⊆ gr ¶ ( f + g ) . Hence Z is a subset of theset of primal solutions. Conversely, if x is a primal solution and x ∗ is a dual solution, then (68) holds and therest of the proof of “ ⇐ ” shows that x ∈ Z . Altogether, Z coincides with the set of primal solutions. (cid:4) Remark 7.2 (sufficient conditions)
On the one hand,(70) the primal problem has at least one solution18f dom f ∩ dom g = ∅ and one of the following holds (see, e.g., [5, Corollary 11.15]): (i) f is supercoercive;(ii) f is coercive and g is bounded below; (iii) 0 ∈ sri ( dom f ∗ + dom g ∗ ) (by, e.g., [5, Proposition 15.13] andsince (62) is the Fenchel dual problem of (63)). On the other hand,(71) the sum rule ¶ ( f + g ) = ¶ f + ¶ g holdswhenever one of the following is satisfied (see [5, Corollary 16.38]): (i) 0 ∈ sri ( dom f − dom g ) ; (ii) dom f ∩ int dom g = ∅ ; (iii) dom g = X ; (iv) X is finite-dimensional and ri dom f ∩ ri dom g = ∅ . If both (70) and(71) hold, then Z = ∅ and Z coincides with the set of primal solutions. In this last section, we sketch first algorithmic consequences and then conclude by commenting on theapplicability of our work to a more general duality framework.
Theorem 8.1 (abstract algorithm)
Suppose that A and B are paramonotone. Let ( x n ) n ∈ N be a sequencesuch that ( x n ) n ∈ N converges (weakly or strongly) to x ∈ Fix
T and ( J A x n ) n ∈ N converges (weakly or in norm)to J A x. Then the following hold. (i) ( ∀ k ∈ K ) J A x = P Z ( x − k ) . (ii) If ( Z − Z ) ⊥ K, then J A x = P Z x.Proof . Combine Corollary 5.5(iii) with Theorem 6.7. (cid:4) We provide three examples.
Example 8.2 (Douglas-Rachford algorithm)
Suppose that A and B are paramonotone and that the se-quence ( x n ) n ∈ N is generated by ( ∀ n ∈ N ) x n + = T x n . The hypothesis in Theorem 8.1 is satisfied, andthe convergence of the sequences is with respect to the weak topology [40]. See also [2] for a much simplerproof and [5, Theorem 25.6] for a powerful generalization. Example 8.3 (Halpern-type algorithm)
Suppose that A and B are paramonotone and that the sequence ( x n ) n ∈ N is generated by ( ∀ n ∈ N ) x n + = ( − l n ) T x n + l n y , where ( l n ) n ∈ N is a sequence of parametersin ] , [ and y ∈ X is given. Under suitable assumptions on ( l n ) n ∈ N , it is known (see, e.g., [25], [41])that x n → x : = P Fix T y with respect to the norm topology . Since J A is (firmly) nonexpansive, it is clearthat the hypothesis of Theorem 8.1 holds. Furthermore, J A x n → J A x = J A P Fix T y . Thus, if k ∈ K , then J A x n → P Z ( y − k ) by Theorem 6.7(i). And if ( Z − Z ) ⊥ K , then J A x n → P Z y by Theorem 6.7(ii). Example 8.4 (Haugazeau-type algorithm)
This is similar to Example 8.3 in that x n → x : = P Fix T y withrespect to the norm topology and where y ∈ X is given. For the precise description of the (somewhatcomplicated) update formula for ( x n ) n ∈ N , we refer the reader to [5, Section 29.2] or [4]; see also [26]. Onceagain, we have J A x n → J A x = J A P Fix T y and thus, if k ∈ K , then J A x n → P Z ( y − k ) by Theorem 6.7(i). And if ( Z − Z ) ⊥ K , then J A x n → P Z y by Theorem 6.7(ii). Consequently, in the context of Example 6.9, we obtain P U x n → P U ∩ V y ; in fact, this is [8, Theorem 3.3], which is the main result of [8].19urning to Eckstein-Ferris-Robinson duality, let us assume the following: • Y is a real Hilbert space (and possibly different from X ); • C is a maximally monotone operator on Y ; • L : X → Y is continuous and linear.Eckstein and Ferris [21] as well as Robinson [35] consider the problem of finding zeros of(72) A + L ∗ CL . This framework is more flexible than the Attouch-Th´era framework, which corresponds to the case when Y = X and L = Id. Note that just as Attouch-Th´era duality relates to classical Fenchel duality in the subdif-ferential case (see Section 7), the Eckstein-Ferris-Robinson duality pertains to classical
Fenchel-Rockafellarduality for the problem of minimizing f + h ◦ L when f ∈ G X and h ∈ G Y , and A = ¶ f and C = ¶ h .The results in the previous sections can be used in the Eckstein-Ferris-Robinson framework thanks toitems (ii) and (iii) of the following result, which allows us to set B = L ∗ CL . Proposition 8.5
The following hold. (i)
If C is paramonotone, then L ∗ CL is paramonotone. (ii) (Pennanen) If R ++ ( ran L − dom C ) is a closed subspace of Y , then L ∗ CL is maximally monotone. (iii)
If C is paramonotone and R ++ ( ran L − dom C ) is a closed subspace of Y , then L ∗ CL is maximallymonotone and paramonotone.Proof . (i): Take x and x in X , and suppose that x ∗ ∈ L ∗ CLx and x ∗ ∈ L ∗ CLx . Then there exist y ∗ ∈ CLx and y ∗ ∈ CLx such that x ∗ = L ∗ y ∗ and x ∗ = L ∗ y ∗ . Thus, h x − x , x ∗ − x ∗ i = h x − x , L ∗ y ∗ − L ∗ y ∗ i = h Lx − Lx , y ∗ − y ∗ i ≥ C is monotone. Hence L ∗ CL is monotone. Now suppose furthermore that h x − x , x ∗ − x ∗ i =
0. Then h Lx − Lx , y ∗ − y ∗ i = C yields y ∗ ∈ C ( Lx ) and y ∗ ∈ C ( Lx ) . Therefore, x ∗ = L ∗ y ∗ ∈ L ∗ CLx and x ∗ = L ∗ y ∗ ∈ L ∗ CLx .(ii): See [34, Corollary 4.4.(c)].(iii): Combine (i) and (ii). (cid:4) Acknowledgments
Part of this research benefited from discussions during a research visit of HHB related to a study leave at theTechnical University of Chemnitz. HHB thanks Dr. Gert Wanka, his optimization group, and the Facultyof Mathematics for their hospitality. HHB was partially supported by the Natural Sciences and Engineering20esearch Council of Canada and by the Canada Research Chair Program. RIB was partially supported bythe German Research Foundation. WLH was partially supported by the Natural Sciences and EngineeringResearch Council of Canada. WMM was partially supported by a University Graduate Fellowship of UBC.
References [1] H. Attouch and M. Th´era, A general duality principle for the sum of two operators,
Journal of ConvexAnalysis http://arxiv.org/abs/1103.0991 [3] H.H. Bauschke, J.M. Borwein, and X. Wang, Fitzpatrick functions and continuous linear monotoneoperators,
SIAM Journal on Optimization
18 (2007), 789–809.[4] H.H. Bauschke and P.L. Combettes, A weak-to-strong convergence principle for Fej´er monotonemethods in Hilbert spaces,
Mathematics of Operations Research
26 (2001), 248–264.[5] H.H. Bauschke and P.L. Combettes,
Convex Analysis and Monotone Operator Theory in HilbertSpaces , Springer, 2011.[6] H.H. Bauschke, P.L. Combettes, and D.R. Luke, Phase retrieval, error reduction algorithm, andFienup variants: a view from convex optimization,
Journal of the Optical Society of America A
Journal of Approximation Theory
127 (2004), 178–192.[8] H.H. Bauschke, P.L. Combettes, and D.R. Luke, A strongly convergent reflection method for findingthe projection onto the intersection of two closed convex sets in a Hilbert space,
Journal of Approxi-mation Theory
141 (2006), 63–69.[9] H.H. Bauschke, X. Wang, and L. Yao, Monotone linear relations: maximality and Fitzpatrick func-tions,
Journal of Convex Analysis
16 (2009), 673–686.[10] H.H. Bauschke, X. Wang, and L. Yao, Examples of discontinuous maximal monotone linear operatorsand the solution to a recent problem posed by B.F. Svaiter,
Journal of Mathematical Analysis andApplications
370 (2010), 224–241.[11] J.M. Borwein and J.D. Vanderwerff,
Convex Functions , Cambridge University Press, 2010.[12] R.I Bot¸, S.M. Grad, and G. Wanka, New regularity conditions for strong and total Fenchel-Lagrangeduality in infinite dimensional spaces,
Nonlinear Analysis: Theory, Methods & Applications
Journal of Mathematical Analysis and Applications
337 (2008), 1315–1325.2114] H. Br´ezis,
Operateurs Maximaux Monotones et Semi-Groupes de Contractions dans les Espaces deHilbert , North-Holland/Elsevier, 1973.[15] R.S. Burachik and A.N. Iusem, A generalized proximal point algorithm for the variational inequalityproblem in a Hilbert space,
SIAM Journal on Optimization
Set-Valued Mappings and Enlargements of Monotone Operators ,Springer-Verlag, 2008.[17] Y. Censor, A.N. Iusem, and S.A. Zenios, An interior point method with Bregman functions forthe variational inequality problem with paramonotone operators,
Mathematical Programming (Se-ries A)
81 (1998), 373–400.[18] P.L. Combettes, Solving monotone inclusions via compositions of nonexpansive averaged operators,
Optimization
53 (2004), 475–504.[19] J. Eckstein,
Splitting Methods for Monotone Operators with Applications to Parallel Optimization ,Ph.D. thesis, MIT, 1989.[20] J. Eckstein and D.P. Bertsekas, On the Douglas-Rachford splitting method and the proximal pointalgorithm for maximal monotone operators,
Mathematical Programming (Series A)
55 (1992), 293–318.[21] J. Eckstein and M.C. Ferris, Smooth methods of multipliers for complementarity problems,
Mathe-matical Programming (Series A)
86 (1999), 65–90.[22] J. Eckstein and B.F. Svaiter, A family of projective splitting methods for the sum of two maximalmonotone operators,
Mathematical Programming (Series B)
111 (2008), 173–199.[23] J. Eckstein and B.F. Svaiter, General projective splitting methods for sums of maximal monotoneoperators,
SIAM Journal on Control and Optimization
48 (2009), 787–811.[24] N. Hadjisavvas and S. Schaible, On a generalization of paramonotone maps and its application tosolving the Stampacchia variational inequality,
Optimization
55 (2006), 593–604.[25] B. Halpern, Fixed points of nonexpanding maps,
Bulletin of the AMS
73 (1967), 957–961.[26] Y. Haugazeau,
Sur les In´equations Variationnelles et la Minimisation de Fonctionnelles Convexes ,th`ese de doctorat, Universit´e de Paris, 1968.[27] A.N. Iusem, On some properties of paramonotone operators,
Journal of Convex Analysis
Proceedings of the London Mathematical Society (Series 3)
55 (1987), 605–624[29] J.-E. Mart´ınez-Legaz, Some generalizations of Rockafellar’s surjectivity theorem,
Pacific Journal ofOptimization
Duke Mathematical Journal
29 (1962),341–346.[31] A. Moudafi, On the stability of the parallel sum of maximal monotone operators,
Journal of Mathe-matical Analysis and Applications
199 (1996), 478–488.[32] N. Ogura and I. Yamada, Nonstrictly convex minimization over the bounded fixed point set of anonexpansive mapping,
Numerical Functional Analysis and Optimization
24 (2003), 129–135.[33] G.B. Passty, The parallel sum of nonlinear monotone operators,
Nonlinear Analysis
10 (1986), 215–227.[34] T. Pennanen, Dualization of generalized equations of maximal monotone type,
SIAM Journal onOptimization
10 (2000), 809–835.[35] S.M. Robinson, Composition duality and maximal monotonicity,
Mathematical Programming (Se-ries A)
85 (1999), 1–13.[36] R.T. Rockafellar,
Convex Analysis , Princeton University Press, Princeton, 1970.[37] R.T. Rockafellar and R. J-B Wets,
Variational Analysis , Springer-Verlag, corrected 3rd printing, 2009.[38] S. Simons,
Minimax and Monotonicity , Springer-Verlag, 1998.[39] S. Simons,
From Hahn-Banach to Monotonicity , Springer-Verlag, 2008.[40] B.F. Svaiter, On weak convergence of the Douglas-Rachford method,
SIAM Journal on Control andOptimization
49 (2011), 280–287.[41] R. Wittmann, Approximation of fixed points of nonexpansive mappings,
Archiv der Mathematik
Convex Analysis in General Vector Spaces , World Scientific Publishing, 2002.[43] C. Z˘alinescu, A new convexity property for monotone operators,
Journal of Convex Analysis
Nonlinear Functional Analysis and Its Applications II/A: Linear Monotone Operators ,Springer-Verlag, 1990.[45] E. Zeidler,
Nonlinear Functional Analysis and Its Applications II/B: Nonlinear Monotone Operators ,Springer-Verlag, 1990.[46] E. Zeidler,