Attouch-Théra Duality, Generalized Cycles and Gap Vectors
aa r X i v : . [ m a t h . O C ] J a n T HE A TTOUCH -T H ´ ERA D UALITY , G
ENERALIZED C Y CLESAND G AP V ECTORS
Salihah Alwadani * , Heinz H. Bauschke † , and Xianfu Wang ‡ January 13, 2021
Abstract
Using the Attouch-Th´era duality, we study the cycles, gap vectors and fixed point sets of com-positions of proximal mappings. Sufficient conditions are given for the existence of cycles andgap vectors. A primal-dual framework provides an exact relationship between the cycles andgap vectors. We also introduce the generalized cycle and gap vectors to tackle the case whenthe classical ones do not exist. Examples are given to illustrate our results.
Primary 47H05, 49J53, 47H09; Secondary 47N10,49N15, 47A06, 47J25, 90C25.
Keywords:
Attouch-Th´era duality, circular right shift operator, convex function, displacementmapping, generalized cycle, generalized gap vector, proximal cycle, proximal mapping.
Throughout this paper, we assume that X is a real Hilbert space with inner product h· , ·i : X × X → R and induced norm k · k = p h· , ·i . The set of proper lower semicontinuous con-vex functions from X to ] − ∞ , + ∞ ] is denoted by Γ ( X ) . In the product space X = X m with m ∈ N = {
1, 2, . . . } , we let ∆ = (cid:8) ( x , . . . , x ) (cid:12)(cid:12) x ∈ X (cid:9) , R : X → X : ( x , x , . . . , x m ) ( x m , x , . . . , x m − ) , and k x k = q h x , x i = q k x k + · · · + k x m k * Mathematics, University of British Columbia, Kelowna, B.C. V1V 1V7, Canada. E-mail: [email protected] . † Mathematics, University of British Columbia, Kelowna, B.C. V1V 1V7, Canada. E-mail: [email protected] . ‡ Mathematics, University of British Columbia, Kelowna, B.C. V1V 1V7, Canada. E-mail: [email protected] . x = ( x , x , . . . , x m ) . For a finite family of functions ( f i ) mi = in Γ ( X ) , define its separablesum by f = f ⊕ · · · ⊕ f m : X → ] − ∞ , + ∞ ] : ( x , . . . , x m ) m ∑ i = f i ( x i ) . (1)The proximal mapping of f i is defined by Prox f i = ( Id + ∂ f i ) − where ∂ f i denotes the subdifferen-tial of f i . A classical cycle of f is a vector z = ( z , . . . , z m ) ∈ X such that z = Prox f z m , z = Prox f z , z = Prox f z , · · · , (2a) z m − = Prox f m − z m − , z m = Prox f m z m − . (2b)The set of all classical cycles of f will be denoted by Z . Since ∂ f = ∂ f × · · · × ∂ f m and Prox f =( Prox f , · · · , Prox f m ) , in the frame work of product space X , with z = ( z , . . . , z m ) , the operatorform of (2a)–(2b) is z = Prox f Rz , equivalently, (3)in terms of monotone operators 0 ∈ ∂ f ( z ) + z − Rz . (4)Here the displacement mapping Id − R is maximally monotone but not a gradient of convex func-tion unless m =
2. See [14, Section VI-B] and [15, Section 4] for proximal cycles from the perspec-tive of Nash equilibrium.Many authors have studied cycles of compositions of proximal mappings or resolvents; see,e.g., [2, 4, 5, 7, 9, 14, 15, 10, 21]. For the compositions of two proximal mappings or resolvents, theinvestigation has matured; see [8, 21]. For the compositions of three or more proximal mappingsor resolvents, many questions await to explore. On the one hand, a systematic study of cycles andgap vectors for compositions of proximal mappings does not exist in the literature; on the otherhand, it is not clear what one should do when the cycles and gap vectors do not exist. In [2], wecarried out the study of extended gap vectors for projection mappings and provided an answerto the geometry conjecture, which concerns the situation where the classical gap vector does notnecessarily exist.
The goal of this paper is to give a systematic study of the cycles, gap vectors and fixed points of compo-sitions of proximal mappings, and to provide generalized cycles and gap vectors when the classical ones donot necessarily exist. Our investigation relies to a large extent on the Attouch-Th´era duality and convexanalysis.
The remainder of the paper is organized as follows. Section 2 collects some preliminary resultsand facts on convex functions and monotone operators needed in the sequel. In Section 3 weprovide a systematic study of classical cycles, gap vector, and fixed point sets of cyclic proximalmappings. It turns out that the fixed point sets of a cyclic composition of proximal mappings is justa shift of the other via gap vectors. In Section 4, using the Attouch-Th´era duality we investigatethe generalized cycle and gap vectors, and show that they can recover the classical ones wheneverthe latter exist. Section 5 is devoted to conditions under which f (cid:3) ι ∆ is lower semicontinuous, inturn, these guarantee that the classical cycles and gap vectors exist. Section 6 gives two examples2o illustrate our results on the generalized cycle and gap vector. Finally, Section 7 shows how touse the forward-backward iteration scheme to compute the generalized cycle and gap vectors. Notation . We follow the notation of [7], where one will find a detailed account of the followingnotions. Let f , g : X → ] − ∞ , + ∞ ] . The Fenchel conjugate of f is f ∗ : X → [ − ∞ , + ∞ ] : x ∗ sup x ∈ X ( h x , x ∗ i − f ( x )) .The infimal convolution of f , g is f (cid:3) g : X → [ − ∞ , + ∞ ] : x inf y ∈ X ( f ( y ) + g ( x − y )) , and it isexact at a point x ∈ X if ( ∃ y ∈ X ) ( f (cid:3) g )( x ) = f ( y ) + g ( x − y ) ; f (cid:3) g is exact if it is exact at everypoint of its domain. The subdifferential of f is the set-valued operator ∂ f : X ⇒ X : x (cid:8) x ∗ ∈ X (cid:12)(cid:12) ( ∀ y ∈ X ) f ( y ) ≥ f ( x ) + h x ∗ , y − x i (cid:9) .We use cl f for the lower semicontinuous hull of f . For a set C ⊂ X , its indicator function isdefined by ι C ( x ) = (
0, if x ∈ C , + ∞ , if x C .The relative interior, interior, and closure of C will be denoted by ri C , int C , and C respectively.When the set C is nonempty closed convex, we write P C = Prox ι C for the projection operator and N C = ∂ι C for the normal cone. An operator N : X → X is nonexpansive if ( ∀ x , y ∈ X ) k Nx − Ny k ≤ k x − y k ; firmly nonexpansive if 2 N − Id is nonexpansive; β -cocercive if β N is firmlynonexpansive for some β ∈ R ++ . Prime examples of firmly nonexpansive mappings are proximalmappings of convex functions. As usual, Fix N = (cid:8) x ∈ X (cid:12)(cid:12) Nx = x (cid:9) denotes the set of fixedpoints of N . For a monotone operator A : X ⇒ X , the sets dom A = (cid:8) x ∈ X (cid:12)(cid:12) Ax = ∅ (cid:9) , ran A = (cid:8) u ∈ X (cid:12)(cid:12) ( ∃ x ∈ X ) u ∈ Ax (cid:9) , ker A = zer ( A ) = A − ( ) are the domain, range, and kernel orzeros of A respectively. It will be convenient to write e A = ( − Id ) ◦ A − ◦ ( − Id ) . Let ( x n ) n ∈ N bea sequence in X . We will write x n → x if ( x n ) n ∈ N converges strongly to x , and x n ⇀ x if ( x n ) n ∈ N converges weakly to x . For valuable references on convex functions and monotone operators, see,e.g., [17, 18, 22, 19]. Blanket assumptions . Throughout the paper, we shall assume that(i) ( f i ) mi = are in Γ ( X ) , and f is given by (1).(ii) dom ( f ∗ + ι ∗ ∆ ) = dom ( f ∗ + ι ∆ ⊥ ) = ∅ , equivalently, dom f ∗ ∩ ∆ ⊥ = ∅ . (5)This will assure that f (cid:3) ι ∆ is proper convex, and possess a continuous affine minorant, see,e.g., [7, Proposition 13.12(ii)]. In this section we give some conditions under which dom f ∗ ∩ ∆ ⊥ = ∅ , subdifferential proper-ties concerning the lower semicontinuous hulls and infimal convolutions, and the Attouch-Th´eraduality for pairs of monotone operators. 3 .1 Properties of f ∗ + ι ∗ ∆ , f (cid:3) ι ∆ , and cl ( f (cid:3) ι ∆ ) Because f is separable, we have f ∗ = f ∗ ⊕ · · · ⊕ f ∗ m . Recall that a function g ∈ Γ ( X ) is coercive iflim k x k→ ∞ g ( x ) = + ∞ , and supercoercive if lim k x k→ ∞ ( g ( x ) / k x k ) = + ∞ . Lemma 2.1.
Suppose that one of the following hold: (i)
There exists x ∗ = ( x ∗ , . . . , x ∗ m ) ∈ ∆ ⊥ and µ , . . . , µ m ∈ R such that ( ∀ ≤ i ≤ m )( ∀ x i ∈ X ) f i ( x i ) ≥ h x ∗ i , x i i − µ i . (ii) One of the functions f i is supercoercive, e.g., having a bounded domain. (iii) inf f i > − ∞ for every i =
1, . . . , m.Then (5) holds.Proof . Because ( f (cid:3) ι ∆ ) ∗ = f ∗ + ι ∗ ∆ , (5) holds if and only if f (cid:3) ι ∆ possesses a continuous minorant.Suppose (i) holds. Then ( ∀ ≤ i ≤ m )( ∀ x i ∈ X ) f i ( x i − d ) ≥ h x ∗ i , x i − d i − µ i = h x ∗ i , x i i − h x ∗ i , d i − µ i .Summing up these m -inequalities gives m ∑ i = f i ( x i − d ) ≥ m ∑ i = h x ∗ i , x i i − m ∑ i = µ i = h x ∗ , x i − m ∑ i = µ i .Taking infimum over d ∈ X gives ( f (cid:3) ι ∆ )( x ) ≥ h x ∗ , x i − ∑ mi = µ i , as required.Suppose that (ii) holds, say f m being supercoercive. For each f i with 1 ≤ i ≤ m −
1, by Bronsted-Rockafellar’s theorem [7, Theorem 16.58] we can choose an affine functional x ∗ i ∈ X , µ i ∈ R suchthat ( ∀ x i ∈ X ) f i ( x i ) ≥ h x ∗ i , x i i − µ i .Put x ∗ m = − ( x ∗ + · · · + x ∗ m − ) . Since f m is supercoercive, the function f m − h x ∗ m , ·i is coercive, so itattains its minimum at − µ m ∈ R . Then f m ≥ h x ∗ m , ·i − µ m . Hence (i) applies.(iii) is a special case of (i) with x ∗ i = i =
1, . . . , m . (cid:4) Some elementary properties of f (cid:3) ι ∆ and its lower semicontinuous hull cl ( f (cid:3) ι ∆ ) come next. Lemma 2.2.
We have ran ∂ [ cl ( f (cid:3) ι ∆ )] ⊆ ∆ ⊥ . Proof . Let x ∗ ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) . Then ( ∀ z ∈ X ) [ cl ( f (cid:3) ι ∆ )]( z ) ≥ [ cl ( f (cid:3) ι ∆ )]( x ) + h x ∗ , z − x i . Let z = u + v with u ∈ dom f and v ∈ ∆ . We have f ( u ) = f ( u ) + ι ∆ ( v ) ≥ [ cl ( f (cid:3) ι ∆ )]( x ) + h x ∗ , u − x i + h x ∗ , v i .Since ∆ is a subspace and f ( u ) < + ∞ , we obtain h x ∗ , v i = (cid:4) emma 2.3. For every d ∈ ∆ and x ∈ X , we have (i) ( f (cid:3) ι ∆ )( x + d ) = ( f (cid:3) ι ∆ )( x ) , and [ cl ( f (cid:3) ι ∆ )]( x + d ) = [ cl ( f (cid:3) ι ∆ )]( x ) .(ii) ∂ ( f (cid:3) ι ∆ )( x + d ) = ∂ ( f (cid:3) ι ∆ )( x ) , and ∂ [ cl ( f (cid:3) ι ∆ )]( x + d ) = ∂ [ cl ( f (cid:3) ι ∆ )]( x ) . Proof . (i): Since ∆ is a subspace, we have d − ∆ = ∆ and − ∆ = ∆ . Then ( f (cid:3) ι ∆ )( x + d ) = inf v ∈ ∆ ( f ( x + d − v ) + ι ∆ ( v )) (6) = inf v ∈ d − ∆ f ( x + v ) = inf v ∈ ∆ f ( x + v ) = inf − v ∈ ∆ ( f ( x − ( − v )) = ( f (cid:3) ι ∆ )( x ) . (7)(ii): Apply (i) and the subdifferential definition. (cid:4) Lemma 2.4.
Let f : X → ] − ∞ , + ∞ ] be proper and convex, and x ∈ X. Then the following hold: (i) If ∂ f ( x ) = ∅ , then f is lower semicontinuous at x. (ii) If f ( x ) = ( cl f )( x ) , that is, f is lower semicontinuous at x, then ∂ f ( x ) = ∂ ( cl f )( x ) . (iii) In general, ∂ f ⊆ ∂ ( cl f ) .Proof . (i): See [22, Theorem 2.4.1(ii)].(ii): Let x ∗ ∈ ∂ ( cl f )( x ) . Then ( ∀ y ∈ X ) ( cl f )( y ) ≥ ( cl f )( x ) + h x ∗ , y − x i . Because f ≥ cl f and f ( x ) = ( cl f )( x ) , we have ( ∀ y ∈ X ) f ( y ) ≥ f ( x ) + h x ∗ , y − x i , so x ∗ ∈ ∂ f ( x ) .Let x ∗ ∈ ∂ f ( x ) . Then ( ∀ y ∈ X ) f ( y ) ≥ f ( x ) + h x ∗ , y − x i . Taking the lower semicontinuousenvelopes with respect to y both sides, we have ( ∀ y ∈ X ) ( cl f )( y ) ≥ f ( x ) + h x ∗ , y − x i . Since f ( x ) = ( cl f )( x ) , we obtain x ∗ ∈ ∂ cl f ( x ) .(iii): Let y ∈ X . If ∂ f ( y ) = ∅ , then ∂ f ( y ) ⊆ ∂ ( cl f )( y ) . If ∂ f ( y ) = ∅ , then f is lower semicontin-uous at y by (i). Apply (ii). (cid:4) The inclusion in Lemma 2.4 (iii) can be proper, e.g., ∂ι C ( x ) = ∅ while ∂ι C ( x ) = ∅ for x ∈ C \ C when C is convex and int C = ∅ . Lemma 2.5.
Let f , g ∈ Γ ( X ) and x , y ∈ X. Then the following hold: (i) If ( f (cid:3) g )( x ) = f ( y ) + g ( x − y ) , then ∂ ( f (cid:3) g )( x ) = ∂ f ( y ) ∩ ∂ g ( x − y ) .(ii) If ∂ f ( y ) ∩ ∂ g ( x − y ) = ∅ , then ( f (cid:3) g )( x ) = f ( y ) + g ( x − y ) and ∂ ( f (cid:3) g )( x ) = ∂ f ( y ) ∩ ∂ g ( x − y ) .(iii) In general, ∂ ( f (cid:3) g )( x ) ⊇ ∂ f ( y ) ∩ ∂ g ( x − y ) .5 roof . (i): See [7, Propostition 16.61(i)].(ii): When ∂ f ( y ) ∩ ∂ g ( x − y ) = ∅ , [7, Proposition 16.61(ii)] gives ( f (cid:3) g )( x ) = f ( y ) + g ( x − y ) .Apply (i) to obtain ∂ ( f (cid:3) g )( x ) = ∂ f ( y ) ∩ ∂ g ( x − y ) .(iii): If ∂ f ( y ) ∩ ∂ g ( x − y ) = ∅ , it is clear. If ∂ f ( y ) ∩ ∂ g ( x − y ) = ∅ , apply (ii). (cid:4) The key tool we shall use is the following Attouch-Th´era duality.
Fact 2.6 (Attouch-Th´era duality [3]) . Let A , B : X ⇒ X be maximally monotone operators. Let S be thesolution set of the primal problem: find x ∈ X such that ∈ Ax + Bx . (8) Let S ∗ be the solution set of the dual problem associated with the ordered pair ( A , B ) :find x ∗ ∈ X such that ∈ A − x ∗ + e B ( x ∗ ) . (9) Then (i) S = (cid:8) x ∈ X (cid:12)(cid:12) ( ∃ x ∗ ∈ S ∗ ) x ∗ ∈ Ax and − x ∗ ∈ Bx (cid:9) . (ii) S ∗ = (cid:8) x ∗ ∈ X (cid:12)(cid:12) ( ∃ x ∈ S ) x ∈ A − x ∗ and − x ∈ e B ( x ∗ ) (cid:9) .Important properties of the circular right shift operator come as follows. Fact 2.7.
For the circular right shift operator R , the following hold: (i) Id − R is maximally monotone. (ii) ( Id − R ) − = Id + N ∆ ⊥ + T where T : X → X is a skew operator defined byT = m m − ∑ k = ( m − k ) R k . In particular, dom ( Id − R ) − = ∆ ⊥ . (iii) ( Id + T ) − = Id − R + P ∆ .Proof . (i): See [11, Theorem 6.1] or [18, Exercise 12.16]. (ii): See [2, Section 2.2] or [1]. (iii): See [2,Theorem 2.4]. (cid:4) Fact 2.8.
Let N : X → X be nonexpansive. Then ( Id − N ) − is -strongly monotone; equivalently, Id − N is -cocoercive.Proof . See [7, Proposition 4.11 and Example 22.7]. (cid:4) Classical cycles, gap vectors and fixed point sets of cyclic proximalmappings
In this section we give a comprehensive study on the relationship among the classical cycles, gapvectors and fixed point sets for compositions of proximal mappings.Define F i : = Fix (cid:0)
Prox f i · · · Prox f Prox f m · · · Prox f i + (cid:1) , and (10) Q i : X → X : z z i .Some basic properties and relationship among the F i ’s, and Z are given below. Lemma 3.1.
For every ≤ i , j ≤ m, the following hold: (i) T mi = argmin f i ⊆ T mi = F i . In particular, if there exists F i = ∅ , then T mi = argmin f i = ∅ . (ii) F i is closed and convex, and F i ⊆ ran ( Prox f i ) . (iii) F i = ∅ if and only if F j = ∅ if and only if Z is nonempty. (iv) Z is closed and convex, and Z ⊆ F × · · · × F m . (v) The mapping Q i : Z → F i is bijective, in particular, Q i ( Z ) = F i .(vi) For ≤ i ≤ m − , Prox f i + ( F i ) = F i + , and Prox f ( F m ) = F . (11) Consequently,
Prox f R ( F × · · · F m ) = F × · · · × F m .(vii) When m = , we have Prox f ( F ) = F , Prox f ( F ) = F . Proof . (i): This follows from argmin f i = Fix Prox f i for 1 ≤ i ≤ m and T mi = Fix Prox f i ⊆ F j for1 ≤ j ≤ m .(ii): Since each Prox f i is firmly nonexpansive, the mapping Prox f i · · · Prox f Prox f m · · · Prox f i + is nonexpansive. By [7, Corollary 4.24], F i is closed and convex. The relation F i ⊆ ran ( Prox f i ) isfrom the definition of F i .(iii): This is clear from the definition of F i , F j , Z .(iv): Since Z = Fix ( Prox f R ) by (3) and Prox f R is nonexpansive, it suffices to apply [7, Corollary4.24]. The relation Z ⊆ F × · · · × F m is from the definition of Z and F i ’s.(v): It suffices to show Q i is injective. Suppose z = ( z , . . . , z m ) , ˜ z = ( ˜ z , . . . , ˜ z m ) ∈ Z and Q i ( z ) = Q i ( ˜ z ) , i.e., z i = ˜ z i . Because z , ˜ z ∈ Z , they are cycles, so we have z i + = Prox f i + z i = Prox f i + ˜ z i = ˜ z i + , . . ., z m = Prox f m z m − = Prox f m ˜ z m − = ˜ z m , z = Prox f z m = Prox f ˜ z m = ˜ z , z = Prox f z = Prox f ˜ z = ˜ z , . . ., z i − = Prox f i − z i − = Prox f ˜ z i − = ˜ z i − . Hence z = ˜ z .(vi): Set f m + = f and F m + = F . We show F i + = Prox f i + ( F i ) for 1 ≤ i ≤ m . To this end, let z ∈ F i . Then z = Prox f i · · · Prox f Prox f m · · · Prox f i + z , soProx f i + z = Prox f i + Prox f i · · · Prox f Prox f m · · · Prox f i + ( Prox f i + z ) f i + z ∈ F i + . Thus, Prox f i + ( F i ) ⊆ F i + . To show the converse, let z ∈ F i + . Then z = Prox f i + · · · Prox f Prox f m · · · Prox f i + z . (12)Define z i + = Prox f i + z , z i + = Prox f i + z i + , . . . , z m = Prox f m z m − , z = Prox f z m , z = Prox f z , . . . , z i = Prox f i z i − .Then (12) implies z = Prox f i + z i , so that z = ( z , . . . , z i , z , z i + , . . . , z m ) ∈ Z , z i ∈ F i and z = Prox f i + z i . Thus, F i + ⊆ Prox f i + ( F i ) . Hence F i + = Prox f i + ( F i ) . Altogether, (11) is established.(vii): Apply (vi) with m = (cid:4) Lemma 3.2.
Suppose that S = T mi = argmin f i = ∅ . Then (i) F i = S for every ≤ i ≤ m. (ii) Z = (cid:8) ( z , . . . , z ) (cid:12)(cid:12) z ∈ S (cid:9) . Proof . Because Prox f i is firmly nonexpansive, Fix ( Prox f i ) = argmin f i , and S = ∅ , by [7, Corollary4.51] for every i we have Fix ( Prox f i · · · Prox f Prox f m · · · Prox f i + ) = S . (13)(i): This is (13). (ii): For every ( z , . . . , z m ) ∈ Z , (2a) gives z ∈ F . Since F = S by (i), (2a) furtherimplies z = z = · · · = z m . (cid:4) Lemma 3.2 shows that it is natural to assume S = T mi = argmin f i = ∅ when studying classicalcycles and gap vectors. Using the Attouch-Th´era duality with A = ∂ f and B = Id − R , and the identity − Id ◦ ( Id − R ) − ◦ ( − Id ) = ( Id − R ) − for the linear relation ( Id − R ) − , we can formulate the primal-dual inclusion problem: ( P ) ∈ ∂ f ( x ) + ( Id − R ) x , (14) ( D ) ∈ ( ∂ f ) − ( y ) + ( Id − R ) − y . (15)While the primal problem ( P ) solves for the classical cycles, the dual ( D ) for the pair ( ∂ f , Id − R ) solves for the classical gap-vectors of f . ( P ) has a solution (respectively, no solution) if and only if ( D ) has a solution (respectively, no solution). Proposition 3.3 (gap vector) . The solution set of ( D ) is at most a singleton (possibly empty). roof . Since ( Id − R ) − = Id + N ∆ ⊥ + T by Fact 2.7(ii) (or use Fact 2.8), the monotone operator ( ∂ f ) − + ( Id − R ) − =
12 Id +( N ∆ ⊥ + T + ( ∂ f ) − ) is strongly monotone, so [( ∂ f ) − + ( Id − R ) − ] − ( ) is at most a singleton. (cid:4) Proposition 3.4 (cycle and gap vectors) . Consider the sets of classical cycle and gap vectors definedrespectively by Z = (cid:8) x ∈ X (cid:12)(cid:12) ∈ ∂ f ( x ) + ( Id − R ) x (cid:9) , (16) G = (cid:8) y ∈ X (cid:12)(cid:12) ∈ ( ∂ f ) − ( y ) + ( Id − R ) − y (cid:9) . (17) We have (i) Z = S y ∈ G ( Id − R ) − ( − y ) ∩ ( ∂ f ) − ( y ) . (ii) G = S (cid:8) Rx − x (cid:12)(cid:12) x ∈ Z (cid:9) . If G = ∅ , then G is a singleton y ∈ ∆ ⊥ and y = Rx − x for every x ∈ Z .Proof . Apply Facts 2.6, 2.7(ii) and Proposition 3.3. (cid:4) F i ’s In view of Lemma 3.1, Propositions 3.4 and 3.3, we have that F i = ∅ if and only if Z = ∅ if andonly if G = ∅ , and that G is either empty or a singleton. Therefore, below it is not surprising thatwe assume that the dual ( D ) has a solution. Corollary 3.5 (fixed point sets of cyclic proximal mapping) . Assume that y = ( y , . . . , y m ) is theunique solution of the dual problem ( D ) . Then the following hold: (i) For ≤ i ≤ m − , the mapping Prox f i + : F i → F i + is bijective and it is given by z z − y i + . The mapping
Prox f : F m → F is bijective and it is given by z z − y .(ii) For ≤ i ≤ m − , the fixed point sets F i + = F i − y i + , and F = F m − y . Consequently, F i isjust a translation of F .Proof . By the assumption, ( D ) has solution, hence ( P ) has a solution by the Attouch-Th´era duality.In view of Lemma 3.1, each fixed point set F i is nonempty.(i): By Lemma 3.1(vi), Prox f i + is onto. Suppose z , ˜ z ∈ F i and Prox f i + z = Prox f i + ˜ z . Because z = Prox f i · · · Prox f Prox f m · · · Prox f i + z ,˜ z = Prox f i · · · Prox f Prox f m · · · Prox f i + ˜ z ,we have z = ˜ z . Hence Prox f i + is injective on F i . 9ow for every z ∈ F i , we have z = Prox f i · · · Prox f Prox f m · · · Prox f i + z . (18)Set z i + = Prox f i + z , z i + = Prox f i + z i + , . . . , z m = Prox f m z m − , z = Prox f z m , . . . , z i − = Prox f i − z i − .Then z = Prox f i z i − by (18), and z = ( z , . . . , z i − , z , z i + , . . . , z m ) is a solution to ( P ) . By Proposition 3.4(ii), ( z m − z , z − z , . . . , z − z i + , . . . , z m − − z m ) = Rz − z = y .Thus, z − z i + = y i + so that z i + = z − y i + , i.e., Prox f i + z = z − y i + .The proof for Prox f is analogous.(ii): This immediate from (i). (cid:4) Remark 3.6.
Corollary 3.5 extends [10, Proposition 3.2(v)] and [21, Theorem 3.3(v)] from compositions oftwo resolvents to compositions of multi-proximal mappings.
Corollary 3.7.
Suppose that one of the fixed point set F i s is nonempty and bounded, e.g., when one of dom f i s is bounded. Then the following are equivalent: (i) F = · · · = F m = ∅ . (ii) T mi = argmin f i = ∅ .Under either of the two conditions, we haveF = · · · = F m = m \ i = argmin f i = m \ i = Fix ( Prox f i ) = ∅ . (19) Proof . (i) ⇒ (ii): By Lemma 3.1(iii) and Proposition 3.4, the dual problem ( D ) has a unique solution,say y . Because that F i = F i + = F i − y i + by Corollary 3.5(ii), and that F i is bounded, closed andconvex, Radstrom’s Cancellation Theorem [7, page 68] implies y i + = i =
1, . . . , m , so y = x being the solution to (4), Rx − x =
0, thus, x = · · · = x m . By(4), (
0, . . . , 0 ) ∈ ∂ f ( x ) × · · · × ∂ f m ( x ) , so x ∈ T mi = argmin f i . This establishes (ii).(ii) ⇒ (i): Apply Lemma 3.2(i).Equation (19) follows from Lemma 3.2(i) and argmin f i = Fix Prox f i . (cid:4) Corollary 3.8.
Suppose that argmin f i = ∅ for ≤ i ≤ m. Then the following are equivalent: (i) F = · · · = F m = ∅ . T mi = argmin f i = ∅ .Under either of the two conditions, we have F = · · · = F m = T mi = argmin f i = T mi = Fix ( Prox f i ) = ∅ . Proof . (i) ⇒ (ii): By Lemma 3.1(iii) and Proposition 3.4, the dual problem ( D ) has a unique solution,say y . The mapping Prox f i + : F i → F i + is given by z → z − y i + by Corollary 3.5. Using theassumption F i = F i + , we have ( ∀ n ∈ N ) Prox nf i + ( z ) = z − ny i + ∈ F i . (20)Take x ∈ argmin f i + = Fix Prox f i + , which is possible by the assumption. As Prox f i + is nonex-pansive, k Prox nf i + ( z ) − x k = k Prox nf i + ( z ) − Prox nf i + ( x ) k ≤ k z − x k ,so ( Prox nf i + ( z )) n ∈ N is bounded, and this implies y i + = f i for 1 ≤ i ≤ m , we obtain y =
0. In view of Proposition 3.4(ii) with x being thesolution to (4), Rx − x =
0, thus, x = · · · = x m . By (4), (
0, . . . , 0 ) ∈ ∂ f ( x ) × · · · × ∂ f m ( x ) , so x ∈ T mi = argmin f i . This establishes (ii).(ii) ⇒ (i): This and the remaining conclusion follow from Lemma 3.2(i). (cid:4) Remark 3.9.
In [20], Suzuki called m \ i = Fix ( Prox f i ) = F = · · · = F m (21) as Bauschke’s condition for the collection of proximal mappings ( Prox f i ) mi = . (i) When T mi = Fix ( Prox f i ) = ∅ , it is well-known that (21) holds ; cf. [7, Corollary 4.51]. (ii) When T mi = Fix ( Prox f i ) = ∅ , we have two cases to consider. Case 1: Z = ∅ . Lemma 3.1 yields F = · · · = F m = ∅ , hence (21) holds because of T mi = Fix ( Prox f i ) = ∅ . See, e.g., Example 6.1. Case 2: Z = ∅ . Lemma 3.1 yields F i = ∅ for ≤ i ≤ m, hence (21) fails because of T mi = Fix ( Prox f i ) = ∅ .Requiring F = · · · = F m implies that F = · · · = F m has to be unbounded by Corollary 3.7. If each argmin f i = ∅ , then F i = F j for some ≤ i , j ≤ m by Corollary 3.8. The question is what should one do if the classical cycle and gap vectors do not exist.
In this section, using the Attouch-Th´era duality we provide generalized cycles and gap vectors forcl ( f (cid:3) ι ∆ ) . One remarkable feature is that the generalized cycles and gap vectors always exist, andthey can recover the classical ones for f whenever the latter exist.To deal with the absence of solutions (or non-attainments of the classical cycle or gap vectors),we need to enlarge the primal ( P ) or dual ( D ) so that it has a solution.11 .1 Extending the dual approach Since the linear relation ( Id − R ) − = Id + N ∆ ⊥ + T by Fact 2.7(ii), and ∂ι ∗ ∆ = ∂ι ∆ ⊥ = N ∆ ⊥ , wehave ( ∂ f ) − + ( Id − R ) − = ∂ f ∗ +
12 Id + T + ∂ι ∗ ∆ = ∂ f ∗ + ∂ι ∗ ∆ +
12 Id + T (22) ⊆ ∂ ( f ∗ + ι ∗ ∆ ) +
12 Id + T (23) = (cid:2) Id + (cid:0) T + ∂ ( f ∗ + ι ∗ ∆ ) (cid:1)(cid:3) . (24)The enlarged dual ( ˜ D ) ∈ ∂ ( f ∗ + ι ∗ ∆ )( y ) + y + T y (25)always has a unique solution. We call the unique y given by (25) as the generalized gap vector ofcl ( f (cid:3) ι ∆ ) . Proposition 4.1 (existence and uniqueness of generalized gap vector) . The following hold: (i)
The enlarged dual ( ˜ D ) always has a unique solution. (ii) If ( D ) has a solution, then it is exactly the solution of ( ˜ D ) .Proof . (i): Because ∂ ( f ∗ + ι ∗ ∆ ) and T are maximally monotone, we have 2 T + ∂ ( f ∗ + ι ∗ ∆ ) maximallymonotone by [7, Corollary 25.5] or [19, Corollary 32.3], so Minty’s theorem [7, Theorem 21.1]applies.(ii): This follows from (22)–(24) and (i). (cid:4) One can also start from the primal ( P ) ∈ ∂ f ( u ) + ( Id − R ) u . Because ∂ f + ( Id − R ) is alreadymaximally monotone by [19], one cannot find enlargements of ( P ) that always have a solutions.We need to rewrite it in an equivalent form then do enlargements. In view of − ( Id − R ) u ∈ ∂ f ( u ) , − ( Id − R ) u ∈ ∆ ⊥ , Lemmas 2.5 and 2.4, we have − ( Id − R ) u ∈ ∂ f ( u ) ∩ ∆ ⊥ = ∂ f ( u ) ∩ ∂ι ∆ ( d ) ⊆ ∂ ( f (cid:3) ι ∆ )( u + d ) (26) ⊆ ∂ [ cl ( f (cid:3) ι ∆ )]( u + d ) , (27)where d ∈ ∆ . As ( Id − R )( d ) =
0, we can write equations (26) as0 ∈ ∂ ( f (cid:3) ι ∆ )( u + d ) + ( Id − R )( u + d ) .However, because f (cid:3) ι ∆ might not be lower semicontinuous, ∂ ( f (cid:3) ι ∆ ) need not be maximallymonotone. Therefore, by (27) we consider the enlargement0 ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( u + d ) + ( Id − R )( u + d ) .12ith d = − P ∆ ( u ) ∈ ∆ and x = u + d = P ∆ ⊥ ( u ) ∈ ∆ ⊥ , we have0 ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R )( x ) , and x ∈ ∆ ⊥ . (28)The solution x given by (28) is called a generalized cycle of cl ( f (cid:3) ι ∆ ) . This is so-called because of (3)and that (28) can be reformulated as x = Prox cl ( f (cid:3) ι ∆ ) ( Rx ) and x ∈ ∆ ⊥ . One amazing property of(28) is that it always has a solution, see Theorem 4.2(ii) below! The generalized cycles and gap vectors of cl ( f (cid:3) ι ∆ ) can be put into the frame work of the Attouch-Th´era duality. Theorem 4.2 (generalized cycle and gap vectors via duality) . Consider the following Attouch-Th´eraprimal-dual problems ( ˜ P ) ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R ) x , and x ∈ ∆ ⊥ , (29) ( ˜ D ) ∈ ∂ ( f ∗ + ι ∗ ∆ )( y ) + y + T y . (30) Then the following hold: (i) ( ˜ D) has a unique solution. (ii) ( ˜ P ) is the Attouch-Th´era dual of ( ˜ D ) associated with the pair ( ∂ ( f ∗ + ι ∗ ∆ ) , Id /2 + T ) , and ( ˜ P) has aunique solution.Proof . (i): ( ˜ D ) has a unique solution by Proposition 4.1(i).(ii): Let us compute the Attouch-Th´era dual of ( ˜ D ) associated with the pair ( ∂ ( f ∗ + ι ∗ ∆ ) , Id /2 + T ) . Because ( f ∗ + ι ∗ ∆ ) ∗ = cl ( f (cid:3) ι ∆ ) and ∂ (( f ∗ + ι ∗ ∆ ) ∗ ) = ( ∂ ( f ∗ + ι ∗ ∆ )) − , we have the Attouch-Th´eradual 0 ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + (cid:18)
12 Id + T (cid:19) − ( x ) . (31)Since ( Id + T ) − = Id − R + P ∆ by Fact 2.7(iii), we obtain0 ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R ) x + P ∆ ( x ) .Because ran ( Id − R ) ⊆ ∆ ⊥ and Lemma 2.2, the above implies − P ∆ ( x ) ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R ) x ⊆ ∆ ⊥ ,from which 2 P ∆ ( x ) ∈ ∆ ∩ ∆ ⊥ , so P ∆ ( x ) =
0, and x ∈ ∆ ⊥ . Hence, (31) is equivalent to0 ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R ) x , and x ∈ ∆ ⊥ , (32)which is precisely (29).Now ( ˜ P ) has solutions because of (i) and Fact 2.6. To prove that the solution of ( ˜ P ) is unique,let x , ˜ x be two solutions to ( ˜ P ). Since − ( Id − R )( x ) , − ( Id − R )( ˜ x ) have to be the unique solution to( ˜ D ), we have x − ˜ x ∈ ker ( Id − R ) = ∆ . Because x − ˜ x ∈ ∆ ⊥ , we conclude that x − ˜ x = (cid:4) .4 Relationship between the classical cycles and generalized cycles It is natural to ask how does the solution of ( ˜ P ) relate to the classical cycles of f . Theorem 4.3.
Let x be the generalized cycle of cl ( f (cid:3) ι ∆ ) , i.e., ( ˜ P ) ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R ) x , and x ∈ ∆ ⊥ . If cl ( f (cid:3) ι ∆ )( x ) = ( f (cid:3) ι ∆ )( x ) and f (cid:3) ι ∆ is exact at x , then ∃ u ∈ X such that x = u + v , v = − P ∆ ( u ) ∈ ∆ ,and ∈ ∂ f ( u ) + ( Id − R ) u . Consequently, u is a classical cycle for f .Proof . As cl ( f (cid:3) ι ∆ )( x ) = ( f (cid:3) ι ∆ )( x ) , by Lemma 2.4 we have ∂ [ cl ( f (cid:3) ι ∆ )]( x ) = ∂ ( f (cid:3) ι ∆ )( x ) . (33)Because f (cid:3) ι ∆ is exact at x , there exist u ∈ X , v ∈ ∆ such that x = u + v and ( f (cid:3) ι ∆ )( x ) = f ( u ) + ι ∆ ( v ) . Using (33) and Lemmas 2.5, we obtain ∂ [ cl ( f (cid:3) ι ∆ )]( x ) = ∂ ( f (cid:3) ι ∆ )( x ) = ∂ f ( u ) ∩ N ∆ ( v ) = ∂ f ( u ) ∩ ∆ ⊥ .Also ( Id − R )( u + v ) = ( Id − R ) u because Rv = v and v ∈ ∆ . Hence, ( ˜ P ) simplifies to0 ∈ ∂ f ( u ) + ( Id − R ) u .Finally, x = u + v , x ∈ ∆ ⊥ and v ∈ ∆ imply 0 = P ∆ ( x ) = P ∆ u + v , so v = − P ∆ ( u ) . (cid:4) Theorem 4.4.
Let u be a classical cycle for f , i.e., ∈ ∂ f ( u ) + ( Id − R ) u . (34) Set v = − P ∆ ( u ) ∈ ∆ and x = u + v = P ∆ ⊥ ( u ) . Then (i) f (cid:3) ι ∆ is lower semicontinuous and exact at x . (ii) x ∈ ∆ ⊥ and x solves ( ˜ P ) ∈ ∂ ( f (cid:3) ι ∆ )( x ) + ( Id − R ) x = ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R ) x . (35) Consequently, x is the generalized cycle for cl ( f (cid:3) ι ∆ ) . (iii) P ∆ ⊥ ( Z ) = { x } .Proof . (i): As Ru − u ∈ ∆ ⊥ , (34) implies that Ru − u ∈ ∂ f ( u ) ∩ ∆ ⊥ = ∂ f ( u ) ∩ ∂ι ∆ ( v ) = ∅ . ByLemma 2.5(ii), f (cid:3) ι ∆ is exact at x = u + v , i.e., ( f (cid:3) ι ∆ )( x ) = f ( u ) + ι ∆ ( v ) . Apply Lemma 2.5(i) toobtain ∂ ( f (cid:3) ι ∆ )( x ) = ∂ f ( u ) ∩ ∆ ⊥ = ∅ . Then by Lemma 2.4 we have cl ( f (cid:3) ι ∆ )( x ) = ( f (cid:3) ι ∆ )( x ) and ∂ [ cl ( f (cid:3) ι ∆ )]( x ) = ∂ ( f (cid:3) ι ∆ )( x ) = ∂ f ( u ) ∩ ∆ ⊥ . (36)14ii): Clearly x ∈ ∆ ⊥ . Moreover, ( Id − R ) x = ( Id − R )( u ) + ( Id − R )( v ) = ( Id − R )( u ) ∈ ∆ ⊥ . (37)Using (36) and (37), we can rewrite (34) as0 ∈ ∂ f ( u ) + ( Id − R ) u ⇒ ∈ ∂ f ( u ) ∩ ∆ ⊥ + ( Id − R ) x (38) ⇒ ∈ ∂ ( f (cid:3) ι ∆ )( x ) + ( Id − R ) x = ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R ) x , (39)which is (35).(iii): Clear from (ii). (cid:4) The following result summarizes the relationship among the classical cycles, generalized cycles,and generalized gap vectors.
Corollary 4.5.
With x and y given in Theorem 4.2, the following hold: (i) x , y ∈ ∆ ⊥ . (ii) y = Rx − x . (iii) x = − y − T y . (iv) Z = ( x + ∆ ) ∩ ( ∂ f ) − ( Rx − x ) = ( Id − R ) − ( − y ) ∩ ( ∂ f ) − ( y ) . (v) Z ⊆ ( F × · · · × F m ) ∩ ( Id − R ) − ( − y ) .Proof . (i): This follows from Theorem 4.2 and dom ( f ∗ + ι ∗ ∆ ) ⊆ dom ι ∗ ∆ = dom ι ∆ ⊥ = ∆ ⊥ .(ii): Set v = Rx − x . By the assumption, − v = x − Rx , v ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) , from which x ∈ ∂ ( f ∗ + ι ∗ ∆ )( v ) and − x ∈ ( Id − R ) − ( v ) . In view of Fact 2.7(ii), it follows that0 ∈ ∂ ( f ∗ + ι ∗ ∆ )( v ) + ( Id − R ) − ( v ) = ∂ ( f ∗ + ι ∗ ∆ )( v ) + v + N ∆ ⊥ ( v ) + T ( v ) (40) = ∂ ( f ∗ + ι ∗ ∆ )( v ) + v + ∆ + T ( v ) = [ ∂ ( f ∗ + ι ∗ ∆ )( v ) + ∆ ] + v + T ( v ) (41) = ∂ ( f ∗ + ι ∗ ∆ )( v ) + v + T ( v ) , (42)which implies v ∈ ∆ ⊥ and v solves ( ˜ D ). By Theorem 4.2(i), we conclude v = y .(iii): Set v = − y − T y . Then − y ∈ ( Id2 + T ) − ( v ) and, by the assumption, y ∈ [ ∂ ( f ∗ + ι ∗ ∆ )] − ( v ) = ∂ [ cl ( f (cid:3) ι ∆ )]( v ) .In view of Fact 2.7(iii), it follows that0 ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( v ) + (cid:18) Id2 + T (cid:19) − ( v ) (43) = ∂ [ cl ( f (cid:3) ι ∆ )]( v ) + ( Id − R + P ∆ )( v ) . (44)15hen − P ∆ ( v ) ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( v ) + ( Id − R ) v ⊆ ∆ ⊥ (45)gives 2 P ∆ ( v ) ∈ ∆ ∩ ∆ ⊥ , so P ∆ ( v ) = v ∈ ∆ ⊥ . Therefore, (45) reduces to0 ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( v ) + ( Id − R ) v , and v ∈ ∆ ⊥ .By Theorem 4.2(ii), v = x .(iv): Suppose u ∈ Z . By Theorem 4.4, x = u + v solves (35), v ∈ ∆ , and ∂ ( f (cid:3) ι ∆ )( x ) = ∂ f ( u ) ∩ ∆ ⊥ . We have u = x − v ∈ x + ∆ and0 ∈ ∂ ( f (cid:3) ι ∆ )( x ) + x − Rx ⇒ ∈ ∂ f ( u ) + x − Rx ,which implies u ∈ ( ∂ f ) − ( Rx − x ) . Then u ∈ ( x + ∆ ) ∩ ( ∂ f ) − ( Rx − x ) .Conversely, let u ∈ ( x + ∆ ) ∩ ( ∂ f ) − ( Rx − x ) . We have u ∈ x − ∆ and u ∈ ( ∂ f ) − ( Rx − x ) . Then Ru − u = Rx − x ∈ ∂ f ( u ) so that 0 ∈ ∂ f ( u ) + u − Ru . Hence u ∈ Z .The second equality follows from (ii) and ( Id − R ) − ( − y ) = x + ∆ .(v): Apply (iv) and Lemma 3.1(v). (cid:4) Remark 4.6.
Corollary 4.5(ii)&(iii) can also be proved by using [6, Remark 5.4] via paramonotonicity.
Recall that the parallel sum of ∂ f and ∂ g is defined by ( ∀ x ∈ X ) ( ∂ f (cid:3) ∂ g )( x ) = S x = u + v ∂ f ( u ) ∩ ∂ g ( v ) . Corollary 4.7 (existence of classical cycles) . Let x be given in Theorem 4.2. Then the following areequivalent: (i) Z = ∅ . (ii) [ cl ( f (cid:3) ι ∆ )]( x ) = ( f (cid:3) ι ∆ )( x ) and f (cid:3) ι ∆ is exact at x . (iii) ( ∂ f (cid:3) ∂ι ∆ )( x ) = ∅ .Proof . Observe that x solving ( ˜ P ) implies ∂ [ cl ( f (cid:3) ι ∆ )]( x ) = ∅ . (46)(i) ⇔ (ii): Combine Theorems 4.3 and 4.4.(ii) ⇒ (iii): Lemma 2.4 gives ∂ ( f (cid:3) ι ∆ )( x ) = ∂ [ cl ( f (cid:3) ι ∆ )]( x ) . (47)Since f (cid:3) ι ∆ is exact at x , by Lemma 2.5(i) there exist u , v ∈ X such that x = u + v and ∂ ( f (cid:3) ι ∆ )( x ) = ∂ f ( u ) ∩ ∂ι ∆ ( v ) . (48)Combining (46), (47) and (48), we get ( ∂ f (cid:3) ∂ι ∆ )( x ) = ∅ .(iii) ⇒ (ii): ( ∂ f (cid:3) ∂ι ∆ )( x ) = ∅ implies there exist u , v ∈ X such that x = u + v and ∂ f ( u ) ∩ ∂ι ∆ ( v ) = ∅ . By Lemma 2.5(ii), f (cid:3) ι ∆ is exact at x , and ∂ ( f (cid:3) ι ∆ )( x ) = ∂ f ( u ) ∩ ∂ι ∆ ( v ) = ∅ , so f (cid:3) ι ∆ islower semicontinuous at x by Lemma 2.4(i). (cid:4) .5 More on generalized cycles Let us reconsider ( ˜ P ) without the restriction x ∈ ∆ ⊥ , namely,0 ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R ) x . (49)Denote the set of solutions to (49) by E . In order to study the set E , we shall need one lemma. Lemma 4.8.
The Attouch-Th´era dual of (49) for the pair ( ∂ [ cl ( f (cid:3) ι ∆ )] , Id − R ) is ( ˜ D ) ∈ ∂ ( f ∗ + ι ∗ ∆ )( y ) + y + T y . (50) In particular, ( ˜ P) and (49) have exactly the same dual problem.Proof . The Attouch-Th´era dual of (49) for the pair ( ∂ [ cl ( f (cid:3) ι ∆ )] , Id − R ) is0 ∈ ∂ ( f ∗ + ι ∗ ∆ )( y ) + ( Id − R ) − ( y ) . (51)Now ( Id − R ) − = Id2 + T + N ∆ ⊥ by Fact 2.7(ii) and ι ∗ ∆ = ι ∆ ⊥ , (51) becomes0 ∈ ∂ ( f ∗ + ι ∆ ⊥ )( y ) + y + T y + N ∆ ⊥ ( y ) = ∂ ( f ∗ + ι ∆ ⊥ )( y ) + N ∆ ⊥ ( y ) + y + T y . (52)Because dom ( f ∗ + ι ∆ ⊥ ) = dom f ∗ ∩ ∆ ⊥ ⊆ ∆ ⊥ , N ∆ ⊥ ( y ) = ∆ if y ∈ ∆ ⊥ and ∅ if y ∆ ⊥ , and 0 ∈ ∆ ,we have ∂ ( f ∗ + ι ∆ ⊥ )( y ) + N ∆ ⊥ ( y ) = ∂ ( f ∗ + ι ∆ ⊥ )( y ) . (53)Hence (50) holds by combining (52) and (53). (cid:4) Theorem 4.9.
Let x be given in Theorem 4.2. Then (i) E = x + ∆ . Consequently, the problem (49) always has infinitely many solutions. (ii) P ∆ ⊥ ( E ) = { x } .Proof . (i): Let d ∈ ∆ . Since ( Id − R )( x + d ) = ( Id − R )( x ) , Lemma 2.3 gives that x + d ∈ E . Then x + ∆ ⊆ E . To show that x + ∆ ⊇ E , let ˜ x ∈ E . By Theorem 4.2 and Lemma 4.8, the uniqueness ofsolution to ( ˜ D ) implies y = Rx − x = R ˜ x − ˜ x , from which ˜ x − x ∈ ker ( Id − R ) = ∆ , so ˜ x ∈ x + ∆ .Then E ⊆ x + ∆ . Altogether E = x + ∆ .(ii): Use (i) and x ∈ ∆ ⊥ . (cid:4) (cid:3) ι ∆ lower semicontinuous and exact? When f (cid:3) ι ∆ is closed and exact, we have that the set of classical cycles of f is nonempty by Corol-lary 4.7, and that Theorem 4.3 allows us to find the classical cycle of f . Thus, it is important to givesome conditions under which f (cid:3) ι ∆ is closed and exact.17 roposition 5.1. Suppose that one of the following holds: (i)
The conical hull of ( dom f ∗ × · · · × dom f ∗ m ) − ∆ ⊥ is a closed subspace. (ii) The function d ( f ( d ) + · · · + f m ( d )) is coercive and the canonical hull of ( dom f × · · · × dom f m ) − ∆ is a closed subspace. (iii) dom f ∗ × · · · × dom f ∗ m = X . (iv) f ∗ ⊕ · · · ⊕ f ∗ m is continuous at some point in ∆ ⊥ .(v) X is finite-dimensional and ( ri dom f ∗ × · · · × ri dom f ∗ m ) ∩ ∆ ⊥ = ∅ .Then f (cid:3) ι ∆ is proper, lower semicontinuous and convex, and exact on X .Proof . Apply [7, Proposition 15.7] and [7, Proposition 15.5] to f and ι ∆ , and use f ∗ = f ∗ ⊕ · · · ⊕ f ∗ m and ι ∗ ∆ = ι ∆ ⊥ . (cid:4) Proposition 5.2.
Let inf f i > − ∞ for i =
1, . . . , m, and suppose that at least one of f i is coercive. Then f (cid:3) ι ∆ is proper, lower semicontinuous and convex, and exact on X .Proof . Without loss of any generality, we can assume that f is coercive. Because dom ( f (cid:3) ι ∆ ) = dom f + ∆ and inf f i > − ∞ for all i , the function f (cid:3) ι ∆ is proper, and convex by [7, Proposition12.11].First, we show that f (cid:3) ι ∆ is exact. By the definition, for x = ( x , . . . , x m ) ∈ X , ( f (cid:3) ι ∆ )( x ) = inf d ∈ X (cid:0) f ( x − d ) + · · · + f m ( x m − d ) (cid:1) = inf d ∈ X g ( d ) ,where g ( d ) = f ( x − d ) + · · · + f m ( x m − d ) . Since f is coercive, and g ( d ) ≥ f ( x − d ) + m ∑ i = inf f i we have that g is coercive, so g has a minimizer over X . Hence f (cid:3) ι ∆ is exact.Next, we show that f (cid:3) ι ∆ is lower semicontinuous. Let x = ( x , . . . , x m ) ∈ X and x n → x with x n = ( x ( n ) , . . . , x ( n ) m ) ∈ X . We show that ( f (cid:3) ι ∆ )( x ) ≤ lim inf n → ∞ ( f (cid:3) ι ∆ )( x n ) . (54)Set µ = lim inf n → ∞ ( f (cid:3) ι ∆ )( x n ) . When µ = + ∞ , (54) clearly holds, so we assume that µ < + ∞ .After passing to a subsequence and relabelling, we can assume that ( f (cid:3) ι ∆ )( x n ) → µ ∈ [ − ∞ , + ∞ [ .Let ( d n ) n ∈ N be a sequence in X such that ( f (cid:3) ι ∆ )( x n ) = f ( x ( n ) − d n ) + · · · + f m ( x ( n ) m − d n ) .We show that ( d n ) n ∈ N is bounded. Suppose this is not the case. Taking a subsequence if necessary,we can assume that k d n k → ∞ . Then + ∞ > µ ← ( f (cid:3) ι ∆ )( x n ) = f ( x ( n ) − d n ) + · · · + f m ( x ( n ) m − d n ) (55)18 f ( x ( n ) − d n ) + m ∑ i = inf f i → + ∞ , (56)because ( k x n k ) n ∈ N is bounded, k d n k → ∞ , and f is coercive. This contradiction shows that ( d n ) n ∈ N has to be bounded. After passing to a subsequence and relabelling, we can assume that d n ⇀ d ∈ X , i.e., d n converges to d weakly. Then x ( n ) i − d n ⇀ x i − d , and using weakly lowersemicontinuity of f i we have µ = lim n → ∞ ( f (cid:3) ι ∆ )( x n ) = lim n → ∞ ( f ( x ( n ) − d n ) + · · · + f m ( x ( n ) m − d n )) (57) ≥ m ∑ i = lim inf n → ∞ f i ( x ( n ) i − d n ) ≥ m ∑ i = f i ( x i − d ) ≥ ( f (cid:3) ι ∆ )( x ) , (58)which gives (54). (cid:4) Corollary 5.3.
Suppose that f is coercive. Then f (cid:3) ι ∆ is proper, lower semicontinuous and convex, andexact.Proof . Because f = f ⊕ · · · ⊕ f m on X , we have that f is coercive if and only if each f i is coerciveon X ; in particular, 0 ∈ int dom f ∗ × · · · × int dom f ∗ m , see, e.g., [7, Proposition 14.16]. ApplyProposition 5.1(i) or Proposition 5.2. (cid:4) Without the coercivity of f i , one can exploit the structure of f i . Recall that when X is finite-dimensional, we call f : X → ] − ∞ , + ∞ ] a polyhedral convex function if f ( x ) = h ( x ) + ι C ( x ) wherefor every x ∈ X , h ( x ) = max (cid:8) h x , b i i − β i (cid:12)(cid:12) i =
1, . . . , k (cid:9) , C = (cid:8) x ∈ X (cid:12)(cid:12) h x , b i i ≤ β i , i = k +
1, . . . , m (cid:9) with fixed b i ∈ X , β i ∈ R ; see, e.g., [17, Section 19, page 172]. Proposition 5.4.
Suppose that X is finite-dimensional, and each f i : X → ] − ∞ , + ∞ ] is a proper poly-hedral convex function for i =
1, . . . , m. If f (cid:3) ι ∆ is proper, then f (cid:3) ι ∆ is a polyhedral convex function (solower semicontinuous) and f (cid:3) ι ∆ is exact.Proof . By [17, Theorem 19.4], f = f ⊕ · · · ⊕ f m is a proper polyhedral convex function on X . Also ι ∆ is a proper polyhedral convex function. Because f (cid:3) ι ∆ is proper, [17, Corollary 19.3.4] gives theresult. (cid:4) Corollary 5.5.
Suppose that X is finite-dimensional, and f i = ι C i with each C i ⊆ X being a polyhedralconvex set for i =
1, . . . , m. Then f (cid:3) ι ∆ = ι C + ∆ is lower semicontinuous and exact; equivalently, C + ∆ isa closed set.Proof . Because ι C (cid:3) ι ∆ = ι C + ∆ is proper, Proposition 5.4 applies. (cid:4) Examples
In this section, we apply the results of Section 4 to concrete examples. The first example illustratesthe concepts of generalized cycles and gap vectors when the classical ones do not exist. The secondexample characterizes when the set of cycles is a singleton or infinite for a finite number of linesin a Hilbert space. For C i ⊆ X , i =
1, . . . , m , we let C = C × · · · × C m ⊆ X . Example 6.1.
Let α ≥ . ConsiderC = epi exp = (cid:8) ( x , r ) (cid:12)(cid:12) r ≥ exp ( − x ) + α and x ∈ R (cid:9) , and C = R × { } . Then (i) ι C has neither a cycle nor a gap vector. (ii) cl ( ι C (cid:3) ι ∆ ) = ι C + ∆ has both a generalized cycle and a generalized gap vector, namely x =( α /2, 0, − α /2 ) , y = ( − α , 0, α ) ∈ R .Proof . Note that C = C × C , ∆ = (cid:8) ( u , v , u , v ) (cid:12)(cid:12) u , v ∈ R (cid:9) are in R × R = R .(i): With z , z ∈ R , we need to show that (
0, 0 ) ∈ ∂ι C ( z ) × ∂ C ( z ) + ( Id − R )( z , z ) (59)has no solution. Suppose (59) has a solution. Then there exist z i ∈ C i and z − z ∈ N C ( z ) , z − z ∈ N C ( z ) = { } × R , so z − z = ( y ) ∈ N C ( z ) for some y ∈ R . Because N C ( x , exp ( − x )) = [ λ ≥ λ ( − exp ( − x ) , − ) ,and N C ( z ) = { } if z ∈ int C , this implies y = z = z . This contradicts C ∩ C = ∅ .Because (59) has no solution, so ι C has no gap vector.(ii): cl ( ι C (cid:3) ι ∆ ) has a generalized cycle. We show that (
0, 0 ) ∈ N C + ∆ ( x ) + ( Id − R )( x ) and x ∈ ∆ ⊥ (60)has a solution ( α /2, 0, − α /2 ) . Indeed, since (
0, exp ( − x ) + α /2, 0, − α /2 ) = ( x , exp ( − x ) + α , x , 0 ) + ( − x , − α /2, − x , − α /2 ) ∈ C + ∆ ,by sending x → ∞ we have ( α /2, 0, − α /2 ) ∈ C + ∆ ∩ ∆ ⊥ . With z = ( α /2 ) = − z , we showthat x = ( z , z ) solves (60), i.e., (
0, 0 ) ∈ N C + ∆ ( α /2, 0, − α /2 ) + ( α , 0, − α ) .This holds because ( α , 0, − α ) ∈ ∆ ⊥ , and for r ≥ exp ( − x ) + α , x , ˜ x ∈ R , ( u , v , u , v ) ∈ ∆ , (cid:10) ( α , 0, − α ) , (cid:0) ( x , r , ˜ x , 0 ) + ( u , v , u , v ) (cid:1) − ( α /2, 0, − α /2 ) (cid:11) h ( α , 0, − α ) , ( x , r , ˜ x , 0 ) − ( α /2, 0, − α /2 ) i = α ( r − α /2 ) − α ( α /2 ) = α ( r − α ) ≥ y = Rx − x = ( − α , 0, α ) . (cid:4) Example 6.2.
Given m sets in X: C i = (cid:8) a i + t i b i (cid:12)(cid:12) t i ∈ R (cid:9) where a i ∈ X and b i ∈ X \ { } for i =
1, . . . , m. Then the following hold: (i) ι C always has a classical cycle, i.e., Z = ∅ . (ii) ι C has a unique classical cycle if and only if the set of vectors (cid:8) b i (cid:12)(cid:12) i =
1, . . . , m (cid:9) is not parallel. (iii) ι C has infinitely many classical cycles if and only if the set of vectors (cid:8) b i (cid:12)(cid:12) i =
1, . . . , m (cid:9) is parallel.Proof . (i): Using the special structures of C and ∆ , one can show that C + ∆ is closed in X . Then ι C (cid:3) ι ∆ = ι C + ∆ is lower semicontinuous and exact, so ι C has a classical cycle by Corollary 4.7.(ii): Write u = ( u , . . . , u m ) . u is a cycle of ι C means 0 ∈ u − Ru + ∂ι C ( u ) , i.e., − ( u − u m ) ∈ ∂ C ( u ) , and u i − u i − ∈ ∂ι C i ( u i ) for i =
2, . . . , m .As ∂ι C i ( u i ) = (cid:8) x ∈ X (cid:12)(cid:12) h b i , x i = (cid:9) for u i ∈ C i , we have h b , u − u m i = h b i , u i − u i − i = i =
2, . . . , m .As u i = a i + t i b i , the above become h b , a − a m i + h b , b i t − h b , b m i t m = h b i , a i − a i − i − h b i , b i − i t i − + h b i , b i i t i = h b m , a m − a m − i − h b m , b m − i t m − + h b m , b m i t m =
0. (63)In the form of At = b with t = ( t , . . . , t m ) ⊺ and b = ( h b , a − a m i , h b , a − a i , . . . , h b m , a m − a m − i ) ⊺ ,the linear system of equations (61)–(63) has A = h b , b i · · · − h b , b m i− h b , b i h b , b i · · · − h b , b i h b , b i · · · · · · · · · − h b i , b i − i h b i , b i i · · · − h b m , b m − i h b m , b m i ,and is consistent by (i). Using co-factor expansions, e.g. [16, p. 478], we obtain the determinantdet A = m ∏ i = h b i , b i i − h b , b m i m ∏ i = h b i , b i − i = m ∏ i = h b i , b i i (cid:18) − h b , b m ik b kk b m k m ∏ i = h b i , b i − ik b i kk b i − k (cid:19) .21hen det A = { b i / k b i k| i =
1, . . . , m } is a collection of parallel vectors. Because At = b has a unique solution if and only if det A =
0, the result follows.(iii): This follows from (ii). (cid:4)
The objective of this final section is to show that one can compute the generalized cycle and gapvectors by the forward-backward algorithms.To solve 0 ∈ ∂ f ( x ) + ( Id − R )( x ) , we start with a fixed point reformulation. Lemma 7.1.
Let γ ∈ ]
0, 1 [ . Then Z = Fix Prox f R = Fix Prox γ f (cid:0) ( − γ ) Id + γ R (cid:1) . (64) In particular, for γ = , we have Z = Fix Prox f /2 (cid:0) ( Id + R ) /2 (cid:1) ; consequently, Z is the fixed point setof a product of firmly nonexpansive mappings or resolvents.Proof . We have x = Prox f Rx ⇔ ∈ ∂ f ( x ) + x − Rx ⇔ ∈ γ∂ f ( x ) + γ ( x − Rx ) (65) ⇔ − γ x + γ Rx ∈ ∂ ( γ f )( x ) ⇔ ( − γ ) x + γ Rx ∈ x + ∂ ( γ f )( x ) (66) ⇔ x = Prox γ f (cid:18) ( − γ ) x + γ Rx (cid:19) . (67) (cid:4) The fixed point characterization of Z , i.e., (64), suggests finding x ∈ Z by the forward-backwarditeration x n + = Prox γ f (cid:18) ( − γ ) x n + γ Rx n (cid:19) where γ ∈ ]
0, 1 [ . When Z = ∅ , x n ⇀ x for some x ∈ Z and Rx n − x n → y the gap vector; when Z = ∅ , k x n k → + ∞ (see [8, Fact 2.2] or [13]), but the asymptotic behavior of ( Rx n − x n ) n ∈ N isnot clear. See also [7, Theorem 5.23] for the convergence behavior of iterates of compositions ofaveraged operators.In view of the possibility of Z = ∅ , one can consider the extended Attouch-Th´era primal-dual: ( EP ) ∈ ∂ [ cl ( f (cid:3) ι ∆ )]( x ) + ( Id − R ) x , (68) ( ED ) ∈ ∂ ( f ∗ + ι ∗ ∆ )( y ) + y + T y . (69)22oth (EP) and (ED) always have solutions. While (EP) gives all generalized cycles, (ED) gives theunique generalized gap vector for cl ( f (cid:3) ι ∆ ) . To make the notation simple in the following proof,let us write g = cl ( f (cid:3) ι ∆ ) .The following forward-backward iteration scheme allows us to find the extended cycles and gapvectors. The elegance of our approach is that ( Rx n − x n ) n ∈ N always converge strongly to theunique extended gap vector, which is precisely the classical one if the latter exists; see Proposi-tion 4.1(ii). Theorem 7.2.
Let γ ∈ ]
0, 1 [ , δ = − γ , and let ( λ n ) n ∈ N be a sequence in ] δ [ such that ∑ n ∈ N λ n ( δ − λ n ) = + ∞ . Let x ∈ X and set for n =
0, 1, . . . (cid:22) y n = ( − γ ) x n + γ Rx n , x n + = x n + λ n ( Prox γ g y n − x n ) . (70) Then the following hold: (i) ( x n ) n ∈ N converges weakly to ˜ x , a generalized cycle of g , i.e., a solution of (EP). (ii) ( R x n − x n ) n ∈ N converges strongly to y = R ˜ x − ˜ x , the unique generalized gap vector of g , i.e., thesolution of (ED). (iii) ( P ∆ ⊥ x n ) n ∈ N converges strongly to P ∆ ⊥ ˜ x = x , the unique solution to ( ˜ P ) .Proof . (i)&(ii): Let A = ∂ g and B = Id − R . Then A : X ⇒ X is maximally monotone, and B : X → X is 1/2-cocercive by Fact 2.8. As mentioned earlier, (EP) always has solutions, i.e., zer ( A + B ) = ∅ .Apply [7, Theorem 26.14(i)&(ii)] with β = P ∆ ⊥ = (cid:18) Id2 + T (cid:19) ( Id − R ) . (71)Indeed, since that (cid:0) Id /2 + T (cid:1) − = Id − R + P ∆ by Fact 2.7(iii), that Id /2 + T is linear, and ∆ ⊆ ker T , we haveId = (cid:18) Id2 + T (cid:19)(cid:18) Id2 + T (cid:19) − = (cid:18) Id2 + T (cid:19) ( Id − R ) + (cid:18) Id2 + T (cid:19) P ∆ (72) = (cid:18) Id2 + T (cid:19) ( Id − R ) + P ∆ , (73)from which P ∆ ⊥ = Id − P ∆ = (cid:18) Id2 + T (cid:19) ( Id − R ) .This, together with (ii), Corollary 4.5(iii) and Theorem 4.9, gives that when n → ∞ , P ∆ ⊥ ( x n ) = (cid:18) Id2 + T (cid:19) ( x n − Rx n ) → (cid:18) Id2 + T (cid:19) ( − y ) = x = P ∆ ⊥ ( ˜ x ) . (74)23 When λ n ≡
1, an immediate consequence of Theorem 7.2 comes as follows.
Corollary 7.3.
Let γ ∈ ]
0, 1 [ , let x ∈ X and set ( ∀ n ∈ N ) x n + = Prox γ g (cid:0) ( − γ ) x n + γ Rx n (cid:1) . (75) Then x n ⇀ ˜ x , a generalized cycle of g ; R x n − x n → y , the unique generalized gap vector of g ; P ∆ ⊥ x n → x ,the unique solution to ( ˜ P ) . Remark 7.4.
In this regard, see [2] for finding cycles and gap vectors of compositions of projections, andalso [8, Section 3.3.3] for an abstract framework.
Acknowledgments
HHB and XW are supported by the Natural Sciences and Engineering Research Council ofCanada.
References [1] S. Alwadani, H.H. Bauschke, J.P. Revalski, and X. Wang, Resolvents and Yosidaapproximations of displacement mappings of isometries, June 2020, submitted, https://arxiv.org/abs/2006.04860 .[2] S. Alwadani, H.B. Bauschke, J.P. Revalski, and X. Wang, The difference vectors forconvex sets and a resolution of the geometry conjecture, December 2020, submitted, https://arxiv.org/pdf/2012.04784.pdf [3] H. Attouch and M. Th´era, A general duality principle for the sum of two operators,
Journalof Convex Analysis
Journal of Functional Analysis
262 (2012),400–408.[5] J.-B. Baillon, P.L. Combettes, and R. Cominetti, Asymptotic behavior of compositions ofunder-relaxed nonexpansive operators,
Journal of Dynamics and Games
Journal of Approximation Theory
164 (2012), 1065–1084.[7] H.H. Bauschke and P.L. Combettes,
Convex Analysis and Monotone Operator Theory in HilbertSpaces , second edition, Springer, 2017.[8] H.H. Bauschke, P.L. Combettes, and S. Reich, The asymptotic behavior of the compositionof two resolvents.
Nonlinear Analysis
60 (2005), 283–301.249] H.H. Bauschke, V. Mart´ın-M´arquez, S.M. Moffat, and X. Wang, Compositions and convexcombinations of asymptotically regular firmly nonexpansive mappings are also asymptoti-cally regular,
Fixed Point Theory and Applications
Nonlinear Analysis
56 (2004), 715–738.[11] H.H. Bauschke, X. Wang, and L. Yao, Rectangularity and paramonotonicity of maximallymonotone operators,
Optimization
63 (2014), 487–504.[12] H.H. Bauschke, S.M. Moffat, and X. Wang, Near equality, near convexity, sums of maxi-mally monotone operators, and averages of firmly nonexpansive mappings,
MathematicalProgramming
139 (2013), Ser. B, 55–70.[13] R.E. Bruck and S. Reich, Nonexpansive projections and resolvents of accretive operators inBanach spaces,
Houston Journal of Mathematics https://arxiv.org/pdf/2008.02260.pdf [15] P.L. Combettes and J.-C. Pesquet, Deep neural network structures solving variational in-equalities,
Set-Valued and Variational Analysis
28 (2020), 491–518.[16] C.D. Meyer,
Matrix Analysis and Applied Linear Algebra , SIAM, 2000.[17] R.T. Rockafellar,
Convex Analysis , Princeton University Press, Princeton, 1970.[18] R.T. Rockafellar and R.J-B Wets,
Variational Analysis , Springer-Verlag, corrected 3rd printing,2009.[19] S. Simons,
From Hahn-Banach to Monotonicity , Springer-Verlag, 2008.[20] T. Suzuki, Some notes on Bauschke’s condition,
Nonlinear Analysis
67 (2007), 2224–2231.[21] X. Wang and H.H. Bauschke, Compositions and averages of two resolvents: relative geom-etry of fixed points sets and a partial answer to a question by C. Byrne,
Nonlinear Analysis