Variable metric algorithms driven by averaged operators
aa r X i v : . [ m a t h . O C ] J u l Chapter 1
Variable metric algorithms driven by averagedoperators
Lilian E. Glaudin
Abstract
The convergence of a new general variable metric algorithm based oncompositions of averaged operators is established. Applications to monotone oper-ator splitting are presented.
Key words: averaged operator, composite algorithm, convex optimization, fixedpoint iteration, monotone operator splitting, primal-dual algorithm, variable metric
AMS 2010 Subject Classification:
Iterations of averaged nonexpansive operators provide a synthetic framework for theanalysis of many algorithms in nonlinear analysis, e.g., [3, 4, 7, 9, 18]. We establishthe convergence of a new general variable metric algorithm based on compositionsof averaged operators. These results are applied to the analysis of the convergenceof a new forward-backward algorithm for solving the inclusion0 ∈ Ax + Bx , (1.1)where A and B are maximally monotone operators on a real Hilbert space. Thetheory of monotone operators is used in many applied mathematical fields, includingoptimization [10], partial differential equations and evolution inclusions [5, 21, 23],signal processing [13, 17], and statistics and machine learning [12, 19, 20]. In recentyears, variants of the forward-backward algorithm with variable metric have beenproposed in [15, 16, 22, 24], as well as variants involving overrelaxations [18]. Thegoal of the present paper is to unify these two approaches in the general context of Sorbonne Université, Laboratoire Jacques-Louis Lions, 4 place Jussieu, 75005 Paris, France e-mail: [email protected] iterations of compositions of averaged operators. In turn, this provides new methodsto solve the problems studied in [1, 4, 6, 8, 9, 14].The paper is organized as follows: Section 1.2 presents the background and no-tation. We establish the proof of the convergence of the general algorithm in Sec-tion 1.3. Special cases are provided in Section 1.4. Finally, by recasting these resultsin certain product spaces, we present and solve a general monotone inclusion in Sec-tion 1.5.
Throughout this paper, H , G , and ( G i ) i m are real Hilbert spaces. We use h· | ·i to denote the scalar product of a Hilbert space and k · k for the associated norm.Weak and strong convergence are respectively denoted by ⇀ and → . We denote by B ( H , G ) the space of bounded linear operators from H to G , and set B ( H ) = B ( H , H ) and S ( H ) = (cid:8) L ∈ B ( H ) (cid:12)(cid:12) L = L ∗ (cid:9) , where L ∗ denotes the adjoint of L , and Id denotes the identity operator. The Loewner partial ordering on S ( H ) isdefined by ( ∀ U ∈ S ( H ))( ∀ V ∈ S ( H )) U < V ⇔ ( ∀ x ∈ H ) h Ux | x i > h V x | x i . (1.2)Let α ∈ ] , + ∞ [ . We set P α ( H ) = (cid:8) U ∈ S ( H ) (cid:12)(cid:12) U < α Id (cid:9) , (1.3)and we denote by √ U the square root of U ∈ P α ( H ) . Moreover, for every U ∈ P α ( H ) , we define a scalar product and a norm by ( ∀ x ∈ H )( ∀ y ∈ H ) h x | y i U = h Ux | y i and k x k U = p h Ux | x i , (1.4)and we denote this Hilbert space by ( H , U ) . Let A : H → H be a set-valuedoperator. We denote by dom A = (cid:8) x ∈ H (cid:12)(cid:12) Ax = ∅ (cid:9) the domain of A , by gra A = (cid:8) ( x , u ) ∈ H × H (cid:12)(cid:12) u ∈ Ax (cid:9) the graph of A , by ran A = (cid:8) u ∈ H (cid:12)(cid:12) ( ∃ x ∈ H ) u ∈ Ax (cid:9) the range of A , by zer A = (cid:8) x ∈ H (cid:12)(cid:12) ∈ Ax (cid:9) the set of zeros of A , and by A − theinverse of A which is the operator with graph (cid:8) ( u , x ) ∈ H × H (cid:12)(cid:12) u ∈ Ax (cid:9) . Theresolvent of A is J A = ( Id + A ) − . Moreover, A is monotone if ( ∀ ( x , u ) ∈ gra A )( ∀ ( y , v ) ∈ gra A ) h x − y | u − v i > , (1.5)and maximally monotone if there exists no monotone operator B : H → H suchthat gra A ⊂ gra B = gra A . The parallel sum of A : H → H and B : H → H is A (cid:3) B = ( A − + B − ) − . (1.6)An operator B : H → H is cocoercive with constant β ∈ ] , + ∞ [ if Variable metric algorithms driven by averaged operators. 3 ( ∀ x ∈ H )( ∀ y ∈ H ) h x − y | Bx − By i > β k Bx − By k . (1.7)Let C be a nonempty subset of H . The interior of C is int C . Finally, the set ofsummable sequences in [ , + ∞ [ is denoted by ℓ + ( N ) . Definition 1.
Let µ ∈ ] , + ∞ [ , let U ∈ P µ ( H ) , let D be a nonempty subset of H ,let α ∈ ] , ] , and let T : H → H be an operator. Then T is an α -averaged operatoron ( H , U ) if ( ∀ x ∈ H )( ∀ y ∈ H ) k T x − Ty k U k x − y k U − − αα k T x − x k U . (1.8)If α = T is nonexpansive on ( H , U ) . Lemma 1. [4, Proposition 4.46]
Let m > be an integer. For every i ∈ { , . . . , m } ,let T i : H → H be averaged. Then T · · · T m is averaged. Lemma 2. [4, Proposition 4.35]
Let µ ∈ ] , + ∞ [ , let U ∈ P µ ( H ) , let α ∈ ] , ] ,and let T be an α -averaged operator on ( H , U ) . Then the operator R = ( − / α ) Id +( / α ) T is nonexpansive on ( H , U ) . Lemma 3. [4, Lemma 5.31]
Let ( α n ) n ∈ N and ( β n ) n ∈ N be sequences in [ , + ∞ [ , let ( η n ) n ∈ N and ( ε n ) n ∈ N be sequences in ∈ ℓ + ( N ) such that ( ∀ n ∈ N ) α n + ( + η n ) α n − β n + ε n . (1.9) Then ( β n ) n ∈ N ∈ ℓ + ( N ) . Lemma 4. [15, Proposition 4.1]
Let α ∈ ] , + ∞ [ , let ( W n ) n ∈ N be in P α ( H ) , let Cbe a nonempty subset of H , and let ( x n ) n ∈ N be a sequence in H such that (cid:0) ∃ ( η n ) n ∈ N ∈ ℓ + ( N ) (cid:1)(cid:0) ∀ z ∈ C (cid:1)(cid:0) ∃ ( ε n ) n ∈ N ∈ ℓ + ( N ) (cid:1) ( ∀ n ∈ N ) k x n + − z k W n + ( + η n ) k x n − z k W n + ε n . (1.10) Then ( x n ) n ∈ N is bounded and, for every z ∈ C, ( k x n − z k W n ) n ∈ N converges. Proposition 1. [15, Theorem 3.3]
Let α ∈ ] , + ∞ [ , and let ( W n ) n ∈ N and W be oper-ators in P α ( H ) such that W n → W pointwise, as is the case when sup n ∈ N k W n k < + ∞ and ( ∃ ( η n ) n ∈ N ∈ ℓ + ( N ))( ∀ n ∈ N ) ( + η n ) W n < W n + . (1.11) Let C be a nonempty subset of H , and let ( x n ) n ∈ N be a sequence in H such that (1.10) is satisfied. Then ( x n ) n ∈ N converges weakly to a point in C if and only if everyweak sequential cluster point of ( x n ) n ∈ N is in C. Proposition 2. [16, Proposition 3.6]
Let α ∈ ] , + ∞ [ , let ( ν n ) n ∈ N ∈ ℓ + ( N ) , and let ( W n ) n ∈ N be a sequence in P α ( H ) such that sup n ∈ N k W n k < + ∞ and ( ∀ n ∈ N ) ( + Lilian E. Glaudin ν n ) W n + < W n . Furthermore, let C be a subset of H such that int C = ∅ and let ( x n ) n ∈ N be a sequence in H such that (cid:0) ∃ ( ε n ) n ∈ N ∈ ℓ + ( N ) (cid:1)(cid:0) ∃ ( η n ) n ∈ N ∈ ℓ + ( N ) (cid:1) ( ∀ x ∈ H )( ∀ n ∈ N ) k x n + − x k W n + ( + η n ) k x n − x k W n + ε n . (1.12) Then ( x n ) n ∈ N converges strongly. Proposition 3. [15, Proposition 3.4]
Let α ∈ ] , + ∞ [ , let ( W n ) n ∈ N be a sequence in P α ( H ) such that sup n ∈ N k W n k < + ∞ , let C be a nonempty closed subset of H , andlet ( x n ) n ∈ N be a sequence in H such that (cid:0) ∃ ( ε n ) n ∈ N ∈ ℓ + ( N ) (cid:1)(cid:0) ∃ ( η n ) n ∈ N ∈ ℓ + ( N ) (cid:1) ( ∀ z ∈ C )( ∀ n ∈ N ) k x n + − z k W n + ( + η n ) k x n − z k W n + ε n . (1.13) Then ( x n ) n ∈ N converges strongly to a point in C if and only if lim d C ( x n ) = . Lemma 5. [16, Lemma 3.1]
Let α ∈ ] , + ∞ [ , let µ ∈ ] , + ∞ [ , and let A and B beoperators in S ( H ) such that µ Id < A < B < α Id . Then the following hold: (i) α − Id < B − < A − < µ − Id . (ii) ( ∀ x ∈ H ) h A − x | x i > k A k − k x k . (iii) k A − k α − . We present our main result.
Theorem 1.
Let α ∈ ] , + ∞ [ , let ( η n ) n ∈ N ∈ ℓ + ( N ) , and let ( U n ) n ∈ N be a sequencein P α ( H ) such that µ = sup n ∈ N k U n k < + ∞ and ( ∀ n ∈ N ) ( + η n ) U n + < U n . (1.14) Let ε ∈ ] , [ , let m > be an integer, and let x ∈ H . For every i ∈ { , . . . , m } andevery n ∈ N , let α i , n ∈ ] , [ , let T i , n : H → H be α i , n -averaged on ( H , U − n ) , let φ n an averageness constant of T , n · · · T m , n , let λ n ∈ ] , φ n [ , and let e i , n ∈ H . Iteratefor n = , , . . . $ y n = T , n (cid:16) T , n (cid:0) · · · T m − , n ( T m , n x n + e m , n ) + e m − , n · · · (cid:1) + e , n (cid:17) + e , n x n + = x n + λ n ( y n − x n ) . (1.15) Suppose that S = \ n ∈ N Fix ( T , n · · · T m , n ) = ∅ (1.16) Variable metric algorithms driven by averaged operators. 5 and ( ∀ i ∈ { , . . . , m } ) ∑ n ∈ N λ n k e i , n k U − n < + ∞ , (1.17) and define ( ∀ i ∈ { , . . ., m } )( ∀ n ∈ N ) T i + , n = ( T i + , n · · · T m , n , if i = m ;Id , if i = m . (1.18) Then the following hold: (i) ∑ n ∈ N λ n ( / φ n − λ n ) k T , n · · · T m , n x n − x n k U − n < + ∞ . (ii) Suppose that ( ∀ n ∈ N ) λ n ∈ ] , ε + ( − ε ) / φ n ] . Then ( ∀ x ∈ S ) max i m ∑ n ∈ N λ n ( − α i , n ) α i , n k ( Id − T i , n ) T i + , n x n − ( Id − T i , n ) T i + , n x k U − n < + ∞ . (1.19)(iii) ( x n ) n ∈ N converges weakly to a point in S if and only if every weak sequentialcluster point of ( x n ) n ∈ N is in S. In this case, the convergence is strong if int S = ∅ . (iv) ( x n ) n ∈ N converges strongly to a point in S if and only if lim d S ( x n ) = .Proof . Let n ∈ N and let x ∈ S . Set T n = T , n · · · T m , n (1.20)and e n = y n − T n x n . (1.21)Using the nonexpansiveness on ( H , U − n ) of the operators ( T i , n ) i m , we first de-rive from (1.21) that k e n k U − n m ∑ i = k e i , n k U − n . (1.22)Let us rewrite (1.15) as x n + = x n + λ n (cid:0) T n x n + e n − x n (cid:1) , (1.23)and set R n = ( − / φ n ) Id +( / φ n ) T n and µ n = φ n λ n . (1.24)Then Fix R n = Fix T n and, by Lemmas 1 and 2, R n is nonexpansive on ( H , U − n ) .Furthermore, (1.23) can be written as x n + = x n + µ n (cid:0) R n x n − x n (cid:1) + λ n e n , where µ n ∈ ] , [ . (1.25)Now set z n = x n + µ n ( R n x n − x n ) . Since x ∈ Fix R n , we derive from [4, Corol-lary 2.14] that Lilian E. Glaudin k z n − x k U − n = ( − µ n ) k x n − x k U − n + µ n k R n x n − R n x k U − n − µ n ( − µ n ) k R n x n − x n k U − n (1.26) k x n − x k U − n − λ n ( / φ n − λ n ) k T n x n − x n k U − n . (1.27)Hence, (1.25), (1.14), and (1.27) yield k x n + − x k U − n + p + η n k z n − x k U − n + λ n p + η n k e n k U − n (1.28) p + η n k x n − x k U − n + λ n p + η n k e n k U − n (1.29)and, since ∑ k ∈ N λ k k e k k U k < + ∞ , it follows from Lemma 4 that ν = ∑ k ∈ N λ k k e k k U − k + k ∈ N k x k − x k U − k < + ∞ . (1.30)Moreover, using (1.28) and (1.27) we write ( + η n ) − k x n + − x k U − n + k z n − x k U − n + ( k z n − x k U − n + λ n k e n k U − n ) λ n k e n k U − n (1.31) k x n − x k U − n − λ n ( / φ n − λ n ) k T n x n − x n k U − n + νλ n k e n k U − n . (1.32)(i): This follows from (1.32), (1.20), (1.16), (1.30), and Lemma 3.(ii): We apply the definition of averageness of the operators ( T i , n ) i m to obtain k T n x n − x k U − n = k T , n · · · T m , n x n − T , n · · · T m , n x k U − n k x n − x k U − n − m ∑ i = − α i , n α i , n k ( Id − T i , n ) T i + , n x n − ( Id − T i , n ) T i + , n x k U − n . (1.33)Note also that λ n ε + − εφ n ⇒ ελ n ( ε − ) φ n ⇔ λ n − (cid:18) ε − (cid:19)(cid:18) φ n − λ n (cid:19) . (1.34)Thus (1.31), the definition of z n , and [4, Corollary 2.14] yield Variable metric algorithms driven by averaged operators. 7 ( + η n ) − k x n + − x k U − n + k ( − λ n )( x n − x ) + λ n ( T n x n − x ) k U − n + νλ n k e n k U − n = ( − λ n ) k x n − x k U − n + λ n k T n x n − x k U − n + λ n ( λ n − ) k T n x n − x n k U − n + νλ n k e n k U − n ( − λ n ) k x n − x k U − n + λ n k T n x n − x k U − n + ε n , (1.35)where ε n = λ n (cid:18) ε − (cid:19)(cid:18) α n − λ n (cid:19) k T n x n − x n k U − n + νλ n k e n k U − n . (1.36)Now set β n = λ n max i m (cid:18) − α i , n α i , n k ( Id − T i , n ) T i + , n x n − ( Id − T i , n ) T i + , n x k U − n (cid:19) . (1.37)On the one hand, it follows from (i), (1.30), and (1.16) that ∑ k ∈ N ε k < + ∞ . (1.38)On the other hand, combining (1.33) and (1.35), we obtain ( + η n ) − k x n + − x k U − n + k x n − x k U − n − β n + ε n . (1.39)Consequently, Lemma 3 implies that ∑ k ∈ N β k < + ∞ .(iii)–(iv): The results follow from (1.39), (1.38), and Proposition 1 for the weakconvergence, and Propositions 2 and 3 for the strong convergence. Remark 1.
Suppose that ( ∀ n ∈ N ) U n = Id and λ n ( − ε )( / φ n + ε ) . Then The-orem 1 reduces to [18, Theorem 3.5] which itself extends [9, Section 3] in the case ( ∀ n ∈ N ) λ n
1. As far as we know, it is the first inexact overrelaxed variable metricalgorithm based on averaged operators.
A special case of Theorem 1 of interest is the following.
Corollary 1.
Let α ∈ ] , + ∞ [ , let ( η n ) n ∈ N ∈ ℓ + ( N ) , and let ( U n ) n ∈ N be a sequencein P α ( H ) such that µ = sup n ∈ N k U n k < + ∞ and ( ∀ n ∈ N ) ( + η n ) U n + < U n . (1.40) Let ε ∈ ] , [ and let x ∈ H . For every n ∈ N , let α , n ∈ ] , / ( + ε )] , let α , n ∈ ] , / ( + ε )] , let T , n : H → H be α , n -averaged on ( H , U − n ) , let T , n : H → H Lilian E. Glaudin be α , n -averaged on ( H , U − n ) , let e , n ∈ H , and let e , n ∈ H . In addition, forevery n ∈ N , let λ n ∈ (cid:20) ε , ε + − εφ n (cid:21) , where φ n = α , n + α , n − α , n α , n − α , n α , n , (1.41) and iterate x n + = x n + λ n (cid:16) T , n (cid:0) T , n x n + e , n (cid:1) + e , n − x n (cid:17) . (1.42) Suppose thatS = \ n ∈ N Fix ( T , n T , n ) = ∅ , ∑ n ∈ N λ n k e , n k < + ∞ , and ∑ n ∈ N λ n k e , n k < + ∞ . (1.43) Then the following hold: (i) ∑ n ∈ N k T , n T , n x n − x n k < + ∞ . (ii) ( ∀ x ∈ S ) ∑ n ∈ N k T , n T , n x n − T , n x n + T , n x − x k < + ∞ . (iii) ( ∀ x ∈ S ) ∑ n ∈ N k T , n x n − x n − T , n x + x k < + ∞ . (iv) Suppose that every weak sequential cluster point of ( x n ) n ∈ N is in S. Then ( x n ) n ∈ N converges weakly to a point in S, and the convergence is strong if int S = ∅ . (v) ( x n ) n ∈ N converges strongly to a point in S if and only if lim d S ( x n ) = .Proof . For every n ∈ N ,1 √ µ k e , n k k e , n k U − n and 1 √ µ k e , n k k e , n k U − n , (1.44)and T , n T , n is φ n -averaged by [4, Proposition 4.44]. Thus, we apply Theorem 1 with m = ( ∀ x ∈ S ) ∑ n ∈ N λ n ( − α , n ) α , n k ( Id − T , n ) T , n x n − ( Id − T , n ) T , n x k U − n < + ∞ ∑ n ∈ N λ n ( − α , n ) α , n k ( Id − T , n ) x n − ( Id − T , n ) x k U − n < + ∞ ∑ n ∈ N λ n (cid:16) φ n − λ n (cid:17) k T , n T , n x n − x n k U − n < + ∞ . (1.45)However, we derive from the assumptions that Variable metric algorithms driven by averaged operators. 9 ( ∀ x ∈ S )( ∀ n ∈ N ) T , n T , n x = x λ n ( − α , n ) α , n > ε λ n ( − α , n ) α , n > ε λ n (cid:16) φ n − λ n (cid:17) > ε − φ n φ n > ε ε + . (1.46)Combining (1.40), (1.45) and (1.46) completes the proof.(iv)–(v): It follows from Theorem 1(iii)–(iv). Remark 2.
This corollary is a variable metric version of [18, Corollary 4.1] where ( ∀ n ∈ N ) U n = Id and λ n ( − ε )( / φ n + ε ) .We recall the definition of a demiregular operator. See [2] for examples ofdemiregular operators. Definition 2. [2, Definition 2.3] An operator A : H → H is demiregular at x ∈ dom A if, for every sequence (( x n , u n )) n ∈ N in gra A and every u ∈ Ax such that x n ⇀ x and u n → u , we have x n → x . Proposition 4.
Let α ∈ ] , + ∞ [ , let U ∈ P α ( H ) , let A : H → H be a maximallymonotone operator, let β ∈ ] , + ∞ [ , let γ ∈ ] , β / k U k ] , and let B a β -cocoerciveoperator. Then the following hold: (i) J γ UA is a / -averaged operator on ( H , U − ) . (ii) Id − γ UB is a γ k U k / ( β ) -averaged operator on ( H , U − ) .Proof . (i): [16, Lemma 3.7].(ii): We derive from (1.7) and Lemma 5(iii) that for every x ∈ H and for every y ∈ H h x − y | UBx − UBy i U − = h x − y | Bx − By i > β h Bx − By | Bx − By i = β h U − ( UBx − UBy ) | UBx − UBy i U − > k U k − β k UBx − UBy k U − . (1.47)Thus, for every x ∈ H and for every y ∈ H k ( x − γ UBx ) − ( y − γ UBy ) k U − = k x − y k U − + k γ UBx − γ UBy k U − − γ h x − y | UBx − UBy i U − (1.48) k x − y k U − − γ ( β / k U k − γ ) k UBx − UBy k U − , (1.49)which concludes the proof.Next, we introduce a new variable metric forward-backward splitting algorithm. Proposition 5.
Let β ∈ ] , + ∞ [ , let ε ∈ ] , min { / , β } [ , let α ∈ ] , + ∞ [ , let ( η n ) n ∈ N ∈ ℓ + ( N ) , and let ( U n ) n ∈ N be a sequence in P α ( H ) such that µ = sup n ∈ N k U n k < + ∞ and ( ∀ n ∈ N ) ( + η n ) U n + < U n . (1.50) Let x ∈ H , let A : H → H be maximally monotone, and let B : H → H be β -cocoercive. Furthermore, let ( a n ) n ∈ N and ( b n ) n ∈ N be sequences in H such that ∑ n ∈ N k a n k < + ∞ and ∑ n ∈ N k b n k < + ∞ . Suppose that zer ( A + B ) = ∅ and, for everyn ∈ N , let γ n ∈ (cid:20) ε , β ( + ε ) k U n k (cid:21) and λ n ∈ (cid:20) ε , + ( − ε ) (cid:18) − γ n k U n k β (cid:19)(cid:21) , (1.51) and iterate x n + = x n + λ n (cid:16) J γ n U n A (cid:0) x n − γ n U n ( Bx n + b n ) (cid:1) + a n − x n (cid:17) . (1.52) Then the following hold: (i) ∑ n ∈ N k J γ n U n A ( x n − γ n U n Bx n ) − x n k < + ∞ . (ii) Let x ∈ zer ( A + B ) . Then ∑ n ∈ N k Bx n − Bx k < + ∞ . (iii) ( x n ) n ∈ N converges weakly to a point in zer ( A + B ) . (iv) Suppose that one of the following holds: (a)
A is demiregular at every point in zer ( A + B ) . (b) B is demiregular at every point in zer ( A + B ) . (c) int S = ∅ .Then ( x n ) n ∈ N converges strongly to a point in zer ( A + B ) .Proof . We apply Corollary 1. Set ( ∀ n ∈ N ) T , n = J γ n U n A , T , n = Id − γ n U n B , e , n = a n , and e , n = − γ n U n b n . (1.53)Then, for every n ∈ N , T , n is α , n -averaged on ( H , U − n ) with α , n = / T , n is α , n -averaged on ( H , U − n ) with α , n = γ n k U n k / ( β ) by Proposition 4. Moreover,for every n ∈ N , φ n = α , n + α , n − α , n α , n − α , n α , n = β β − γ n k U n k (1.54)and, therefore, (1.51) yields λ n ∈ (cid:20) ε , ε + − εφ n (cid:21) . (1.55)Hence, we derive from (1.54) and (1.55) that ( ∀ n ∈ N ) λ n + ε . Consequently, Variable metric algorithms driven by averaged operators. 11 ( ∑ n ∈ N λ n k e , n k = ( + ε ) ∑ n ∈ N k a n k < + ∞∑ n ∈ N λ n k e , n k β ( + ε ) µα − ∑ n ∈ N k b n k < + ∞ . (1.56)Furthermore, it follows from [4, Proposition 26.1(iv)] that ( ∀ n ∈ N ) S = zer ( A + B ) = Fix ( T , n T , n ) = ∅ . (1.57)Hence, the assumptions of Corollary 1 are satisfied.(i): This is a consequence of Corollary 1(i) and (1.53).(ii): Corollary 1(ii), (1.53), and Lemma 5(iii) yield ∑ n ∈ N k Bx n − Bx k = ∑ n ∈ N γ − n k U − n ( T , n x n − x n − T , n x + x ) k ε α ∑ n ∈ N k T , n x n − x n − T , n x + x k < + ∞ . (1.58)(iii): Let ( k n ) n ∈ N be a strictly increasing sequence in N and let y ∈ H be suchthat x k n ⇀ y . In view of Corollary 1(iv), it remains to show that y ∈ zer ( A + B ) . Set ( ∀ n ∈ N ) y n = J γ n U n A ( x n − γ n U n Bx n ) u n = γ − n U − n ( x n − y n ) − Bx n v n = Bx n (1.59)and let z ∈ zer ( A + B ) . Hence, we derive from (i) that y n − x n →
0. Then y k n ⇀ y and by (ii) Bx n → Bz . Altogether, y k n ⇀ y , v k n ⇀ Bz , y k n − x k n → u k n + v k n →
0, and, for every n ∈ N , u k n ∈ Ay k n and v k n ∈ Bx k n . It therefore follows from [11,Lemma 4.5(ii)] that y ∈ zer ( A + B ) .(iv): The proof is the same that in [18, Proposition 4.4(iv)]. Remark 3.
Suppose that ( ∀ n ∈ N ) U n = Id and λ n ( − ε )( / φ n + ε ) . Then Propo-sition 5 captures [18, Proposition 4.4]. Now suppose that ( ∀ n ∈ N ) λ n
1. ThenProposition 5 captures [16, Theorem 4.1].Using the averaged operators framework allows us to obtain an extended forward-backward splitting algorithm in Euclidean spaces.
Example 1.
Let α ∈ ] , + ∞ [ , let ( η n ) n ∈ N ∈ ℓ + ( N ) , and let ( U n ) n ∈ N be a sequence in P α ( H ) such that µ = sup n ∈ N k U n k < + ∞ and ( ∀ n ∈ N ) ( + η n ) U n + < U n . (1.60)Let ε ∈ ] , / [ , let A : H → H be a maximally monotone operator, let β ∈ ] , + ∞ [ , let B a β -cocoercive operator, and let ( γ n ) n ∈ N and ( µ n ) n ∈ N be sequencesin [ ε , + ∞ [ such that φ n = µ n β β − k U n k γ n − ε . (1.61)Let x ∈ H and iterate ( ∀ n ∈ N ) x n + = x n + µ n (cid:16) J γ n U n A ( x n − γ n U n B ) − x n (cid:17) . (1.62)Suppose that H is finite-dimensional and that zer ( A + B ) = ∅ . Then ( x n ) n ∈ N con-verges to a point in zer ( A + B ) . Proof . Set ( ∀ n ∈ N ) T n = Id + µ n ( J γ n U n A ( Id − γ n U n B ) − Id ) . Remark that, for every n ∈ N , T n is φ n -averaged. Hence we apply Theorem 1 with m = λ ≡ T n x n − x n →
0. Since H is finite-dimensional, the claim follows from Theorem 1(iii). Remark 4.
An underrelaxation or an appropriate choice of the metric of the algo-rithm allows us to exceed the classical bound 2 / β for ( γ n ) n ∈ N . For instance, theparameters γ n ≡ . / β , µ n ≡ /
2, and U n ≡ Id satisfy the assumptions.
We study the composite monotone inclusion presented in [14].
Problem 1.
Let z ∈ H , let A : H → H be maximally monotone, let µ ∈ ] , + ∞ [ ,let C : H → H be µ -cocoercive, and let m be a strictly positive integer. For every i ∈ { , . . . , m } , let r i ∈ G i , let B i : G i → G i be maximally monotone, let ν i ∈ ] , + ∞ [ ,let D i : G i → G i be maximally monotone and ν i -strongly monotone, and supposethat 0 = L i ∈ B ( H , G i ) . The problem is to find x ∈ H such that z ∈ Ax + m ∑ i = L ∗ i (cid:0) ( B i (cid:3) D i )( L i x − r i ) (cid:1) + Cx , (1.63)the dual problem of which is to find v ∈ G , . . . , v m ∈ G m such that ( ∃ x ∈ H ) ( z − ∑ mi = L ∗ i v i ∈ Ax + Cx ( ∀ i ∈ { , . . . , m } ) v i ∈ ( B i (cid:3) D i )( L i x − r i ) . (1.64)The following corollary is an overrelaxed version of [16, Corollary 6.2]. Corollary 2.
In Problem 1, suppose thatz ∈ ran (cid:18) A + m ∑ i = L ∗ i (cid:0) ( B i (cid:3) D i )( L i · − r i ) (cid:1) + C (cid:19) , (1.65) and set β = min { µ , ν , . . . , ν m } . (1.66) Variable metric algorithms driven by averaged operators. 13
Let ε ∈ ] , min { , β } [ , let α ∈ ] , + ∞ [ , let ( λ n ) n ∈ N be a sequence in ] , + ∞ [ , letx ∈ H , let ( a n ) n ∈ N and ( c n ) n ∈ N be absolutely summable sequences in H , andlet ( U n ) n ∈ N be a sequence in P α ( H ) such that ( ∀ n ∈ N ) U n + < U n . For everyi ∈ { , . . . , m } , let v i , ∈ G i , and let ( b i , n ) n ∈ N and ( d i , n ) n ∈ N be absolutely summablesequences in G i , and let ( U i , n ) n ∈ N be a sequence in P α ( G i ) such that ( ∀ n ∈ N ) U i , n + < U i , n . For every n ∈ N , set δ n = s m ∑ i = k p U i , n L i √ U n k ! − − , (1.67) suppose that ζ n = + δ n ( + δ n ) max {k U n k , k U , n k , . . . , k U m , n k} > β − ε , (1.68) and let λ n ∈ (cid:20) ε , + ( − ε ) (cid:18) − ζ n β (cid:19)(cid:21) . (1.69) Iteratefor n = , , . . . p n = J U n A (cid:16) x n − U n (cid:0) ∑ mi = L ∗ i v i , n + Cx n + c n − z (cid:1)(cid:17) + a n y n = p n − x n x n + = x n + λ n ( p n − x n ) for i = , . . . , m $ q i , n = J U i , n B − i (cid:16) v i , n + U i , n (cid:0) L i y n − D − i v i , n − d i , n − r i (cid:1)(cid:17) + b i , n v i , n + = v i , n + λ n ( q i , n − v i , n ) . (1.70) Then the following hold for some solution x to (1.63) and some solution ( v , . . . , v m ) to (1.64) : (i) x n ⇀ x. (ii) ( ∀ i ∈ { , . . . , m } ) v i , n ⇀ v i . (iii) Suppose that C is demiregular at x. Then x n → x. (iv) Suppose that, for some j ∈ { , . . ., m } , D − j is demiregular at v j . Then v j , n → v j .Proof . Set G = G ⊕ · · · ⊕ G m , K = H ⊕ G , and e A : K → K : ( x , v , . . . , v m ) ( ∑ mi = L ∗ i v i − z + Ax ) × ( r − L x + B − v ) × · · · × ( r m − L m x + B − m v m ) e B : K → K : ( x , v , . . . , v m ) (cid:0) Cx , D − v , . . . , D − m v m (cid:1)e S : K → K : ( x , v , . . . , v m ) (cid:18) ∑ mi = L ∗ i v i , − L x , . . . , − L m x (cid:19) . (1.71)Now, for every n ∈ N , define e U n : K → K : ( x , v , . . . , v m ) (cid:16) U n x , U , n v , . . . , U m , n v m (cid:17)e V n : K → K : ( x , v , . . . , v m ) (cid:18) U − n x − ∑ mi = L ∗ i v i , (cid:0) − L i x + U − i , n v i (cid:1) i m (cid:19) (1.72)and e x n = ( x n , v , n , . . . , v m , n ) e y n = ( p n , q , n , . . . , q m , n ) e a n = ( a n , b , n , . . . , b m , n ) e c n = ( c n , d , n , . . . , d m , n ) e d n = ( U − n a n , U − , n b , n , . . . , U − m , n b m , n ) and e b n = ( e S + e V n ) e a n + e c n − e d n . (1.73)It follows from the proof of [16, Corollary 6.2] that (1.70) is equivalent to ( ∀ n ∈ N ) e x n + = e x n + λ n (cid:16) J e V − n e A (cid:0)e x n − e V − n ( e B e x n + e b n ) (cid:1) + e a n − e x n (cid:17) , (1.74)that the operators e A and e B are maximally monotone, and e B is β -cocoercive on H .Furthermore, for every ( x , v ) ∈ zer ( e A + e B ) , x solves (1.63) and v solves (1.64). Nowset ρ = / α + p ∑ mi = k L i k . We deduce from the proof of [16, Corollary 6.2] that ( ∀ n ∈ N ) k e V − n k ζ − n β − ε and e V − n + < e V − n ∈ P / ρ ( K ) . We observe that(1.74) has the structure of the variable metric forward-backward splitting algorithm(1.52) and that all the conditions of Proposition 5 are satisfied.(i)&(ii): Proposition 5(iii) asserts that there exists e x = ( x , v , . . . , v m ) ∈ zer ( e A + e B ) (1.75)such that e x n ⇀ e x .(iii)&(iv): It follows from Proposition 5(ii) that e B e x n → e B e x . Hence, (1.71), (1.73),and (1.75) yield Cx n → Cx and (cid:0) ∀ i ∈ { , . . . , m } (cid:1) D − i v i , n → D − i v i . (1.76)We derive the results from Definition 2 and (i)–(ii) above. Remark 5.
Suppose that ( ∀ n ∈ N ) λ n
1. Then Corollary 2 captures [15, Corol-lary 6.2].
Acknowledgement.
The author thanks his Ph.D. advisor P. L. Combettes for hisguidance during this work, which is part of his Ph.D. dissertation.
Variable metric algorithms driven by averaged operators. 15
References
1. Alotaibi, A., Combettes, P.L., and Shahzad, N.: Solving coupled composite monotone inclu-sions by successive Fejér approximations of their Kuhn-Tucker set. SIAM J. Optim. , 2076–2095 (2014)2. Attouch, H., Briceño-Arias, L.M., and Combettes, P.L.: A parallel splitting method for coupledmonotone inclusions. SIAM J. Control Optim. , 3246–3270 (2010)3. Baillon, J.-B., Bruck, R.E., and Reich, S.: On the asymptotic behavior of nonexpansive map-pings and semigroups, Houston J. Math., , 1–9 (1978)4. Bauschke, H.H. and Combettes, P.L.: Convex Analysis and Monotone Operator Theory inHilbert Spaces.
Opérateurs Maximaux Monotones et Semi-groupes de Contractions dans les Es-paces de Hilbert.
North-Holland/Elsevier, New York (1973)6. Briceño-Arias, L.M. and Combettes, P.L.: A monotone + skew splitting model for compositemonotone inclusions in duality. SIAM J. Optim. , 1230–1250 (2011)7. Cegielski, A., Iterative Methods for Fixed Point Problems in Hilbert Spaces,
Lecture Notes inMathematics, . Springer, Heidelberg (2012)8. Chambolle, A. and Pock, T.: A first-order primal-dual algorithm for convex problems withapplications to imaging. J. Math. Imaging Vision , 120–145 (2011)9. Combettes, P.L.: Solving monotone inclusions via compositions of nonexpansive averagedoperators. Optimization , 475–504 (2004)10. Combettes, P.L.: Monotoned operator theory in convex optimization. Math. Program. ,177–206 (2018)11. Combettes, P.L. and Glaudin, L.E.: Quasi-nonexpansive iterations on the affine hull of orbits:from Mann’s mean value algorithm to inertial methods. SIAM J. Optim. , 2356–2380 (2017)12. Combettes, P.L. and Müller, C.L.: Perspective functions: proximal calculus and applicationsin high-dimensional statistics. J. Math. Anal. Appl. , 1283–1306 (2018)13. Combettes, P.L. and Pesquet, J.-C.: Proximal splitting methods in signal processing, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, 185–212. Springer, NewYork (2011)14. Combettes, P.L. and Pesquet, J.-C.: Primal-dual splitting algorithm for solving inclusions withmixtures of composite, Lipschitzian, and parallel-sum type monotone operators. Set-ValuedVar. Anal. , 307–330 (2012)15. Combettes, P.L. and V˜u, B.C: Variable metric quasi-Fejér monotonicity. Nonlinear Anal. ,17–31 (2013)16. Combettes, P.L. and V˜u, B.C.: Variable metric forward-backward splitting with applicationsto monotone inclusions in duality. Optimization , 1289–1318 (2014)17. Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting., Multi-scale Model. Simul. , 1168–1200 (2005)18. Combettes, P.L. and Yamada, I.: Compositions and convex combinations of averaged nonex-pansive operators. J. Math. Anal. Appl. , 55–70 (2015)19. Duchi, J. and Singer, Y.: Efficient online and batch learning using forward backward splitting.J. Mach. Learn. Res. , 2899–2934 (2009)20. Jenatton, R., Mairal, J., Obozinski, G., and Bach, F.: Proximal methods for hierarchical sparsecoding. J. Mach. Learn. Res. , 2297–2334 (2011)21. Peypouquet, J. and Sorin, S.: Evolution equations for maximal monotone operators: asymp-totic analysis in continuous and discrete time. J. Convex Anal. , 1113–1163 (2010)22. Salzo, S.: The variable metric forward-backward splitting algorithm under mild differentiabil-ity assumptions. SIAM J. Optim. , 2153–2181 (2017)23. Showalter, R.E.: Monotone Operators in Banach Space and Nonlinear Partial DifferentialEquations.
American Mathematical Society, Providence, RI (1997)24. V˜u, B.C.: A splitting algorithm for dual monotone inclusions involving cocoercive operators.Adv. Comput. Math.38