A Warped Resolvent Algorithm to Construct Nash Equilibria
aa r X i v : . [ m a t h . O C ] J a n A Warped Resolvent Algorithmto Construct Nash Equilibria * Minh N. B`ui and Patrick L. Combettes
North Carolina State University, Department of Mathematics, Raleigh, NC 27695-8205, USA [email protected] and [email protected]
Abstract . We propose an asynchronous block-iterative decomposition algorithm to solve Nash equi-librium problems involving a mix of nonsmooth and smooth functions acting on linear mixtures ofstrategies. The methodology relies heavily on monotone operator theory and in particular on warpedresolvents.
We consider a noncooperative game with m players indexed by I = { , . . . , m } , in which the strategy x i of player i ∈ I lies in a real Hilbert space H i . A strategy profile is a point x = ( x i ) i ∈ I in the Hilbertdirect sum H = L i ∈ I H i , and the associated profile of the players other than i ∈ I is the vector x r i = ( x j ) j ∈ I r { i } in H r i = L j ∈ I r { i } H j . Given an index i ∈ I and a vector ( x i , y ) ∈ H i × H , we set ( x i ; y r i ) = ( y , . . . , y i − , x i , y i +1 , . . . , y m ) .A fundamental equilibrium notion was introduced by Nash in [20, 21] to describe a state in whichthe loss of each player cannot be reduced by unilateral deviation. In our context, a formulation of theNash equilibrium problem isfind x ∈ H such that ( ∀ i ∈ I ) x i ∈ Argmin x i ∈H i θ i ( x i ) + ℓ i ( x i ; x r i ) , (1.1)where the global loss function of player i ∈ I is the sum of an individual loss θ i : H i → ] −∞ , + ∞ ] anda joint loss ℓ i : H → ] −∞ , + ∞ ] that models the interactions with the other players. Under convexityassumptions, numerical methods to solve (1.1) have been investigated since the early 1970s [4] andthey have since involved increasingly sophisticated tools from nonlinear analysis; see [1, 5, 8, 11,14, 15, 16, 17, 18, 19, 25]. In the present paper, we consider the following highly modular Nashequilibrium problem wherein the functions ( θ i ) i ∈ I and ( ℓ i ) i ∈ I of (1.1) are decomposed into elementarycomponents that are easier to process numerically. * Contact author: P. L. Combettes. Email: [email protected] . Phone: +1 919 515 2671. This work was supported bythe National Science Foundation under grant DMS-1818946. roblem 1.1 Let ( H i ) i ∈ I , ( K i ) i ∈ I , and ( G k ) k ∈ K be finite families of real Hilbert spaces, and set H = L i ∈ I H i , K = L i ∈ I K i , and G = L k ∈ K G k . Suppose that the following are satisfied:[a] For every i ∈ I , ϕ i : H i → ] −∞ , + ∞ ] is proper, lower semicontinuous, and convex, α i ∈ [0 , + ∞ [ ,and ψ i : H i → R is convex and differentiable with an α i -Lipschitzian gradient.[b] For every i ∈ I , f i : K → R is such that, for every y ∈ K , f i ( · ; y r i ) : K i → R is convex andGˆateaux differentiable, and we denote its gradient at y i ∈ K i by ∇ i f i ( y ) . Further, the opera-tor Q : K → K : y ( ∇ i f i ( y )) i ∈ I is monotone and Lipschitzian. Finally, ( χ i ) i ∈ I are positivenumbers such that ( ∀ y ∈ K )( ∀ y ′ ∈ K ) h y − y ′ | Qy − Qy ′ i X i ∈ I χ i k y i − y ′ i k . (1.2)[c] For every k ∈ K , g k : G k → ] −∞ , + ∞ ] is proper, lower semicontinuous, and convex, β k ∈ [0 , + ∞ [ , and h k : G k → R is convex and differentiable with a β k -Lipschitzian gradient.[d] For every i ∈ I and every k ∈ K , M i : H i → K i and L k,i : H i → G k are linear and bounded, and,for every x ∈ H , we write L k, r i x r i = P j ∈ I r { i } L k,j x j and M x = ( M j x j ) j ∈ I .The goal is tofind x ∈ H such that ( ∀ i ∈ I ) x i ∈ Argmin x i ∈H i ϕ i ( x i ) + ψ i ( x i ) + f i (cid:0) M i x i ; ( M x ) r i (cid:1) + X k ∈ K ( g k + h k )( L k,i x i + L k, r i x r i ) . (1.3)In Problem 1.1, the individual loss of player i ∈ I consists of a nonsmooth component ϕ i and asmooth component ψ i , while his joint loss is decomposed into a smooth function f i and a sum of non-smooth functions ( g k ) k ∈ K and smooth functions ( h k ) k ∈ K acting on linear mixtures of the strategies.We aim at solving (1.3) with a numerical procedure that can be implemented in a flexible fashion andthat is able to cope with possibly very large scale problems. This leads us to adopt the following designprinciples:• Decomposition:
Each function and each linear operator in Problem 1.1 is activated separately.•
Block-iterative implementation:
Only a subgroup of functions needs to be activated at anyiteration. This makes it possible to best modulate and adapt the computational load of eachiteration in large-scale problems.•
Asynchronous implementation:
The computations are asynchronous in the sense that the re-sult of calculations initiated at earlier iterations can be incorporated at the current one.Our methodology is to first transform (1.3) into a system of monotone set-valued inclusions andthen approach it via monotone operator splitting techniques. Since no splitting technique tailored to(1.3) and compliant with the above principles appears to be available, we adopt a fresh perspectivehinging on the theory of warped resolvents [9]. In Section 2 we provide the necessary notation andbackground on monotone operator theory. Section 3 is devoted to the derivation of the proposedasynchronous block-iterative algorithm to solve Problem 1.1. Application examples are provided inSection 4. 2
Notation and background
General background on monotone operators and related notions can be found in [3].Let H be a real Hilbert space. We denote by H the power set of H and by Id the identity operatoron H . The weak convergence and the strong convergence of a sequence ( x n ) n ∈ N in H to a point x in H are denoted by x n ⇀ x and x n → x , respectively. Let A : H → H . The domain of A isdom A = (cid:8) x ∈ H | Ax = ∅ (cid:9) , the range of A is ran A = S x ∈ dom A Ax , the graph of A is gra A = (cid:8) ( x, x ∗ ) ∈ H × H | x ∗ ∈ Ax (cid:9) , the set of zeros of A is zer A = (cid:8) x ∈ H | ∈ Ax (cid:9) , and the inverse of A is A − : H → H : x ∗ (cid:8) x ∈ H | x ∗ ∈ Ax (cid:9) . Now suppose that A is monotone, that is, (cid:0) ∀ ( x, x ∗ ) ∈ gra A (cid:1)(cid:0) ∀ ( y, y ∗ ) ∈ gra A (cid:1) h x − y | x ∗ − y ∗ i > . (2.1)Then A is maximally monotone if, for every monotone operator e A : H → H , gra A ⊂ gra e A ⇒ A = e A ; A is strongly monotone with constant ς ∈ ]0 , + ∞ [ if A − ς Id is monotone; and A is ∗ monotone if ( ∀ x ∈ dom A )( ∀ x ∗ ∈ ran A ) sup ( y,y ∗ ) ∈ gra A h x − y | y ∗ − x ∗ i < + ∞ . (2.2) Γ ( H ) is the set of lower semicontinuous convex functions ϕ : H → ] −∞ , + ∞ ] which are proper inthe sense that dom ϕ = (cid:8) x ∈ H | ϕ ( x ) < + ∞ (cid:9) = ∅ . Let ϕ ∈ Γ ( H ) . Then ϕ is supercoercive if lim k x k→ + ∞ ϕ ( x ) / k x k = + ∞ and uniformly convex if there exists an increasing function φ : [0 , + ∞ [ → [0 , + ∞ ] that vanishes only at such that ( ∀ x ∈ dom ϕ )( ∀ y ∈ dom ϕ )( ∀ α ∈ ]0 , ϕ (cid:0) αx + (1 − α ) y (cid:1) + α (1 − α ) φ (cid:0) k x − y k (cid:1) αϕ ( x ) + (1 − α ) ϕ ( y ) . (2.3)For every x ∈ H , prox ϕ x denotes the unique minimizer of ϕ + (1 / k · − x k . The subdifferential of ϕ isthe maximally monotone operator ∂ϕ : H → H : x (cid:8) x ∗ ∈ H | ( ∀ y ∈ H ) h y − x | x ∗ i + ϕ ( x ) ϕ ( y ) (cid:9) .Let C be a convex subset of H . The indicator function of C is ι C : H → [0 , + ∞ ] : x ( , if x ∈ C ;+ ∞ , otherwise , (2.4)and the strong relative interior of C issri C = x ∈ C (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12) [ λ ∈ ]0 , + ∞ [ λ ( C − x ) is a closed vector subspace of H . (2.5)The following notion of a warped resolvent will be instrumental to our approach. Definition 2.1 ([9])
Suppose that X is a real Hilbert space. Let D be a nonempty subset of X , let K : D → X , and let A : X → X be such that ran K ⊂ ran ( K + A ) and K + A is injective. The warpedresolvent of A with kernel K is J KA = ( K + A ) − ◦ K .We now provide a warped resolvent algorithm to find a zero of a maximally monotone operator A : X → X , where X is a real Hilbert space. This algorithm has a simple geometric interpretation:3t iteration n , we use the evaluation of the warped resolvent J K n A at a perturbation e x n of the currentiterate x n to construct a point ( y n , y ∗ n ) ∈ gra A . By monotonicity of A ,zer A ⊂ H n = (cid:8) z ∈ X | h z − y n | y ∗ n i (cid:9) , (2.6)and the update x n +1 is a relaxed projection of x n onto the half-space H n . Proposition 2.2
Let X be a real Hilbert space and let A : X → X be a maximally monotone operatorsuch that zer A = ∅ . Let x ∈ X , let ε ∈ ]0 , , let ς ∈ ]0 , + ∞ [ , and let ̟ ∈ ] ς, + ∞ [ . Further, for every n ∈ N , let λ n ∈ [ ε, − ε ] , let e x n ∈ X , and let K n : X → X be a ς -strongly monotone and ̟ -Lipschitzianoperator. Iteratefor n = 0 , , . . . y n = J K n A e x n ; y ∗ n = K n e x n − K n y n ; if h y n − x n | y ∗ n i < (cid:22) x n +1 = x n + λ n h y n − x n | y ∗ n ik y ∗ n k y ∗ n ; else (cid:4) x n +1 = x n . (2.7) Then the following hold: (i) P n ∈ N k x n +1 − x n k < + ∞ . (ii) Suppose that e x n − x n → . Then x n − y n → and ( x n ) n ∈ N converges weakly to a point in zer A .Proof . It follows from [9, Proposition 3.9(i)[d]&(ii)[b]] that the warped resolvents ( J K n A ) n ∈ N in (2.7)are well defined. In turn, we derive (i) and the weak convergence claim from [9, Theorem 4.2 andRemark 4.3]. It thus remains to prove that x n − y n → . It is shown in the proof of [9, Theorem 4.2(ii)]that K n e x n − K n y n → . At the same time, for every n ∈ N , every x ∈ X , and every y ∈ X , we deducefrom the Cauchy–Schwarz inequality that ς k x − y k h x − y | K n x − K n y i k x − y k k K n x − K n y k ,from which it follows that ς k x − y k k K n x − K n y k . (2.8)Therefore, k x n − y n k k x n − e x n k + k e x n − y n k k x n − e x n k + (1 /ς ) k K n e x n − K n y n k → , as desired. As mentioned in Section 1, there exists no method tailored to the format of Problem 1.1 that can solveit in an asynchronous block-iterative fashion. Our methodology to design such an algorithm can bebroken down in the following steps: We rephrase (1.3) as a monotone inclusion problem in H , namely,find x ∈ H such that ∈ Ax + M ∗ (cid:0) Q ( M x ) (cid:1) + L ∗ (cid:0) B ( Lx ) (cid:1) , (3.1)4here Q and M are defined in Problem 1.1[b] and Problem 1.1[d], respectively, and A : H → H : x × i ∈ I (cid:0) ∂ϕ i ( x i ) + ∇ ψ i ( x i ) (cid:1) B : G → G : z × k ∈ K (cid:0) ∂g k ( z k ) + ∇ h k ( z k ) (cid:1) L : H → G : x (cid:0) P i ∈ I L k,i x i (cid:1) k ∈ K . (3.2) The inclusion in (3.1) involves more than two operators, namely A , B , Q , L , and M . Hence, inthe spirit of the decomposition methodologies of [9, 12, 13], a space bigger than H is requiredto devise a splitting method to solve it. We set X = H ⊕ K ⊕ G ⊕ K ⊕ G and consider the inclusionproblemfind x ∈ X such that ∈ Ax , (3.3)where A : X → X : ( x , y , z , u ∗ , v ∗ ) ( Ax + M ∗ u ∗ + L ∗ v ∗ ) × { Qy − u ∗ } × ( Bz − v ∗ ) × { y − M x } × { z − Lx } . (3.4) We show that, if x = ( x , y , z , u ∗ , v ∗ ) solves (3.3), then x solves (3.1) and, therefore, (1.3). To solve (3.3), we implement the warped resolvent algorithm of Proposition 2.2 with a specificchoice of the auxiliary points ( e x n ) n ∈ N and the kernels ( K n ) n ∈ N that will lead to an asynchronousblock-iterative splitting algorithm.The methodology just described is put in motion in our main theorem, which we now state andprove. Theorem 3.1
Consider the setting of Problem . Let η ∈ ]0 , + ∞ [ and ε ∈ ]0 , be such that /ε > max { α i + η, β k + η, χ i + η } i ∈ I,k ∈ K , let ( λ n ) n ∈ N be in [ ε, − ε ] , and let D ∈ N . Suppose that the followingare satisfied: [a] For every i ∈ I and every n ∈ N , τ i ( n ) ∈ N satisfies n − D τ i ( n ) n , γ i,n ∈ [ ε, / ( α i + η )] , µ i,n ∈ [ ε, / ( χ i + η )] , σ i,n ∈ [ ε, /ε ] , x i, ∈ H i , y i, ∈ K i , and u ∗ i, ∈ K i . [b] For every k ∈ K and every n ∈ N , δ k ( n ) ∈ N satisfies n − D δ k ( n ) n , ν k,n ∈ [ ε, / ( β k + η )] , ̺ k,n ∈ [ ε, /ε ] , z k, ∈ G k , and v ∗ k, ∈ G k . [c] ( I n ) n ∈ N are nonempty subsets of I and ( K n ) n ∈ N are nonempty subsets of K such that, for some P ∈ N , I = I, K = K, and ( ∀ n ∈ N ) n + P [ j = n I j = I and n + P [ j = n K j = K. (3.5)5 terate for n = 0 , , . . . for every i ∈ I n q i,n = y i,τ i ( n ) + µ i,τ i ( n ) (cid:0) u ∗ i,τ i ( n ) − ∇ i f i (cid:0) y τ i ( n ) (cid:1)(cid:1) ; c ∗ i,n = u ∗ i,τ i ( n ) + σ i,τ i ( n ) (cid:0) M i x i,τ i ( n ) − y i,τ i ( n ) (cid:1) ; x ∗ i,n = x i,τ i ( n ) − γ i,τ i ( n ) (cid:0) ∇ ψ i (cid:0) x i,τ i ( n ) (cid:1) + M ∗ i u ∗ i,τ i ( n ) + P k ∈ K L ∗ k,i v ∗ k,τ i ( n ) (cid:1) ; a i,n = prox γ i,τi ( n ) ϕ i x ∗ i,n ; s ∗ i,n = γ − i,τ i ( n ) ( x ∗ i,n − a i,n ) + ∇ ψ i ( a i,n ) + M ∗ i c ∗ i,n ; c i,n = q i,n − M i a i,n ; for every i ∈ I r I n (cid:4) q i,n = q i,n − ; c ∗ i,n = c ∗ i,n − ; a i,n = a i,n − ; s ∗ i,n = s ∗ i,n − ; c i,n = c i,n − ; for every k ∈ K n d ∗ k,n = z k,δ k ( n ) + ν k,δ k ( n ) (cid:0) v ∗ k,δ k ( n ) − ∇ h k (cid:0) z k,δ k ( n ) (cid:1)(cid:1) ; b k,n = prox ν k,δk ( n ) g k d ∗ k,n ; e ∗ k,n = v ∗ k,δ k ( n ) + ̺ k,δ k ( n ) (cid:0) P i ∈ I L k,i x i,δ k ( n ) − z k,δ k ( n ) (cid:1) ; b ∗ k,n = ν − k,δ k ( n ) ( d ∗ k,n − b k,n ) + ∇ h k ( b k,n ) − e ∗ k,n ; e k,n = b k,n − P i ∈ I L k,i a i,n ; for every k ∈ K r K n (cid:4) b k,n = b k,n − ; e ∗ k,n = e ∗ k,n − ; b ∗ k,n = b ∗ k,n − ; e k,n = b k,n − P i ∈ I L k,i a i,n ; for every i ∈ I (cid:22) a ∗ i,n = s ∗ i,n + P k ∈ K L ∗ k,i e ∗ k,n ; q ∗ i,n = ∇ i f i ( q n ) − c ∗ i,n ; π n = P i ∈ I (cid:0) h a i,n − x i,n | a ∗ i,n i + h q i,n − y i,n | q ∗ i,n i + h c i,n | c ∗ i,n − u ∗ i,n i (cid:1) + P k ∈ K (cid:0) h b k,n − z k,n | b ∗ k,n i + h e k,n | e ∗ k,n − v ∗ k,n i (cid:1) ; if π n < θ n = λ n π n / (cid:0) P i ∈ I (cid:0) k a ∗ i,n k + k q ∗ i,n k + k c i,n k (cid:1) + P k ∈ K (cid:0) k b ∗ k,n k + k e k,n k (cid:1)(cid:1) ; for every i ∈ I (cid:4) x i,n +1 = x i,n + θ n a ∗ i,n ; y i,n +1 = y i,n + θ n q ∗ i,n ; u ∗ i,n +1 = u ∗ i,n + θ n c i,n ; for every k ∈ K (cid:4) z k,n +1 = z k,n + θ n b ∗ k,n ; v ∗ k,n +1 = v ∗ k,n + θ n e k,n ; else for every i ∈ I (cid:4) x i,n +1 = x i,n ; y i,n +1 = y i,n ; u ∗ i,n +1 = u ∗ i,n ; for every k ∈ K (cid:4) z k,n +1 = z k,n ; v ∗ k,n +1 = v ∗ k,n . (3.6) Furthermore, suppose that there exist b x ∈ H , b u ∗ ∈ K , and b v ∗ ∈ G such that ( ∀ i ∈ I ) b u ∗ i = ∇ i f i ( M b x )( ∀ k ∈ K ) b v ∗ k ∈ ( ∂g k + ∇ h k ) (cid:0) P j ∈ I L k,j b x j (cid:1) ( ∀ i ∈ I ) − M ∗ i b u ∗ i − P k ∈ K L ∗ k,i b v ∗ k ∈ ∂ϕ i ( b x i ) + ∇ ψ i ( b x i ) . (3.7) Then ( x n ) n ∈ N converges weakly to a solution to Problem . roof . Set X = H ⊕ K ⊕ G ⊕ K ⊕ G and consider the operators defined in (3.2) and (3.4). Let us firstexamine some properties of the operator A in (3.4). For every i ∈ I , it results from Problem 1.1[a] and[3, Theorem 20.25 and Proposition 17.31(i)] that ∂ϕ i and ∇ ψ i are maximally monotone and, there-fore, from [3, Corollary 25.5(i)] that ∂ϕ i + ∇ ψ i is maximally monotone. Thus, in view of (3.2) and [3,Proposition 20.23], A is maximally monotone. Likewise, B is maximally monotone. Hence, since Q is maximally monotone by virtue of Problem 1.1[b] and [3, Corollary 20.28], [3, Proposition 20.23]implies that the operator R : X → X : ( x , y , z , u ∗ , v ∗ ) Ax × { Qy } × Bz × { } × { } (3.8)is maximally monotone. On the other hand, since the operator S : X → X : ( x , y , z , u ∗ , v ∗ ) ( M ∗ u ∗ + L ∗ v ∗ ) × {− u ∗ } × {− v ∗ } × { y − M x } × { z − Lx } (3.9)is linear and bounded with S ∗ = − S , (3.10)we deduce from [3, Example 20.35] that S is maximally monotone. In turn, it follows from (3.4),(3.8), and [3, Corollary 25.5(i)] that A = R + S is maximally monotone . (3.11)Upon setting b y = M b x and b z = L b x , we derive from (3.7) and (3.2) that b u ∗ = Q b y and b v ∗ ∈ B b z .Further, since M ∗ : K → H : u ∗ ( M ∗ i u ∗ i ) i ∈ I and L ∗ : G → H : v ∗ X k ∈ K L ∗ k,i v ∗ k ! i ∈ I , (3.12)it results from (3.7) and (3.2) that − M ∗ b u ∗ − L ∗ b v ∗ ∈ A b x . Therefore, we infer from (3.4) that ( b x , b y , b z , b u ∗ , b v ∗ ) ∈ zer A and, hence, thatzer A = ∅ . (3.13)Define ( ∀ i ∈ I )( ∀ n ∈ N ) ℓ i ( n ) = max (cid:8) j ∈ N | j n and i ∈ I j (cid:9) and ℓ i ( n ) = τ i (cid:0) ℓ i ( n ) (cid:1) (3.14)and ( ∀ k ∈ K )( ∀ n ∈ N ) ϑ k ( n ) = max (cid:8) j ∈ N | j n and k ∈ K j (cid:9) and ϑ k ( n ) = δ k (cid:0) ϑ k ( n ) (cid:1) . (3.15)In addition, let κ ∈ ]0 , + ∞ [ be a Lipschitz constant of Q in Problem 1.1[b], set ( α = q (cid:0) ε − + max i ∈ I α i (cid:1) , β = q (cid:0) ε − + max k ∈ K β k (cid:1) , χ = p ε − + κ ) ς = min { ε, η } , ̟ = k S k + max { α, β, χ, /ε } , (3.16)7nd define ( ∀ n ∈ N ) E n : H → H : x (cid:0) γ − i,ℓ i ( n ) x i − ∇ ψ i ( x i ) (cid:1) i ∈ I F n : K → K : y (cid:0) µ − i,ℓ i ( n ) y i − ∇ i f i ( y ) (cid:1) i ∈ I G n : G → G : z (cid:0) ν − k,ϑ k ( n ) z k − ∇ h k ( z k ) (cid:1) k ∈ K T n : X → X : ( x , y , z , u ∗ , v ∗ ) (cid:16) E n x , F n y , G n z , (cid:0) σ − i,ℓ i ( n ) u ∗ i (cid:1) i ∈ I , (cid:0) ̺ − k,ϑ k ( n ) v ∗ k (cid:1) k ∈ K (cid:17) K n = T n − S . (3.17)Fix temporarily n ∈ N . Then, using [a], the Cauchy–Schwarz inequality, and Problem 1.1[a], weobtain ( ∀ x ∈ H )( ∀ x ′ ∈ H ) h x − x ′ | E n x − E n x ′ i = X i ∈ I (cid:0) γ − i,ℓ i ( n ) k x i − x ′ i k − h x i − x ′ i | ∇ ψ i ( x i ) − ∇ ψ i ( x ′ i ) i (cid:1) > X i ∈ I (cid:0) ( α i + η ) k x i − x ′ i k − k x i − x ′ i k k∇ ψ i ( x i ) − ∇ ψ i ( x ′ i ) k (cid:1) > X i ∈ I (cid:0) ( α i + η ) k x i − x ′ i k − α i k x i − x ′ i k (cid:1) = η k x − x ′ k (3.18)and ( ∀ x ∈ H )( ∀ x ′ ∈ H ) k E n x − E n x ′ k = X i ∈ I (cid:13)(cid:13) γ − i,ℓ i ( n ) ( x i − x ′ i ) − (cid:0) ∇ ψ i ( x i ) − ∇ ψ i ( x ′ i ) (cid:1)(cid:13)(cid:13) X i ∈ I (cid:0) γ − i,ℓ i ( n ) k x i − x ′ i k + k∇ ψ i ( x i ) − ∇ ψ i ( x ′ i ) k (cid:1) X i ∈ I (cid:0) ε − k x i − x ′ i k + α i k x i − x ′ i k (cid:1) α k x − x ′ k . (3.19)Thus, E n is η -strongly monotone and α -Lipschitzian . (3.20)Similarly, ( F n is η -strongly monotone and χ -Lipschitzian G n is η -strongly monotone and β -Lipschitzian . (3.21)In turn, invoking (3.17), [a], [b], and (3.16), we deduce that T n is strongly monotone with constant ς and Lipschitzian with constant max { α, β, χ, /ε } . It therefore follows from (3.17) and (3.16) that K n is ς -strongly monotone and ̟ -Lipschitzian . (3.22)8et us define ( ( ∀ i ∈ I ) E i,n : H i → H i : x i γ − i,ℓ i ( n ) x i − ∇ ψ i ( x i )( ∀ k ∈ K ) G k,n : G k → G k : z k ν − k,ϑ k ( n ) z k − ∇ h k ( z k ) (3.23)and let us introduce the variables x n = ( x n , y n , z n , u ∗ n , v ∗ n ) , y n = ( a n , q n , b n , c ∗ n , e ∗ n ) , y ∗ n = ( a ∗ n , q ∗ n , b ∗ n , c n , e n )( ∀ i ∈ I ) e x ∗ i,n = E i,n x i,ℓ i ( n ) − E i,n x n + M ∗ i (cid:0) u ∗ i,n − u ∗ i,ℓ i ( n ) (cid:1) + P k ∈ K L ∗ k,i (cid:0) v ∗ k,n − v ∗ k,ℓ i ( n ) (cid:1)e q ∗ i,n = µ − i,ℓ i ( n ) (cid:0) y i,ℓ i ( n ) − y i,n (cid:1) + ∇ i f i ( y n ) − ∇ i f i (cid:0) y ℓ i ( n ) (cid:1) + u ∗ i,ℓ i ( n ) − u ∗ i,n e c ∗ i,n = σ − i,ℓ i ( n ) (cid:0) u ∗ i,ℓ i ( n ) − u ∗ i,n (cid:1) + M i (cid:0) x i,ℓ i ( n ) − x i,n (cid:1) − y i,ℓ i ( n ) + y i,n ( ∀ k ∈ K ) ( e d ∗ k,n = G k,n z k,ϑ k ( n ) − G k,n z k,n + v ∗ k,ϑ k ( n ) − v ∗ k,n e e ∗ k,n = ̺ − k,ϑ k ( n ) (cid:0) v ∗ k,ϑ k ( n ) − v ∗ k,n (cid:1) − z k,ϑ k ( n ) + z k,n + P i ∈ I L k,i (cid:0) x i,ϑ k ( n ) − x i,n (cid:1) e ∗ n = ( e x ∗ n , e q ∗ n , e d ∗ n , e c ∗ n , e e ∗ n ) . (3.24)Note that, by (3.6), (3.14), and (3.15), we have ( ∀ i ∈ I ) q i,n = q i,ℓ i ( n ) , c ∗ i,n = c ∗ i,ℓ i ( n ) , a i,n = a i,ℓ i ( n ) , s ∗ i,n = s ∗ i,ℓ i ( n ) , c i,n = c i,ℓ i ( n ) ( ∀ k ∈ K ) b k,n = b k,ϑ k ( n ) , e ∗ k,n = e ∗ k,ϑ k ( n ) , b ∗ k,n = b ∗ k,ϑ k ( n ) . (3.25)Hence, for every i ∈ I , we deduce from (3.6), (3.14), and (3.23) that γ − i,ℓ i ( n ) x ∗ i,ℓ i ( n ) = γ − i,ℓ i ( n ) x i,ℓ i ( n ) − ∇ ψ i (cid:0) x i,ℓ i ( n ) (cid:1) − M ∗ i u ∗ i,ℓ i ( n ) − X k ∈ K L ∗ k,i v ∗ k,ℓ i ( n ) = E i,n x i,ℓ i ( n ) − M ∗ i u ∗ i,ℓ i ( n ) − X k ∈ K L ∗ k,i v ∗ k,ℓ i ( n ) = E i,n x i,n − M ∗ i u ∗ i,n − X k ∈ K L ∗ k,i v ∗ k,n + e x ∗ i,n , (3.26)that µ − i,ℓ i ( n ) q i,n = µ − i,ℓ i ( n ) q i,ℓ i ( n ) = µ − i,ℓ i ( n ) y i,ℓ i ( n ) − ∇ i f i (cid:0) y ℓ i ( n ) (cid:1) + u ∗ i,ℓ i ( n ) = µ − i,ℓ i ( n ) y i,n − ∇ i f i ( y n ) + u ∗ i,n + e q ∗ i,n , (3.27)and that σ − i,ℓ i ( n ) c ∗ i,n = σ − i,ℓ i ( n ) c ∗ i,ℓ i ( n ) = σ − i,ℓ i ( n ) u ∗ i,ℓ i ( n ) − y i,ℓ i ( n ) + M i x i,ℓ i ( n ) = σ − i,ℓ i ( n ) u ∗ i,n − y i,n + M i x i,n + e c ∗ i,n . (3.28)In a similar fashion, ( ∀ k ∈ K ) ( ν − k,ϑ k ( n ) d ∗ k,ϑ k ( n ) = G k,n z k,n + v ∗ k,n + e d ∗ k,n ̺ − k,ϑ k ( n ) e ∗ k,n = ̺ − k,ϑ k ( n ) v ∗ k,n − z k,n + P i ∈ I L k,i x i,n + e e ∗ k,n . (3.29)9herefore, it results from (3.17), (3.23), (3.24), (3.9), (3.12), (3.2), and Problem 1.1[d] that K n x n + e ∗ n = T n x n − Sx n + e ∗ n = (cid:16)(cid:0) γ − i,ℓ i ( n ) x ∗ i,ℓ i ( n ) (cid:1) i ∈ I , (cid:0) µ − i,ℓ i ( n ) q i,n (cid:1) i ∈ I , (cid:0) ν − k,ϑ k ( n ) d ∗ k,ϑ k ( n ) (cid:1) k ∈ K , (cid:0) σ − i,ℓ i ( n ) c ∗ i,n (cid:1) i ∈ I , (cid:0) ̺ − k,ϑ k ( n ) e ∗ k,n (cid:1) k ∈ K (cid:17) . (3.30)On the other hand, in the light of (3.17), (3.11), (3.8), (3.2), and [3, Proposition 16.44] we get ( K n + A ) − : X → X : ( x ∗ , y ∗ , z ∗ , u , v ) (cid:16)(cid:0) prox γ i,ℓi ( n ) ϕ i ( γ i,ℓ i ( n ) x ∗ i ) (cid:1) i ∈ I , (cid:0) µ i,ℓ i ( n ) y ∗ i (cid:1) i ∈ I , (cid:0) prox ν k,ϑk ( n ) g k ( ν k,ϑ k ( n ) z ∗ k ) (cid:1) k ∈ K , (cid:0) σ i,ℓ i ( n ) u i (cid:1) i ∈ I , (cid:0) ̺ k,ϑ k ( n ) v k (cid:1) k ∈ K (cid:17) . (3.31)Hence, since (3.25), (3.6), (3.14), and (3.15) entail that ( ∀ i ∈ I ) a i,n = a i,ℓ i ( n ) = prox γ i,ℓi ( n ) ϕ i x ∗ i,ℓ i ( n ) ( ∀ k ∈ K ) b k,n = b k,ϑ k ( n ) = prox ν k,ϑk ( n ) g k d ∗ k,ϑ k ( n ) , (3.32)we invoke (3.24) to get y n = ( K n + A ) − ( K n x n + e ∗ n ) . (3.33)At the same time, it follows from (3.22) and [3, Corollary 20.28 and Proposition 22.11(ii)] that K n issurjective and, in turn, that there exists e x n ∈ X such that K n e x n = K n x n + e ∗ n . (3.34)Thus, (3.33) and Definition 2.1 yield y n = J K n A e x n . (3.35)In view of (3.6), (3.25), and (3.14), we derive from (3.26) that ( ∀ i ∈ I ) a ∗ i,n = s ∗ i,n + X k ∈ K L ∗ k,i e ∗ k,n = s ∗ i,ℓ i ( n ) + X k ∈ K L ∗ k,i e ∗ k,n = γ − i,ℓ i ( n ) (cid:0) x ∗ i,ℓ i ( n ) − a i,ℓ i ( n ) (cid:1) + ∇ ψ i (cid:0) a i,ℓ i ( n ) (cid:1) + M ∗ i c ∗ i,ℓ i ( n ) + X k ∈ K L ∗ k,i e ∗ k,n = γ − i,ℓ i ( n ) x ∗ i,ℓ i ( n ) − γ − i,ℓ i ( n ) a i,n − ∇ ψ i ( a i,n ) − M ∗ i c ∗ i,n − X k ∈ K L ∗ k,i e ∗ k,n ! = E i,n x i,n − M ∗ i u ∗ i,n − X k ∈ K L ∗ k,i v ∗ k,n ! − E i,n a i,n − M ∗ i c ∗ i,n − X k ∈ K L ∗ k,i e ∗ k,n ! + e x ∗ i,n , (3.36)10rom (3.27) that ( ∀ i ∈ I ) q ∗ i,n = ∇ i f i ( q n ) − c ∗ i,n = (cid:0) µ − i,ℓ i ( n ) y i,n − ∇ i f i (cid:0) y n (cid:1) + u ∗ i,n (cid:1) − (cid:0) µ − i,ℓ i ( n ) q i,n − ∇ i f i ( q n ) + c ∗ i,n (cid:1) + e q ∗ i,n , (3.37)and from (3.28) that ( ∀ i ∈ I ) c i,n = c i,ℓ i ( n ) = q i,ℓ i ( n ) − M i a i,ℓ i ( n ) = q i,n − M i a i,n = (cid:0) σ − i,ℓ i ( n ) u ∗ i,n − y i,n + M i x i,n (cid:1) − (cid:0) σ − i,ℓ i ( n ) c ∗ i,n − q i,n + M i a i,n (cid:1) + e c ∗ i,n . (3.38)A similar analysis shows that ( ∀ k ∈ K ) b ∗ k,n = (cid:0) G k,n z k,n + v ∗ k,n (cid:1) − (cid:0) G k,n b k,n + e ∗ k,n (cid:1) + e d ∗ k,n (3.39)and ( ∀ k ∈ K ) e k,n = ̺ − k,ϑ k ( n ) v ∗ k,n − z k,n + X i ∈ I L k,i x i,n ! − ̺ − k,ϑ k ( n ) e ∗ k,n − b k,n + X i ∈ I L k,i a i,n ! + e e ∗ k,n . (3.40)Altogether, it follows from (3.24), (3.36)–(3.40), (3.23), (3.17), (3.9), (3.12), (3.2), and (3.34) that y ∗ n = ( T n x n − Sx n ) − ( T n y n − Sy n ) + e ∗ n = K n x n − K n y n + e ∗ n = K n e x n − K n y n . (3.41)Further, in view of (3.6) and (3.24), we have π n = h y n − x n | y ∗ n i and x n +1 = x n + λ n π n k y ∗ n k y ∗ n , if π n < x n , otherwise . (3.42)Combining (3.11), (3.13), (3.22), (3.35), (3.41), and (3.42), we conclude that (3.6) is an instantia-tion of (2.7). Hence, Proposition 2.2(i) yields X n ∈ N k x n +1 − x n k < + ∞ . (3.43)For every i ∈ I and every integer n > P , (3.5) entails that i ∈ S nj = n − P I j and, in turn, (3.14) and [a]imply that n − P − D ℓ i ( n ) − D τ i ( ℓ i ( n )) = ℓ i ( n ) ℓ i ( n ) n . Consequently, ( ∀ i ∈ I )( ∀ n ∈ N ) n > P + D ⇒ k x n − x ℓ i ( n ) k P + D X j =0 k x n − x n − j k , (3.44)and we therefore infer from (3.43) that ( ∀ i ∈ I ) x n − x ℓ i ( n ) → . (3.45)Likewise, ( ∀ k ∈ K ) x n − x ϑ k ( n ) → . (3.46)11ence, we deduce from (3.24), (3.23), (3.17), and (3.20) that ( ∀ i ∈ I ) k e x ∗ i,n k k E i,n x i,ℓ i ( n ) − E i,n x n k + k M ∗ i k k u ∗ i,n − u ∗ i,ℓ i ( n ) k + X k ∈ K k L ∗ k,i k k v ∗ k,n − v ∗ k,ℓ i ( n ) k k E n x ℓ i ( n ) − E n x n k + k M ∗ i k k u ∗ n − u ∗ ℓ i ( n ) k + X k ∈ K k L ∗ k,i k k v ∗ n − v ∗ ℓ i ( n ) k α k x ℓ i ( n ) − x n k + k M ∗ i k k u ∗ n − u ∗ ℓ i ( n ) k + X k ∈ K k L ∗ k,i k k v ∗ n − v ∗ ℓ i ( n ) k→ . (3.47)Moreover, using (3.24), [a], and (3.45), we get ( ∀ i ∈ I ) k e q ∗ i,n k µ − i,ℓ i ( n ) k y i,ℓ i ( n ) − y i,n k + (cid:13)(cid:13) ∇ i f i ( y n ) − ∇ i f i (cid:0) y ℓ i ( n ) (cid:1)(cid:13)(cid:13) + k u ∗ i,ℓ i ( n ) − u ∗ i,n k ε − k y ℓ i ( n ) − y n k + k Qy n − Qy ℓ i ( n ) k + k u ∗ ℓ i ( n ) − u ∗ n k ( ε − + κ ) k y ℓ i ( n ) − y n k + k u ∗ ℓ i ( n ) − u ∗ n k→ (3.48)and ( ∀ i ∈ I ) k e c ∗ i,n k σ − i,ℓ i ( n ) k u ∗ i,ℓ i ( n ) − u ∗ i,n k + k M i k k x i,ℓ i ( n ) − x i,n k + k y i,ℓ i ( n ) − y i,n k ε − k u ∗ ℓ i ( n ) − u ∗ n k + k M i k k x ℓ i ( n ) − x n k + k y ℓ i ( n ) − y n k→ . (3.49)A similar analysis shows that ( ∀ k ∈ K ) k e d ∗ k,n k → and k e e ∗ k,n k → . (3.50)Altogether, we invoke (3.24) and (3.47)–(3.50) to get e ∗ n → . (3.51)Hence, arguing as in (2.8), (3.22) and (3.34) give k e x n − x n k k K n e x n − K n x n k ς = k e ∗ n k ς → . (3.52)Hence, Proposition 2.2(ii) asserts that there exists x = ( x , y , z , u ∗ , v ∗ ) ∈ zer A such that x n ⇀ x . Thisyields x n ⇀ x . It remains to verify that x solves (1.3). Towards this end, let i ∈ I and set f i = f i (cid:0) · ; ( M x ) r i (cid:1) and ( ∀ k ∈ K ) e g k = ( g k + h k )( · + L k, r i x r i ) . (3.53)Then, by Problem 1.1[b], f i : K i → R is convex and Gˆateaux differentiable, with ∇ f i ( M i x i ) = ∇ i f i ( M x ) . In addition, ( ∀ k ∈ K )( ∀ z k ∈ G k ) ∂ e g k ( z k ) = ( ∂g k + ∇ h k )( z k + L k, r i x r i ) . At the same time,we deduce from (3.4) that u ∗ = Qy = Q ( M x ) , z = Lx , v ∗ ∈ Bz , and ∈ Ax + M ∗ u ∗ + L ∗ v ∗ .Thus, it results from (3.2) and Problem 1.1[d] that u ∗ i = ∇ i f i ( M x ) = ∇ f i ( M i x i )( ∀ k ∈ K ) z k = P j ∈ I L k,j x j = L k,i x i + L k, r i x r i ( ∀ k ∈ K ) v ∗ k ∈ ∂g k ( z k ) + ∇ h k ( z k ) = ( ∂g k + ∇ h k )( L k,i x i + L k, r i x r i ) = ∂ e g k ( L k,i x i ) (3.54)12nd, in turn, from (3.12) and [3, Proposition 16.6(ii)] that ∈ ∂ϕ i ( x i ) + ∇ ψ i ( x i ) + M ∗ i u ∗ i + X k ∈ K L ∗ k,i v ∗ k ⊂ ∂ϕ i ( x i ) + ∇ ψ i ( x i ) + M ∗ i (cid:0) ∇ f i ( M i x i ) (cid:1) + X k ∈ K L ∗ k,i (cid:0) ∂ e g k ( L k,i x i ) (cid:1) ⊂ ∂ ϕ i + ψ i + f i ◦ M i + X k ∈ K e g k ◦ L k,i ! ( x i ) . (3.55)Consequently, appealing to Fermat’s rule [3, Theorem 16.3] and (3.53), we arrive at x i ∈ Argmin x i ∈H i ϕ i ( x i ) + ψ i ( x i ) + f i ( M i x i ) + X k ∈ K e g k ( L k,i x i )= Argmin x i ∈H i ϕ i ( x i ) + ψ i ( x i ) + f i (cid:0) M i x i ; ( M x ) r i (cid:1) + X k ∈ K ( g k + h k )( L k,i x i + L k, r i x r i ) , (3.56)which completes the proof. Remark 3.2
Let us confirm that algorithm (3.6) complies with the principles laid out in Section 1.•
Decomposition:
In (3.6), the nonsmooth functions ( ϕ i ) i ∈ I and ( g k ) k ∈ K are activated separatelyvia their proximity operators, while the smooth functions ( ψ i ) i ∈ I , ( f i ) i ∈ I , and ( h k ) k ∈ K are acti-vated separately via their gradients.• Block-iterative implementation:
At any iteration n , the functions ( f i ) i ∈ I are activated andwe require only that the subfamilies ( ϕ i ) i ∈ I n , ( ψ i ) i ∈ I n , ( g k ) k ∈ K n , and ( h k ) k ∈ K n be used. Toguarantee convergence, we ask in condition [c] of Theorem 3.1 that each of these functions beactivated frequently enough.• Asynchronous implementation:
Given i ∈ I and k ∈ K , the asynchronous character of thealgorithm is materialized by the variables τ i ( n ) and δ k ( n ) which signal when the underlyingcomputations incorporated at iteration n were initiated. Conditions [a] and [b] of Theorem 3.1ask that the lag between the initiation and the incorporation of such computations do not exceed D iterations. The introduction of such techniques in monotone operator splitting were initiatedin [13]. Remark 3.3
Consider the proof of Theorem 3.1. Since Proposition 2.2(ii) yields x n − y n → , weobtain x n − a n → via (3.24) and thus a n ⇀ x . At the same time, by (3.6), given i ∈ I , the sequence ( a i,n ) n ∈ N lies in dom ∂ϕ i ⊂ dom ϕ i . In particular, if a constraint on x i is enforced via ϕ i = ι C i ,then ( a i,n ) n ∈ N converges to the i th component of a solution x while being feasible in the sense that C i ∋ a i,n ⇀ x i . Remark 3.4
The proof of Theorem 3.1 implicitly establishes the convergence of an asynchronousblock-iterative algorithm to solve the more general system of monotone inclusionsfind x ∈ H such that ( ∀ i ∈ I ) 0 ∈ A i x i + R i x i + M ∗ i (cid:0) Q i ( M x ) (cid:1) + X k ∈ K L ∗ k,i ( B k + D k ) X j ∈ I L k,j x j !! (3.57)under the following assumptions: 13a] For every i ∈ I , A i : H i → H i is maximally monotone, α i ∈ [0 , + ∞ [ , and R i : H i → H i ismonotone and α i -Lipschitzian.[b] For every i ∈ I , Q i : K → K i . It is assumed that the operator Q : K → K : y ( Q i y ) i ∈ I ismonotone and Lipschitzian. Furthermore, ( χ i ) i ∈ I are positive numbers such that ( ∀ y ∈ K )( ∀ y ′ ∈ K ) h y − y ′ | Qy − Qy ′ i X i ∈ I χ i k y i − y ′ i k . (3.58)[c] For every k ∈ K , B k : G k → G k is maximally monotone, β k ∈ [0 , + ∞ [ , and D k : G k → G k ismonotone and β k -Lipschitzian.[d] For every i ∈ I and every k ∈ K , M i : H i → K i and L k,i : H i → G k are linear and bounded.Moreover, we set M : H → K : x ( M i x i ) i ∈ I .Indeed, denote by Z the set of points ( x , u ∗ , v ∗ ) ∈ H ⊕ K ⊕ G such that ( ∀ i ∈ I ) u ∗ i = Q i ( M x )( ∀ k ∈ K ) v ∗ k ∈ ( B k + D k ) (cid:0) P j ∈ I L k,j x j (cid:1) ( ∀ i ∈ I ) − M ∗ i u ∗ i − P k ∈ K L ∗ k,i v ∗ k ∈ A i x i + R i x i . (3.59)Suppose that Z = ∅ and execute (3.6) with the following modifications:• For every i ∈ I and every n ∈ N , prox γ i,n ϕ i is replaced by J Id γ i,n A i , ∇ ψ i by R i , and ∇ i f i by Q i .• For every k ∈ K and every n ∈ N , prox ν k,n g k is replaced by J Id ν k,n B k , and ∇ h k by D k .Then there exists ( x , u ∗ , v ∗ ) ∈ Z such that ( x n , u ∗ n , v ∗ n ) ⇀ ( x , u ∗ , v ∗ ) and x solves (3.57). Remark 3.5
By invoking [9, Theorem 4.8] and arguing as in the proof of Proposition 2.2, we obtain astrongly convergent counterpart of Proposition 2.2 which, in turn, yields a strongly convergent versionof Theorem 3.1.Theorem 3.1 requires that (3.7) be satisfied. With the assistance of monotone operator theoryarguments applied to a set of primal-dual inclusions, we provide below sufficient conditions for that.Let us start with a technical fact.
Lemma 3.6
Let H and G be real Hilbert spaces, let B : G → G be ∗ monotone, and let L : H → G belinear and bounded. Then L ∗ ◦ B ◦ L is ∗ monotone.Proof . Set A = L ∗ ◦ B ◦ L . First, we deduce from [3, Proposition 20.10] that A is monotone. Next,take x ∈ dom A and x ∗ ∈ ran A . On the one hand, Lx ∈ dom B and there exists z ∗ ∈ ran B such that x ∗ = L ∗ z ∗ . On the other hand, for every ( y , y ∗ ) ∈ gra A , there exists v ∗ ∈ G such that ( Ly , v ∗ ) ∈ gra B and y ∗ = L ∗ v ∗ , from which we obtain h x − y | y ∗ − x ∗ i = h x − y | L ∗ v ∗ − L ∗ z ∗ i = h Lx − Ly | v ∗ − z ∗ i sup ( w , w ∗ ) ∈ gra B h Lx − w | w ∗ − z ∗ i . (3.60)14herefore, by ∗ monotonicity of B , sup ( y , y ∗ ) ∈ gra A h x − y | y ∗ − x ∗ i sup ( w , w ∗ ) ∈ gra B h Lx − w | w ∗ − z ∗ i < + ∞ . (3.61)Consequently, A is ∗ monotone. Proposition 3.7
Consider the setting of Problem and set C = ( X i ∈ I L k,i x i − z k ! k ∈ K (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ( ∀ i ∈ I ) x i ∈ dom ϕ i and ( ∀ k ∈ K ) z k ∈ dom g k ) . (3.62) Suppose that ∈ sri C and that one of the following is satisfied: [a] For every i ∈ I , one of the following holds: ∂ ( ϕ i + ψ i ) is surjective. ϕ i + ψ i is supercoercive.
3/ dom ϕ i is bounded. ϕ i + ψ i is uniformly convex. [b] Q is ∗ monotone and one of the following holds: M ∗ ◦ Q ◦ M is surjective. Q is surjective and, for every i ∈ I , M i is bijective.Then (3.7) holds.Proof . Let A , B , and L be as in (3.2) and define T : H → H : x Ax + L ∗ (cid:0) B ( Lx ) (cid:1) + M ∗ (cid:0) Q ( M x ) (cid:1) . (3.63)Suppose that b x ∈ zer T and set b u ∗ = Q ( M b x ) . On the one hand, in view of Problem 1.1[b], ( ∀ i ∈ I ) b u ∗ i = ∇ i f i ( M b x ) . On the other hand, it results from (3.63) that there exists b v ∗ ∈ B ( L b x ) such that − M ∗ b u ∗ − L ∗ b v ∗ ∈ A b x or, equivalently, by (3.12) and (3.2), ( ∀ i ∈ I ) − M ∗ i b u ∗ i − P k ∈ K L ∗ k,i b v ∗ k ∈ ∂ϕ i ( b x i )+ ∇ ψ i ( b x i ) . Further, using (3.2), we obtain ( ∀ k ∈ K ) b v ∗ k ∈ ( ∂g k + ∇ h k )( P j ∈ I L k,j b x j ) . Altogether,we have shown that zer T = ∅ ⇒ (3.7) holds. Therefore, it suffices to show that zer T = ∅ . To do so,define ϕ : H → ] −∞ , + ∞ ] : x P i ∈ I (cid:0) ϕ i ( x i ) + ψ i ( x i ) (cid:1) g : G → ] −∞ , + ∞ ] : z P k ∈ K (cid:0) g k ( z k ) + h k ( z k ) (cid:1) P = A + L ∗ ◦ B ◦ L . (3.64)Then, by (3.2) and [3, Proposition 16.9], A = ∂ ϕ and B = ∂ g . In turn, since (3.62) and (3.2)imply that ∈ sri C = sri ( L ( dom ϕ ) − dom g ) , we derive from [3, Theorem 16.47(i)] that P = A + L ∗ ◦ B ◦ L = ∂ ( ϕ + g ◦ L ) . Therefore, in view of [3, Theorem 20.25 and Example 25.13], A , B , and P are maximally monotone and ∗ monotone . (3.65)15a]: Fix temporarily i ∈ I . By [3, Theorem 20.25], ∂ ( ϕ i + ψ i ) is maximally monotone. First, if[a]2/ holds, then [3, Corollary 16.30, and Propositions 14.15 and 16.27] entail that ran ∂ ( ϕ i + ψ i ) = dom ∂ ( ϕ i + ψ i ) ∗ = H i and, hence, [a]1/ holds. Second, if [a]3/ holds, then dom ∂ ( ϕ i + ψ i ) ⊂ dom ( ϕ i + ψ i ) = dom ϕ i is bounded and, therefore, it follows from [3, Corollary 21.25] that [a]1/holds. Finally, if [a]4/ holds, then [3, Proposition 17.26(ii)] implies that [a]2/ holds and, in turn, that[a]1/ holds. Altogether, it is enough to assume that the operators ( ∂ ( ϕ i + ψ i )) i ∈ I are surjective and toshow that zer T = ∅ . Assume that ( ∂ ( ϕ i + ψ i )) i ∈ I are surjective and set R = − M ◦ P − ◦ ( − M ∗ ) + Q − . Then we derive from (3.2) that A is surjective. On the other hand, Lemma 3.6 asserts that L ∗ ◦ B ◦ L is ∗ monotone. Hence, (3.65) and [3, Corollary 25.27(i)] yields dom P − = ran P = H .In turn, since P − and Q − are maximally monotone, [3, Theorem 25.3] implies that R is likewise.Furthermore, we observe that dom Q − ⊂ K = dom ( − M ◦ P − ◦ ( − M ∗ )) and, by virtue of (3.65),[3, Proposition 25.19(i)], and Lemma 3.6, that − M ◦ P − ◦ ( − M ∗ ) is ∗ monotone. Therefore,since ran Q − = dom Q = K , [3, Corollary 25.27(ii)] entails that R is surjective and, in turn, thatzer R = ∅ . Consequently, [3, Proposition 26.33(iii)] asserts that zer T = ∅ .[b]1/: Lemma 3.6 asserts that M ∗ ◦ Q ◦ M is ∗ monotone. At the same time, since Q is maximallymonotone and dom Q = K , it results from (3.65) and [3, Theorem 25.3] that T = P + M ∗ ◦ Q ◦ M is maximally monotone. Hence, since M ∗ ◦ Q ◦ M is surjective, we derive from (3.65) and [3,Corollary 25.27(i)] that T is surjective and, therefore, that zer T = ∅ .[b]2/ ⇒ [b]1/: Since the assumption implies that M is bijective, so is M ∗ . This makes M ∗ ◦ Q ◦ M surjective. Remark 3.8
Sufficient conditions for ∈ sri C to hold in Proposition 3.7 can be found in [12, Propo-sition 5.3]. We discuss problems which are shown to be realizations of Problem 1.1 and which can therefore besolved by the asynchronous block-iterative algorithm (3.6) of Theorem 3.1.
Example 4.1 (quadratic coupling)
Let K be a real Hilbert space and let I be a nonempty finite set.For every i ∈ I , let H i be a real Hilbert space, let ϕ i ∈ Γ ( H i ) , let α i ∈ [0 , + ∞ [ , let ψ i : H i → R beconvex and differentiable with an α i -Lipschitzian gradient, let M i : H i → K be linear and bounded,let Λ i be a nonempty finite set, let ( ω i,ℓ,j ) ℓ ∈ Λ i ,j ∈ I r { i } be in [0 , + ∞ [ , and let ( κ i,ℓ ) ℓ ∈ Λ i be in ]0 , + ∞ [ .Additionally, set H = L i ∈ I H i and K = L i ∈ I K . The problem is tofind x ∈ H such that ( ∀ i ∈ I ) x i ∈ Argmin x i ∈H i ϕ i ( x i ) + ψ i ( x i ) + X ℓ ∈ Λ i κ i,ℓ (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) M i x i − X j ∈ I r { i } ω i,ℓ,j M j x j (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) . (4.1)It is assumed that ( ∀ y ∈ K )( ∀ y ′ ∈ K ) X i ∈ I X ℓ ∈ Λ i κ i,ℓ (cid:28) y i − y ′ i (cid:12)(cid:12)(cid:12)(cid:12) y i − y ′ i − X j ∈ I r { i } ω i,ℓ,j ( y j − y ′ j ) (cid:29) > . (4.2)16efine ( ∀ i ∈ I ) f i : K → R : y X ℓ ∈ Λ i κ i,ℓ (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) y i − X j ∈ I r { i } ω i,ℓ,j y j (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) . (4.3)Then, for every i ∈ I and every y ∈ K , f i ( · ; y r i ) is convex and differentiable with ∇ i f i ( y ) = X ℓ ∈ Λ i κ i,ℓ y i − X j ∈ I r { i } ω i,ℓ,j y j ! . (4.4)Hence, in view of (4.2), the operator Q : K → K : y ( ∇ i f i ( y )) i ∈ I is monotone and Lipschitzian.Thus, (4.1) is a special case of (1.3) with ( ∀ i ∈ I ) K i = K and ( ∀ k ∈ K ) g k = h k = 0 . In particular,suppose that, for every i ∈ I , H i = K , C i is a nonempty closed convex subset of H i , ϕ i = ι C i , ψ i = 0 , M i = Id , Λ i ⊂ I r { i } , and ( ∀ ℓ ∈ Λ i ) κ i,ℓ = 1( ∀ j ∈ I r { i } ) ω i,ℓ,j = ( , if j = ℓ ;0 , if j = ℓ. (4.5)Then (4.1) becomesfind x ∈ H such that ( ∀ i ∈ I ) x i ∈ Argmin x i ∈ C i X ℓ ∈ Λ i k x i − x ℓ k . (4.6)This unifies models found in [2]. Example 4.2 (minimax)
Let I be a finite set and suppose that ∅ = J ⊂ I . Let ( H i ) i ∈ I be realHilbert spaces, and set U = L i ∈ I r J H i and V = L j ∈ J H j . For every i ∈ I , let ϕ i ∈ Γ ( H i ) , let α i ∈ [0 , + ∞ [ , let ψ i : H i → R be convex and differentiable with an α i -Lipschitzian gradient. Further,let L : U ⊕ V → R be differentiable with a Lipschitzian gradient and such that, for every u ∈ U andevery v ∈ V , the functions − L ( u , · ) and L ( · , v ) are convex. Finally, for every i ∈ I r J and every j ∈ J , let L j,i : H i → H j be linear and bounded. Consider the multivariate minimax problemminimize u ∈ U maximize v ∈ V X i ∈ I r J (cid:0) ϕ i ( u i )+ ψ i ( u i ) (cid:1) − X j ∈ J (cid:0) ϕ j ( v j )+ ψ j ( v j ) (cid:1) + L ( u , v )+ X i ∈ I r J X j ∈ J h L j,i u i | v j i . (4.7)Now set H = U ⊕ V and define ( ∀ i ∈ I ) f i : H → R : ( u , v ) ( L ( u , v ) + (cid:10) u i | P j ∈ J L ∗ j,i v j (cid:11) , if i ∈ I r J ; − L ( u , v ) − (cid:10)P k ∈ I r J L i,k u k | v i (cid:11) , if i ∈ J. (4.8)Then H = L i ∈ I H i and (4.7) can be put in the formfind x ∈ H such that ( ∀ i ∈ I ) x i ∈ Argmin x i ∈H i ϕ i ( x i ) + ψ i ( x i ) + f i ( x i ; x r i ) . (4.9)17et us verify Problem 1.1[b]. On the one hand, we have ( ∀ i ∈ I )( ∀ x ∈ H ) ∇ i f i ( x ) = ( ∇ i L ( x ) + P j ∈ J L ∗ j,i x j , if i ∈ I r J ; −∇ i L ( x ) − P k ∈ I r J L i,k x k , if i ∈ J. (4.10)On the other hand, the operator R : H → H : x (cid:0)(cid:0) ∇ i L ( x ) (cid:1) i ∈ I r J , (cid:0) −∇ j L ( x ) (cid:1) j ∈ J (cid:1) (4.11)is monotone [22, 23] and Lipschitzian, while the bounded linear operator S : H → H : x X j ∈ J L ∗ j,i x j ! i ∈ I r J , − X k ∈ I r J L i,k x k ! i ∈ J ! (4.12)satisfies S ∗ = − S and it is therefore monotone [3, Example 20.35]. Hence, since the operator Q inProblem 1.1[b] can be written as Q = R + S , it is therefore monotone and Lipschitzian. Altogether,(4.7) is an instantiation of (1.3). Special cases of (4.7) can be found in [14, 24]. Example 4.3
In Problem 1.1, consider the following scenario: K = { } , G is the standard Euclideanspace R M , r ∈ G , g = ι E , where E = r + [0 , + ∞ [ M , h = 0 , and, for every i ∈ I , H i is the standardEuclidean space R N i , ψ i = 0 , and ϕ i = ι C i , where C i is a nonempty closed convex subset of H i . Then,upon setting N = P i ∈ I N i , we obtain the modelfind x ∈ R N such that ( ∀ i ∈ I ) x i ∈ Argmin x i ∈ C i L ,i x i + L , r i x r i ∈ E f i ( x i ; x r i ) (4.13)investigated in [25]. Example 4.4 (minimization)
Consider the setting of Problem 1.1 where [b] is replaced by[b’] For every i ∈ I , f i = f , where f : K → R is a differentiable convex function such that Q = ∇ f is Lipschitzian,and, in addition, the following is satisfied:[e] For every k ∈ K , g k : G k → R is Gˆateaux differentiable.Then (1.3) reduces to the multivariate minimization problemminimize x ∈ H X i ∈ I (cid:0) ϕ i ( x i ) + ψ i ( x i ) (cid:1) + f ( M x ) + X k ∈ K ( g k + h k ) X j ∈ I L k,j x j ! . (4.14)The only asynchronous block-iterative algorithm we know of to solve (4.14) is [10, Algorithm 4.5],which is based on different decomposition principles. Special cases of (4.14) are found in partialdifferential equations [1], machine learning [6], and signal recovery [7], where they were solvedusing synchronous and non block-iterative methods.18 eferences [1] H. Attouch, J. Bolte, P. Redont, and A. Soubeyran, Alternating proximal algorithms for weakly coupledconvex minimization problems. Applications to dynamical games and PDE’s, J. Convex Anal. , vol. 15, pp.485–506, 2008.[2] J.-B. Baillon, P. L. Combettes, and R. Cominetti, There is no variational characterization of the cycles inthe method of periodic projections,
J. Funct. Anal. , vol. 262, pp. 400–408, 2012.[3] H. H. Bauschke and P. L. Combettes,
Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2nded. Springer, New York, 2017.[4] A. Bensoussan, J.-L. Lions, and R. Temam, Sur les m´ethodes de d´ecomposition, de d´ecentralisation et decoordination, et applications,
Cahier IRIA , no. 11, pp. 5–189, 1972.[5] M. Bravo, D. Leslie, and P. Mertikopoulos, Bandit learning in concave N -person games, Proc. Adv. NeuralInform. Process. Syst. Conf. , vol. 31, pp. 5661–5671, 2018.[6] L. M. Brice˜no-Arias, G. Chierchia, E. Chouzenoux, and J.-C. Pesquet, A random block-coordinate Douglas–Rachford splitting method with low computational complexity for binary logistic regression,
Comput.Optim. Appl. , vol. 72, pp. 707–726, 2019.[7] L. M. Brice˜no-Arias and P. L. Combettes, Convex variational formulation with smooth coupling for multi-component signal decomposition and recovery,
Numer. Math. Theory Methods Appl. , vol. 2, pp. 485–508,2009.[8] L. M. Brice˜no-Arias and P. L. Combettes, Monotone operator methods for Nash equilibria in non-potentialgames, in:
Computational and Analytical Mathematics , (D. Bailey et al., eds.), pp. 143–159. Springer, NewYork, 2013.[9] M. N. B`ui and P. L. Combettes, Warped proximal iterations for monotone inclusions,
J. Math. Anal. Appl. ,vol. 491, art. 124315, 21 pp., 2020.[10] M. N. B`ui and P. L. Combettes, Multivariate monotone inclusions in saddle form. https://arxiv.org/abs/2002.06135 [11] G. Cohen, Nash equilibria: Gradient and decomposition algorithms,
Large Scale Syst. , vol. 12, pp. 173–184, 1987.[12] P. L. Combettes, Systems of structured monotone inclusions: Duality, algorithms, and applications,
SIAMJ. Optim. , vol. 23, pp. 2420–2447, 2013.[13] P. L. Combettes and J. Eckstein, Asynchronous block-iterative primal-dual decomposition methods formonotone inclusions,
Math. Program. , vol. B168, pp. 645–672, 2018.[14] P. L. Combettes and J.-C. Pesquet, Fixed point strategies in data science. https://arxiv.org/abs/2008.02260 [15] R. Cominetti, F. Facchinei, and J. B. Lasserre,
Modern Optimization Modelling Techniques . Birkh¨auser,Basel, 2012.[16] F. Facchinei, A. Fischer, and V. Piccialli, On generalized Nash games and variational inequalities,
Oper.Res. Lett. , vol. 35, pp. 159–164, 2007.[17] A. von Heusinger and C. Kanzow, Relaxation methods for generalized Nash equilibrium problems withinexact line search,
J. Optim. Theory Appl. , vol. 143, pp. 159–183, 2009.[18] S. Hoda, A. Gilpin, J. Pe˜na, and T. Sandholm, Smoothing techniques for computing Nash equilibria ofsequential games,
Math. Oper. Res. , vol. 35, pp. 494–512, 2010.[19] A. Kannan and U. V. Shanbhag, Distributed computation of equilibria in monotone Nash games via itera-tive regularization techniques,
SIAM J. Optim. , vol. 22, pp. 1177–1205, 2012.[20] J. F. Nash, Jr., Equilibrium points in n -person games, Proc. Nat. Acad. Sci. USA , vol. 36, pp. 48–49, 1950.
21] J. Nash, Non-cooperative games,
Ann. Math. , vol. 54, pp. 286–295, 1951.[22] R. T. Rockafellar, Monotone operators associated with saddle-functions and minimax problems, in:
Non-linear Functional Analysis, Part 1 , (F. E. Browder, ed.), pp. 241–250. AMS, Providence, RI, 1970.[23] R. T. Rockafellar, Saddle-points and convex analysis, in:
Differential Games and Related Topics , (H. W.Kuhn and G. P. Szeg¨o, eds.), pp. 109–127. North-Holland, Amsterdam, 1971.[24] R. T. Rockafellar, Monotone relations and network equilibrium, in:
Variational Inequalities and NetworkEquilibrium Problems , (F. Giannessi and A. Maugeri, eds.), pp. 271–288. Plenum Press, New York, 1995.[25] P. Yi and L. Pavel, An operator splitting approach for distributed generalized Nash equilibria computation,
Automatica , vol. 102, pp. 111–121, 2019., vol. 102, pp. 111–121, 2019.