Incentive Mechanism Design for Federated Learning: Hedonic Game Approach
IIncentive Mechanism Design for FederatedLearning: Hedonic Game Approach
Cengis Hasan
University of LuxembourgSnT - Interdisciplinary Centre for Security, Reliability and Trust [email protected]
Abstract —Incentive mechanism design is crucial for enablingfederated learning. We deal with clustering problem of agents con-tributing to federated learning setting. Assuming agents behaveselfishly, we model their interaction as a stable coalition partitionproblem using hedonic games where agents and clusters are theplayers and coalitions, respectively. We address the followingquestion: is there any utility allocation method ensuring a Nash-stable coalition partition? We propose the Nash-stable set andanalyze the conditions of non-emptiness. Besides, we deal withthe decentralized coalition partition. We formulate the problemas a non-cooperative game and prove the existence of a potential.
I. I
NTRODUCTION
Data protection is a major concern. If we do not trustsomeone withholding our data, we may opt for federatedlearning by privately developing intelligent systems to createprivacy-preserving AI.
Federated learning enables privacy-preserving machine learning in a decentralized way [1]. It isused in situations where data is distributed among differentagents and training is impossible due to the difficulty tocollect data centrally. All data is kept on device while ashared (global) learning model is trained in each device andaggregated (combined) centrally. Formally, we consider thefollowing setting: i) data owner agents which locally trains theshared learning model, and ii) model aggregating entity (MAE)which combines learning model of its own with the agents.MAE and agents contribute to the same shared learning model.Federated learning has been identified as a distributed machinelearning framework which sees rapid advances and broadadoption in next generation networking and edge systems [1]–[8]. Obviously, the motivation to implement federated learningis to reduce the variance in a learned model by accessing moredata.A very crucial question is how would MAE motivate theagents to participate in federated learning. Designing themechanism of agents’ incentives can be performed by utilizingvarious frameworks such game theory, auction theory, etc [5].Any clustering among agents (players) being able to makestrategic decisions becomes a coalition formation game whenthe players –for various individual reasons– may wish to belongto a relative small coalition rather than the grand coalition –the set of all players. Players’ moves from one to anothercoalition are governed by a set of rules. Basically, an agent(player) will move to a new coalition when it may obtain abetter utility from this coalition. We shall not consider any permission requirements, which means that a player is alwaysaccepted by a coalition to which the player is willing to join.Based on those rules, the crucial question in the game contextis how a stable partition exists . This is essential to enablefederated learning .In the following, we study the hedonic coalition formationgames and the stability notion that we analyze is the
Nashstability [9]. A coalition formation game is called hedonic ifeach player’s preferences over partitions of players depend onlyon the members of his/her coalition. Finding a stable coalitionpartition is the main question in a coalition formation game.We refer to [10] discussing the stability concepts associated to hedonic conditions . In the sequel, we concentrate on the Nashstability. The definition of the Nash stability is quite simple: apartition of players is Nash stable whenever no player deviatesfrom its coalition to another coalition in the partition .In this work, the following problem is studied: having coali-tions associated with their utilities, we seek the answer of howmust the coalition utilities be allocated to the players in orderto obtain a stable coalition partition. Clearly, the fundamentalquestion is to determine which utility allocation methods mayensure a Nash-stable partition. We first propose the definitionof the Nash-stable set which is the set of all possible utilityallocation methods resulting in Nash-stable partitions. We showthat additively separable and symmetric utility allocation alwaysensures Nash-stable partitions. Moreover, our work aims alsoat finding the partitions in a decentralized setting. We modelthe problem of finding a Nash-stable partition by formulatingit as a non-cooperative game and show that such a game is apotential game.II. M
OTIVATION AND P ROBLEM D ESCRIPTION
We consider a set of agents denoted by 𝑁 = { , , . . . , 𝑛 } that can participate in the federated learning setting, and a model aggregating entity (MAE) which aggregates (combines) learning model of its own with the agents. MAE and agentscontribute in the same global learning model. The parametersof learning model of MAE and agent 𝑖 are represented by θ MAE and θ 𝑖 , respectively where every agent 𝑖 has 𝑚 𝑖 datasamples. We also consider that the communication link betweenMAE and agent 𝑖 can be characterized by a probability ofreliable transmission (e.g. low bit error rate) denoted by 𝑝 𝑖 .If a cluster , say 𝑆 ⊆ 𝑁 , of agents are federated then theaggregated (combined) learning model is found by using an a r X i v : . [ c s . G T ] J a n ig. 1. Federated learning framework. aggregation method which in generic case is denoted as θ 𝑆 = A ( θ , θ , . . . , θ | 𝑆 | ) where A (·) shows the aggregation function.On the other hand, in this paper, we consider the followingaggregation method: θ 𝑆 = (cid:205) 𝑖 ∈ 𝑆 𝑥 𝑖 𝑚 𝑖 θ 𝑖 (cid:205) 𝑖 ∈ 𝑆 𝑥 𝑖 𝑚 𝑖 , 𝑥 𝑖 = (cid:40) , reception successful , , otherwise . (1)in which 𝑥 𝑖 = if MAE receives successfully information of θ 𝑖 .This is nothing more than choosing agent 𝑖 with probability 𝑝 𝑖 where we have x = ( 𝑥 , 𝑥 , . . . , 𝑥 𝑛 ) and p = ( 𝑝 , 𝑝 , . . . , 𝑝 𝑛 ) .Furthermore, we denote by L ( θ 𝑆 | x ) the loss of learning modelgiven by parameters θ 𝑆 . If two disjoint clusters 𝑆 and 𝑇 , i.e. 𝑆 ∩ 𝑇 = ∅ , are federated, then we denote the new parametersas θ 𝑆 ∪ 𝑇 . It is reasonable to assume that the loss of θ 𝑆 ∪ 𝑇 islower than average loss of θ 𝑆 and θ 𝑇 : L ( θ 𝑆 ∪ 𝑇 | x ) ≤ L ( θ 𝑆 | x ) + L ( θ 𝑇 | x ) . (2)From the perspective of agents, we assume that the modelaggregating entity can assign a utility to all possible clusters.Note that from the MAE point-view, this is the cost that mustbe paid to the cluster; however, we shall formulate the problemusing the utility term which is the monetary cost charged toMAE.Then, we come up with the question how to design theincentives in order that the agents are willing to participate infederated learning setting taking into account their preferences.We represent by 𝑢 the utility which assigns a real value forevery subset of 𝑁 , i.e. 𝑢 : 2 𝑁 → R where 𝑁 is the collectionof all possible non-empty subsets of 𝑁 and empty set ∅ , andwe set 𝑢 (∅) = . Any agent 𝑖 can join a cluster if guaranteedto be paid at least 𝑢 ( 𝑖 ) = 𝜋 𝑖 price which is given by 𝜋 𝑖 = ˜ 𝑓 (cid:16) 𝑝 𝑖 L( θ 𝑖 ) (cid:17) (3)where ˜ 𝑓 : R → R is a monotonically increasing function andinversely proportional to L ( θ 𝑖 ) meaning that the less loss themore gain. Besides, 𝑝 𝑖 is the second parameter which hasan impact on the price asked by agent. It corresponds to thefact that as 𝑝 𝑖 has a poor value, the agent asks lower price toparticipate in the federation. We define the utility received by agent 𝑖 being in cluster 𝑆 as following: utility of agent 𝑖 = 𝜋 𝑖 + 𝜙 𝑆𝑖 (4)where 𝜙 𝑆𝑖 is the gain of agent 𝑖 by joining to cluster 𝑆 , and weset 𝜙 𝑖𝑖 = for all 𝑖 ∈ 𝑁 . On the other hand, MAE determinesthe utility of any cluster 𝑆 ∈ 𝑁 : 𝑢 ( 𝑆 ) = 𝑓 (| 𝑆 |) (cid:124)(cid:123)(cid:122)(cid:125) raw utility + ˜ 𝑓 (cid:16) E [L( θ 𝑆 ) ] (cid:17)(cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) learning utility (5)where 𝑓 : R → R is a subadditive function, i.e. 𝑓 (| 𝑆 ∪ 𝑇 |) ≤ 𝑓 (| 𝑆 |) + 𝑓 (| 𝑇 |) for all 𝑆, 𝑇 ∈ 𝑁 and 𝑆 ∩ 𝑇 = ∅ . The rationalbehind this choice is that the total cost of two disjoint clustersis more than the cost when they are merged. The expectedvalue of loss function is given by E [L ( θ 𝑆 )] = ∑︁ x ∈X L ( θ 𝑆 | x ) P [ x ] (6) P [ x ] = (cid:214) 𝑖 ∈ 𝑁 𝑝 𝑥 𝑖 𝑖 ( − 𝑝 𝑖 ) − 𝑥 𝑖 (7)where X with |X| = 𝑛 is the set of all possible combinationsof x vectors.The utility function 𝑢 is superadditive if 𝑢 ( 𝑆 ∪ 𝑇 ) ≥ 𝑢 ( 𝑆 ) + 𝑢 ( 𝑇 ) for all possible 𝑆 and 𝑇 in 𝑁 such that 𝑆 ∩ 𝑇 = ∅ . ˜ 𝑓 (cid:16) E [L( θ 𝑆 ∪ 𝑇 ) ] (cid:17) − ˜ 𝑓 (cid:16) E [L( θ 𝑆 ) ] (cid:17) − ˜ 𝑓 (cid:16) E [L( θ 𝑇 ) ] (cid:17) ≥ 𝑓 (| 𝑆 |) + 𝑓 (| 𝑇 |) − 𝑓 (| 𝑆 ∪ 𝑇 |) , ∀ 𝑆, 𝑇 ∈ 𝑁 , 𝑆 ∩ 𝑇 = ∅ (8) Remark 2.1:
Note that if superadditivity would hold, wewould have all the agents in the same cluster. On the otherhand, the theorem implies that superadditivity can rarely existsince it may be very difficult to find a loss function whichsatisfies the conditions.Considering that the agents strategically decide to whichcluster to join, we can define the problem of clustering as acoalition formation game. We then change the language ofproblem formulation using game theoretic terms, i.e. agents → players , cluster → coalition Players may not be always fully cooperative and behaveselfishly. We then face the problem of finding coalitiontructures which are stable under selfishness. In the sequel, wedeal with such a setting.III. H
EDONIC G AME
A hedonic game is given by a pair (cid:104) 𝑁, (cid:31)(cid:105) , where (cid:31) = ((cid:23) , (cid:23) , . . . , (cid:23) 𝑛 ) denotes the preference profile , specifying foreach player 𝑖 ∈ 𝑁 his preference relation (cid:23) 𝑖 , i.e. a reflexive,complete and transitive binary relation. Definition 3.1:
A coalition structure or a coalition partition isa set Π which partitions the players’ set 𝑁 such that (cid:208) 𝑆 ∈ Π 𝑆 = 𝑁 . All coalitions in Π are disjoint coalitions, i.e., 𝑆 ∩ 𝑇 = ∅ for all 𝑆, 𝑇 ∈ Π .Given Π and 𝑖 , 𝑆 Π ( 𝑖 ) denotes the set 𝑆 ∈ Π such that 𝑖 ∈ 𝑆 .Moreover, P is the set of all possible coalition partitions over 𝑁 . In its partition form, a coalition formation game is definedon the set 𝑁 by associating a utility value 𝑢 ( 𝑆 | Π ) to eachsubset of any partition Π of 𝑁 . In its characteristic form, theutility value of a set is independent of the other coalitions,and therefore, 𝑢 ( 𝑆 | Π ) = 𝑢 ( 𝑆 ) . The games of this form aremore restrictive but present interesting properties to reachan equilibrium. Practically speaking, this assumption meansthat the gain of a group is independent of the other playersoutside the group. Hedonic games fall into this category withan additional assumption: Definition 3.2:
A coalition formation game is hedonic if • the gain of any player depends solely on the members ofthe coalition to which the player belongs , and • the coalitions arise as a result of the preferences of theplayers over their possible coalitions’ set . A. Preference Relation
The preference relation of a player can be defined over a preference function . We consider the case where the preferencerelation is chosen to be the utility allocated to the player in acoalition. Thus, player 𝑖 prefers the coalition 𝑆 to 𝑇 iff, 𝜙 𝑆𝑖 ≥ 𝜙 𝑇𝑖 ⇔ 𝑆 (cid:23) 𝑖 𝑇 . (9)
B. The Nash Stability
The stability concepts for a hedonic game are various. Inthe literature, a hedonic game is said individually stable, Nashstable, core stable, strict core stable, Pareto optimal, strongNash stable, or, strict strong Nash stable . We refer to [10] for athorough definition of these different stability concepts. In thispaper, we are only interested in the
Nash stability becausethe players do not cooperate to take their decisions jointly.
Definition 3.3 (Nash Stability):
A partition of players is Nash-stable whenever no player has incentive to unilaterally changehis or her coalition to another coalition in the partition whichcan be mathematically formulated as: partition Π NS is said tobe Nash-stable if no player can benefit from moving from hiscoalition 𝑆 Π NS ( 𝑖 ) to another existing coalition 𝑇 ∈ Π NS , i.e.: 𝑆 Π NS ( 𝑖 ) (cid:23) 𝑖 𝑇 ∪ 𝑖, ∀ 𝑇 ∈ Π NS ∪ ∅ ; ∀ 𝑖 ∈ 𝑁. (10)which can be similarly defined over preference function asfollowing: 𝜙 𝑆 Π NS ( 𝑖 ) 𝑖 ≥ 𝜙 𝑇 ∪ 𝑖𝑖 , ∀ 𝑇 ∈ Π NS ∪ ∅ ; ∀ 𝑖 ∈ 𝑁. (11) Nash-stable partitions are immune to individual movementseven when a player who wants to change does not needpermission to join or leave an existing coalition [11]. Remark 3.1:
Stability concepts being immune to individualdeviation are
Nash stability, individual stability, contractualindividual stability . Nash stability is the strongest within above.The notion of core stability has been used already in somemodels where immunity to coalition deviation is required [9].
Remark 3.2 (Impossibility of a stability concept):
In [12], theauthors propose some set of axioms which are non-emptiness,symmetry pareto optimality, self-consistency ; and they analyzethe existence of any stability concept that can satisfy theseaxioms. It is proven that for any game | 𝑁 | > , there does notexist any solution which satisfies these axioms. In this work,we show how non-emptiness can be guaranteed using Nashstability as a solution concept. C. Aggregated Learning Model Parameters
When a stable partition exists, then this means that all theplayers (agents) are agreed to participate to federation. Asa result of this, MAE utilizes the following aggregation oflearning model parameters: θ 𝑓 = 𝑤 θ MAE + ( − 𝑤 ) θ 𝑁 (12) θ 𝑁 = (cid:205) 𝑖 ∈ 𝑁 𝑥 𝑖 𝑚 𝑖 θ 𝑖 (cid:205) 𝑖 ∈ 𝑁 𝑥 𝑖 𝑚 𝑖 (13)where ≤ 𝑤 ≤ is a weighting parameter showing how muchMAE favors the aggregated learning model parameters of agents(players), θ MAE shows the learning parameters of MAE’s localmodel. In summary, we have the following procedure: while
Nash-stable partition Π NS exists1. Player (agent) 𝑖 sends information of θ 𝑖 , for all 𝑖 ∈ 𝑁
2. MAE calculate aggregated learning model parame-ters θ 𝑓 end Given θ 𝑓 , the expected value of loss function in federationcan be calculated as following: E [L ( θ 𝑓 )] = ∑︁ x ∈X L ( θ 𝑓 | x ) P [ x ]≥ L ( E [ θ 𝑓 ]) ( Jensen’s inequality ) (14)where E [ θ 𝑓 ] = 𝑤 θ MAE + ( − 𝑤 ) E [ θ 𝑁 ] = 𝑤 θ MAE + ( − 𝑤 ) ∑︁ x ∈X (cid:205) 𝑖 ∈ 𝑁 𝑥 𝑖 𝑚 𝑖 θ 𝑖 (cid:205) 𝑖 ∈ 𝑁 𝑥 𝑖 𝑚 𝑖 P [ x ] (15)IV. T HE N ASH - STABLE S ET As the utility 𝑢 associated with all possible coalitions areknown, we are interested in finding a utility distribution toensure Nash stability. We thus define a utility allocation method φ ∈ R 𝜅 where 𝜅 = 𝑛 𝑛 − as following: φ = { 𝜙 𝑆𝑖 : ∀ 𝑖 ∈ 𝑆, ∀ 𝑆 ∈ 𝑁 } (16)hich directly sets up a preference profile. The set of allpossible utility allocation methods is denoted by F ⊂ R 𝜅 . Letus define the mapping M : F → P . Clearly, for any utilityallocation method φ , mapping M finds corresponding partition,i.e. M ( φ ) ∈ P . We define the Nash-stable set which includesall those efficient allocation methods that build the followingset: 𝒩 stable = (cid:8) φ ∈ R 𝜅 : 𝑆 M ( φ ) ( 𝑖 ) (cid:23) 𝑖 𝑇 ∪ 𝑖, ∀ 𝑇 ∈ M ( φ ) ∪ ∅ ; ∀ 𝑖 ∈ 𝑁 } . (17)Let us define the set of constraints stemming from thepreference function in order to check if the Nash-stable set isnon-empty. Due to the utility bound, for any utility allocationmethod φ , we have (cid:205) 𝑖 ∈ 𝑆 ( 𝜋 𝑖 + 𝜙 𝑆𝑖 ) ≤ 𝑢 ( 𝑆 ) for all 𝑆 ∈ 𝑁 called as budged balanced utility allocation which further canbe given by ∑︁ 𝑖 ∈ 𝑆 ˜ 𝑓 (cid:16) L( θ 𝑖 ) (cid:17) + ∑︁ 𝑖 ∈ 𝑆 𝜙 𝑆𝑖 ≤ 𝑓 (cid:16) | 𝑆 |L( θ 𝑆 ) (cid:17) , ∀ 𝑆 ∈ 𝑁 . (18)For simplicity, let us define marginal utility as following: Δ θ ( 𝑆 ) = (cid:40) 𝑓 (cid:16) | 𝑆 |L( θ 𝑆 ) (cid:17) − (cid:205) 𝑖 ∈ 𝑆 ˜ 𝑓 (cid:16) L( θ 𝑖 ) (cid:17) , ∀ 𝑆 ∈ 𝑁 \ 𝑖, ∀ 𝑖 ∈ 𝑁 , ∀ 𝑖 ∈ 𝑁. (19)which results in the following constraints: 𝒞 θ ( φ ) : = (cid:40)∑︁ 𝑖 ∈ 𝑆 𝜙 𝑆𝑖 ≤ Δ θ ( 𝑆 ) , ∀ 𝑆 ∈ 𝑁 (cid:41) . (20)On the other hand, for any φ , the constraints that ensure theNash stability are given by 𝒞 θ ( φ ) : = (cid:110) 𝜙 𝑆 M ( φ ) ( 𝑖 ) 𝑖 ≥ 𝜙 𝑇 ∪ 𝑖𝑖 , ∀ 𝑇 ∈ M ( φ ) ∪ ∅ ; ∀ 𝑖 ∈ 𝑁 (cid:111) , (21)Then, the Nash-stable set is non-empty, iif: 𝒩 stable ( θ ) = (cid:8) φ ∈ R 𝜅 : 𝒞 θ ( φ ) and 𝒞 θ ( φ ) (cid:9) , (22)which allows us to conclude: Theorem 4.1:
The Nash-stable set can be non-empty.
Proof 4.1:
We can check if the Nash-stable is non-emptyby solving the following optimization problem: max φ ∑︁ 𝑆 ∈ 𝑁 ∑︁ 𝑖 ∈ 𝑆 𝜙 𝑆𝑖 subject to 𝒞 θ ( φ ) and 𝒞 θ ( φ ) . If there exists any feasible solution of this problem, thenwe conclude that there is at least one utility allocationmethod which provides a Nash-stable partition.However, searching in an exhaustive manner over the wholepartitions is NP-hard as the number of partitions growsaccording to the Bell number. Typically, with only players,the number of partitions is as large as , . A. Superadditive Utility
If the utility function 𝑢 is superadditive, then it is trivial tocheck that the marginal utility is also superadditive: Δ θ ( 𝑆 ∪ 𝑇 ) ≥ Δ θ ( 𝑆 ) + Δ θ ( 𝑇 ) , for all possible 𝑆 and 𝑇 such that 𝑆 ∩ 𝑇 = ∅ .Due to eq. (18), we have ∑︁ 𝑖 ∈ 𝑆 ∪ 𝑇 𝜙 𝑆 ∪ 𝑇𝑖 = ∑︁ 𝑖 ∈ 𝑆 𝜙 𝑆 ∪ 𝑇𝑖 + ∑︁ 𝑖 ∈ 𝑇 𝜙 𝑆 ∪ 𝑇𝑖 ≤ Δ θ ( 𝑆 ∪ 𝑇 ) ∑︁ 𝑖 ∈ 𝑆 𝜙 𝑆𝑖 + ∑︁ 𝑖 ∈ 𝑇 𝜙 𝑇𝑖 ≤ Δ θ ( 𝑆 ) + Δ θ ( 𝑇 )⇒ ∑︁ 𝑖 ∈ 𝑆 𝜙 𝑆 ∪ 𝑇𝑖 + ∑︁ 𝑖 ∈ 𝑇 𝜙 𝑆 ∪ 𝑇𝑖 ≥ ∑︁ 𝑖 ∈ 𝑆 𝜙 𝑆𝑖 + ∑︁ 𝑖 ∈ 𝑇 𝜙 𝑇𝑖 . This result means that any player is better off in a largercoalition which ultimately all players have the most gain inthe grand coalition. This is obvious from eq. (21) where forevery player 𝑖 ∈ 𝑁 , 𝜙 𝑁𝑖 ≥ 𝜙 𝑆𝑖 for all 𝑆 ∈ 𝑁 . B. Additively Separable and Symmetric Utility
Preferences of a player are additively separable wheneverthe preference can be stated with a function characterizinghow a player prefers another player in each coalition. Thismeans that the player’s preference for a coalition is based onindividual preferences. This can be formalized as follows:
Definition 4.1:
The preferences of a player are said to be additively separable if there exists a function 𝑣 𝑖 : 𝑁 → R such that ∑︁ 𝑗 ∈ 𝑆 𝑣 𝑖 ( 𝑗 ) ≥ ∑︁ 𝑗 ∈ 𝑇 𝑣 𝑖 ( 𝑗 ) ⇔ 𝑆 (cid:23) 𝑖 𝑇, ∀ 𝑆, 𝑇 ∈ 𝑁 . (23) 𝑣 𝑖 ( 𝑖 ) is normalized and set to 𝑣 𝑖 ( 𝑖 ) = . A profile of additivelyseparable preferences satisfies symmetry if 𝑣 𝑖 ( 𝑗 ) = 𝑣 𝑗 ( 𝑖 ) = 𝑣 ( 𝑖, 𝑗 ) , for all 𝑖, 𝑗 ∈ 𝑁 . The meaning of 𝑣 ( 𝑖, 𝑗 ) is the mutualgain of player 𝑖 and 𝑗 when they are in the same coalition. Let V ( 𝑆 ) be the all possible bipartite coalitions which can occurin coalition 𝑆 such that: V ( 𝑆 ) : = {( 𝑖, 𝑗 ) ∈ 𝑆 : 𝑗 > 𝑖 } , | 𝑆 | ≥ . We then define v ∈ R |V ( 𝑁 ) | which shall serve as a utility allo-cation method to generate additively separable and symmetricpreferences: v = { 𝑣 ( 𝑖, 𝑗 ) : ∀( 𝑖, 𝑗 ) ∈ V ( 𝑁 )} and mapping M shall find corresponding partition, i.e. M ( v ) ∈P . The constraints that define the Nash-stable set are thendefined over v : 𝒞 θ ( φ ) → 𝒞 θ ( v ) and 𝒞 θ ( φ ) → 𝒞 θ ( v ) Further, note that the utility that player 𝑖 gains in coalition 𝑆 is given by 𝜋 𝑖 + 𝜙 𝑆𝑖 = ˜ 𝑓 (cid:16) L( θ 𝑖 ) (cid:17) + ∑︁ 𝑗 ∈ 𝑆 𝑣 ( 𝑖, 𝑗 )⇒ 𝜙 𝑆𝑖 = ∑︁ 𝑗 ∈ 𝑆 𝑣 ( 𝑖, 𝑗 ) (24)n the other hand, due to the symmetry property of mutualgain, we have the following: ∑︁ 𝑖, 𝑗 ∈ 𝑆 𝑣 ( 𝑖, 𝑗 ) = ∑︁ ( 𝑖, 𝑗 ) ∈V ( 𝑆 ) 𝑣 ( 𝑖, 𝑗 ) . For example, if 𝑆 = ( , , ) , then (cid:205) 𝑖, 𝑗 ∈ 𝑆 𝑣 ( 𝑖, 𝑗 ) = [ 𝑣 ( , ) + 𝑣 ( , ) + 𝑣 ( , )] . Theorem 4.2:
Additively separable and symmetric pref-erences always admit a Nash-stable partition. Therefore,constraints in 𝒞 θ ( v ) are always satisfied [9].Based on this theorem, we only need to satisfy the constraintsgiven by 𝒞 θ ( v ) . Thus, we define the Nash-stable set whichgenerates additively separable and symmetric preferences 𝒩 Astable ( θ ) ⊂ 𝒩 stable ( θ ) as following: 𝒩 Astable ( θ ) = (cid:40) v ∈ R V ( 𝑁 ) : ∑︁ ( 𝑖, 𝑗 ) ∈V ( 𝑆 ) 𝑣 ( 𝑖, 𝑗 ) ≤ Δ θ ( 𝑆 ) , ∀ 𝑆 ∈ 𝑁 (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) 𝒞 θ ( v ) (cid:41) (25)Finding the values of 𝑣 ( 𝑖, 𝑗 ) in eq. (25) satisfying 𝒞 θ ( v ) conditions can be done straightforward. However, we proposeto formulate as an optimization problem for finding the valuesof 𝑣 ( 𝑖, 𝑗 ) . A feasible solution of the following linear programguarantees the non-emptiness of 𝒩 Astable ( θ ) : max v ∑︁ ( 𝑖, 𝑗 ) ∈V ( 𝑁 ) 𝑣 ( 𝑖, 𝑗 ) subject to ∑︁ ( 𝑖, 𝑗 ) ∈V ( 𝑆 ) 𝑣 ( 𝑖, 𝑗 ) ≤ Δ θ ( 𝑆 ) , ∀ 𝑆 ∈ 𝑁 , (26)where note that any feasible solution v ∗ is upper bounded by (cid:205) ( 𝑖, 𝑗 ) ∈V ( 𝑁 ) 𝑣 ∗ ( 𝑖, 𝑗 ) ≤ Δ θ ( 𝑁 )/ . Furthermore, the coalitionpartition that stems from v ∗ is given by Π NS : = M ( v ∗ ) whichis Nash-stable.V. D ECENTRALIZED C OALITION P ARTITION
In this section, we study finding a Nash-stable partition in adecentralized setting. We, in fact, model the problem of findinga Nash-stable partition as a non-cooperative game.A hedonic coalition formation game is equivalent to a non-cooperative game. Denote as Σ the set of strategies . We assumethat the number of strategies is equal to the number of players,i.e. | Σ | = | 𝑁 | . This is sufficient to represent all possible choices.Indeed, the players that select the same strategy are interpretedas a coalition. For example, if every player chooses differentstrategies, then this corresponds to the coalition partitioncomprised of singletons.Consider the best-reply dynamics where in a particular step 𝑙 , only one player chooses its best strategy. A strategy tuple in step 𝑙 is denoted as σ 𝑙 = { 𝜎 𝑙 , 𝜎 𝑙 , . . . , 𝜎 𝑙𝑛 } , where 𝜎 𝑙𝑖 is thestrategy of player 𝑖 in step 𝑙 . In every step, only one dimensionis changed in σ 𝑙 . We further denote as Π ( σ 𝑙 ) the partition instep 𝑙 . Define as 𝑆 ( 𝑙 ) σ ( 𝑖 ) = { 𝑗 : 𝜎 𝑙𝑖 = 𝜎 𝑙𝑗 , ∀ 𝑗 ∈ 𝑁 } the set of players that share the same strategy with player 𝑖 . Thus, notethat ∪ 𝑖 ∈ 𝑁 𝑆 ( 𝑙 ) σ ( 𝑖 ) = 𝑁 for each step. The utility of player 𝑖 is 𝜙 𝑖 ( σ 𝑙 ) and verifies the following equivalence: 𝜙 𝑖 ( σ 𝑙 ) ≥ 𝜙 𝑖 ( σ 𝑙 − ) ⇔ 𝑆 ( 𝑙 ) σ ( 𝑖 ) (cid:23) 𝑖 𝑆 ( 𝑙 − ) σ ( 𝑖 ) , (27)where player 𝑖 is the one that takes its turn in step 𝑙 . Anysequence of strategy-tuple in which each strategy-tuple differsfrom the preceding one in only one coordinate is called a path ,and a unique deviator in each step strictly increases the utilityhe receives is an improvement path . Obviously, any maximalimprovement path which is an improvement path that can notbe extended is terminated by stability. A. Stability Analysis
The Nash stability is defined as following: 𝜎 NS 𝑖 ∈ arg max 𝜎 𝑖 ∈ Σ 𝑖 𝜙 𝑖 ( 𝜎 𝑖 , 𝜎 − 𝑖 ) , ∀ 𝑖 ∈ 𝑁. (28)Moreover, Nash-stable coalition partition is given by collection: 𝑆 σ NS ( 𝑗 ) = { 𝑘 ∈ 𝑁 : 𝜎 NS 𝑘 = 𝑗 } , ∀ 𝑗 ∈ ΣΠ NS = { 𝑆 σ NS ( 𝑗 ) : ∀ 𝑗 ∈ Σ } . (29)In the sequel, we prove that the additively separable andsymmetric utilities result in a potential game. Theorem 5.1:
Any additively separable and symmetricutility results in a potential game with potential : 𝑃 v ( σ ) = ∑︁ 𝑆 ∈ Π ( σ ) ∑︁ ( 𝑖, 𝑗 ) ∈V ( 𝑆 ) 𝑣 ( 𝑖, 𝑗 ) . (30) Proof 5.1:
A non-cooperative game is a potential gamewhenever there exists a function 𝑃 v such that: 𝑃 v ( 𝜎 𝑖 , 𝜎 − 𝑖 ) − 𝑃 v ( 𝜎 (cid:48) 𝑖 , 𝜎 − 𝑖 ) = 𝜙 ( 𝜎 𝑖 , 𝜎 − 𝑖 ) − 𝜙 ( 𝜎 (cid:48) 𝑖 , 𝜎 − 𝑖 ) where ( 𝜎 𝑖 , 𝜎 − 𝑖 ) = σ . This means that when player 𝑖 switches from strategy 𝜎 𝑖 to 𝜎 (cid:48) 𝑖 the difference of its utilitycan be given by the difference of a function 𝑃 . We choosethe following potential function: 𝑃 v ( σ ) = ∑︁ 𝑆 ∈ Π ( 𝜎 ) ∑︁ ( 𝑖, 𝑗 ) ∈V ( 𝑆 ) 𝑣 ( 𝑖, 𝑗 ) (31)Let us denote as 𝑖 ∈ 𝑆 and 𝑖 ∉ 𝑆 (cid:48) the coalitions when player 𝑖 switches from strategy 𝜎 𝑖 to 𝜎 (cid:48) 𝑖 , respectively. Potentialfunction is given as following before and after switching 𝑃 v ( 𝜎 𝑖 , 𝜎 − 𝑖 ) = ∑︁ ( 𝑖, 𝑗 ) ∈V ( 𝑆 ) 𝑣 ( 𝑖, 𝑗 ) + ∑︁ ( 𝑘, 𝑗 ) ∈V ( 𝑆 (cid:48) ) 𝑣 ( 𝑘, 𝑗 )+ ∑︁ 𝑇 ∈ Π ( σ )\{ 𝑆,𝑆 (cid:48) } ∑︁ ( 𝑘, 𝑗 ) ∈V ( 𝑇 ) 𝑣 ( 𝑘, 𝑗 ) 𝑃 v ( 𝜎 (cid:48) 𝑖 , 𝜎 − 𝑖 ) = ∑︁ ( 𝑘, 𝑗 ) ∈V ( 𝑆 \ 𝑖 ) 𝑣 ( 𝑘, 𝑗 ) + ∑︁ ( 𝑘, 𝑗 ) ∈V ( 𝑆 (cid:48) ∪ 𝑖 ) 𝑣 ( 𝑘, 𝑗 )+ ∑︁ 𝑇 ∈ Π ( 𝜎 )\{ 𝑆,𝑆 (cid:48) } ∑︁ ( 𝑘, 𝑗 ) ∈V ( 𝑇 ) 𝑣 ( 𝑘, 𝑗 ) here note that we have 𝑆 → 𝑆 \ 𝑖 and 𝑆 (cid:48) → 𝑆 (cid:48) ∪ 𝑖 afterswitching. Thus, we have 𝑃 v ( 𝜎 𝑖 , 𝜎 − 𝑖 ) − 𝑃 v ( 𝜎 (cid:48) 𝑖 , 𝜎 − 𝑖 ) = ∑︁ ( 𝑖, 𝑗 ) ∈V ( 𝑆 ) 𝑣 ( 𝑖, 𝑗 ) + ∑︁ ( 𝑘, 𝑗 ) ∈V ( 𝑆 (cid:48) ) 𝑣 ( 𝑘, 𝑗 )− ∑︁ ( 𝑘, 𝑗 ) ∈V ( 𝑆 \ 𝑖 ) 𝑣 ( 𝑘, 𝑗 ) − ∑︁ ( 𝑘, 𝑗 ) ∈V ( 𝑆 (cid:48) ∪ 𝑖 ) 𝑣 ( 𝑘, 𝑗 ) = ∑︁ 𝑗 ∈ 𝑆 𝑣 ( 𝑖, 𝑗 ) − ∑︁ 𝑗 ∈ 𝑆 (cid:48) ∪ 𝑖 𝑣 ( 𝑖, 𝑗 ) On the other hand, the utility shift before and after strategyswitch is given by 𝜙 ( 𝜎 𝑖 , 𝜎 − 𝑖 ) − 𝜙 ( 𝜎 (cid:48) 𝑖 , 𝜎 − 𝑖 ) = ∑︁ 𝑗 ∈ 𝑆 𝑣 ( 𝑖, 𝑗 ) − ∑︁ 𝑗 ∈ 𝑆 (cid:48) ∪ 𝑖 𝑣 ( 𝑖, 𝑗 ) which concludes the proof that 𝑃 v ( 𝜎 𝑖 , 𝜎 − 𝑖 )− 𝑃 v ( 𝜎 (cid:48) 𝑖 , 𝜎 − 𝑖 ) = 𝜙 ( 𝜎 𝑖 , 𝜎 − 𝑖 ) − 𝜙 ( 𝜎 (cid:48) 𝑖 , 𝜎 − 𝑖 ) .In a potential game, a Nash equilibrium shall result in anoptimum in potential 𝑃 v . Therefore, σ ∗ ∈ arg max σ 𝑃 v ( σ ) corresponds to a coalition partition Π ( 𝜎 ∗ ) : = M ( v ) which isNash-stable. VI. C ONCLUSIONS
We analyzed stable clustering problem in federated learningsetting. Clusters are made up of the agents contributing tofederated learning. We considered that every agent gains autility when switching from one cluster to another one. Wemodeled the decisions of agents in the framework hedonicgames which is a widely used cooperative game model for thistype of problems. A fundamental question in hedonic gamesis to analyze the conditions how stable coalition partitions canoccur. We studied the existence of stable coalition partitionsby introducing the Nash-stable set, and analyzed the existenceof decentralized coalition partitions.R
EFERENCES[1] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning:Challenges, methods, and future directions,”
IEEE Signal ProcessingMagazine , vol. 37, no. 3, pp. 50–60, 2020.[2] A. Elgabli, J. Park, A. S. Bedi, M. Bennis, and V. Aggarwal, “Gadmm:fast and communication efficient framework for distributed machinelearning,”
Journal of Machine Learning Research , vol. 21, no. 76, pp.1–39, 2020.[3] E. by: Peter Kairouz and H. B. McMahan, “Advances andopen problems in federated learning,”
Foundations and Trends inMachine Learning , vol. 14, no. 1, pp. –, 2021. [Online]. Available:http://dx.doi.org/10.1561/2200000083[4] S. Zheng, C. Shen, and X. Chen, “Design and analysis of uplink anddownlink communications for federated learning,”
IEEE Journal onSelected Areas in Communications , pp. 1–1, 2020.[5] S. Samarakoon, M. Bennis, W. Saad, and M. Debbah, “Federated learningfor ultra-reliable low-latency v2v communications,” in , 2018, pp. 1–7.[6] L. U. Khan, S. R. Pandey, N. H. Tran, W. Saad, Z. Han, M. N. H.Nguyen, and C. S. Hong, “Federated learning for edge networks: Resourceoptimization and incentive mechanism,”
IEEE Communications Magazine ,vol. 58, no. 10, pp. 88–93, 2020. [7] S. Hosseinalipour, C. G. Brinton, V. Aggarwal, H. Dai, and M. Chiang,“From federated to fog learning: Distributed machine learning overheterogeneous wireless networks,”
IEEE Communications Magazine ,vol. 58, no. 12, pp. 41–47, 2020.[8] K. Yang, T. Jiang, Y. Shi, and Z. Ding, “Federated learning via over-the-air computation,”
IEEE Transactions on Wireless Communications ,vol. 19, no. 3, pp. 2022–2035, 2020.[9] J. Hajduková, “Coalition formation games: A survey,”
InternationalGame Theory Review , vol. 8, no. 4, pp. 613–641, 2006.[10] H. Aziz and F. Brandl, “Existence of stability in hedonic coalitionformation games,” in
Proceedings of the 11th International Conferenceon Autonomous Agents and Multiagent Systems (AAMAS 2012) , 2012.[11] A. Bogomonlaia and M. Jackson, “The stability of hedonic coalitionstructures,”
Games and Economic Behavior , vol. 38, pp. 201–230, jan2002.[12] S. Barber and A. Gerber, “A note on the impossibility of a satisfactoryconcept of stability for coalition formation games,”