GraphFederator: Federated Visual Analysis for Multi-party Graphs
Dongming Han, Wei Chen, Rusheng Pan, Yijing Liu, Jiehui Zhou, Ying Xu, Tianye Zhang, Changjie Fan, Jianrong Tao, Xiaolong, Zhang
aa r X i v : . [ c s . H C ] A ug GraphFederator: Federated Visual Analysis for Multi-party Graphs
Dongming Han, Wei Chen, Rusheng Pan, Yijing Liu, Jiehui Zhou, Ying Xu, Tianye Zhang,Changjie Fan, Jianrong Tao, and Xiaolong (Luke) Zhang
A1A3
F1 F2
B DD1 EE1 E2C1C2C3CA2A . . . . . . . . E + 4 E + 4 . . . . . . E + 3 . . . . . . . E + 3 . . . . . . . . . . . . . . E + 4
E + 4 . . . . . . E + 3 . . . . . . . E + 3 . . . . . . F3 D2 Figure 1: Illustration of GraphFederator with case study 1 (Section 7.2.1). The server view (A) supports model configuration (A1),client selection (A2), and process monitoring (A3). The client view (B) shows the information of selected clients. The user selectsfederated graph representations of multi-party graphs (F1) or a single client from the training process. The representations arevisualized in other views (C) (D1) (E1). The embedding view (C), consists of the projection view (C1), the cluster view (C2),and the anomaly view (C3). Federated structure representations are shown in the structure view (D) by using a node-link diagram(D1). The attribute view (E) employs histograms (E1) to visualize federated attribute representations. When detected anomalousplayers are selected (F2), their federated graph representations are visualized in other views (D1) (E1). The filtering condition ofbanned status is set from the corresponding bin of the histogram (F3), and federated graph representations of banned players fromanomalous players are shown in other views (D2) (E2). The structure view (D2) shows links among players. It indicates that somebanned players know each other and may belong to a studio that controls many plug-in players accounts.
Abstract — This paper presents GraphFederator, a novel approach to construct joint representations of multi-party graphs and sup-ports privacy-preserving visual analysis of graphs. Inspired by the concept of federated learning, we reformulate the analysis ofmulti-party graphs into a decentralization process. The new federation framework consists of a shared module that is responsiblefor joint modeling and analysis, and a set of local modules that run on respective graph data. Specifically, we propose a federatedgraph representation model (FGRM) that is learned from encrypted characteristics of multi-party graphs in local modules. We alsodesign multiple visualization views for joint visualization, exploration, and analysis of multi-party graphs. Experimental results withtwo datasets demonstrate the effectiveness of our approach.
Index Terms —Federated Learning, Graph Analysis, Graph Visualization, Representation Learning • Dongming Han, Wei Chen, Rusheng Pan, Yijing Liu, Jiehui Zhou, Ying Xu,and Tianye Zhang are with the State Key Lab of CAD & CG, ZhejiangUniversity, Hangzhou, Zhejiang 310058, China. E-mail:[email protected]• Changjie Fan and Jianrong Tao are with NetEase Fuxi AI Lab, Hangzhou,China. E-mail: [email protected],[email protected]• Xiaolong (Luke) Zhang is with Penn State, University Park, Pennsylvania,United States. E-mail: [email protected]• Corresponding authors: Wei Chen. E-mail: [email protected] received xx xxx. 201x; accepted xx xxx. 201x. Date of Publication
NTRODUCTION
Visual analysis of multi-party graphs plays an important role in helpingus understand real-world complex data [6,48,49], such as ego-networkanalysis in social media [52, 56], disease diagnosis in healthcare [24],and anomaly detection in public security [7, 55]. Various features ormodels extracted from multi-party graphs can be integrated to supporta comprehensive understanding of the entire graph data. Using the xx xxx. 201x; date of current version xx xxx. 201x. For information onobtaining reprints of this article, please send e-mail to: [email protected] Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx ntegrated information, we can conduct more comprehensive investi-gations. For instance, by combining knowledge graphs of patients anddiseases from multiple hospitals, doctors can gain a deeper understand-ing of diseases and develop best treatment plans.One main bottleneck of exploiting multi-party graphs is data acces-sibility. Early studies on graph visual analysis assume that the graphdata is freely accessible. Currently, however, more and more graphdata are distributed (e.g., on servers in different organizations). To an-alyze such data, we need to combine multi-party graphs and examinethem as an entirety. Considering of privacy and security, raw data indistributed clients may be prohibited from accessing. This leads totwo challenges for visual analysis of multi-party graphs. The first isthe federation of multi-party graphs. Because raw data should be keptlocally, creating a joint representation of data in all clients must resortto privacy-preserving feature extraction techniques. This situation iseven exaggerated by the fact that features of different graphs may bedifferent. A uniformed joint representation that is capable of charac-terizing the essential features of each graph is needed. The secondchallenge is that the federated analysis based on joint representationsis difficult. Designing a generalized framework for federating variousand complex analysis tasks remains a huge challenge.Visual analysis of multi-party graphs based on the joint represen-tations requires new approaches. Conventional graph analysis, whichis performed in a centralized model, or only uses limited accessiblegraph data, cannot be applied to multi-party graph analysis. Existingsolutions for distributed graph analysis [14, 28] mainly focus on thepartitions of data and analysis tasks for the purposes of performanceimprovement, and are incapable of supporting multi-party graph anal-ysis either.One way to support privacy-preserving decentralized graph analy-sis is to build a decentralized federation of both data features and theanalysis. We propose GraphFederator, a novel federation approachthat constructs joint representations of multi-party graphs, and sup-ports privacy-preserving visual analysis of graphs. Inspired by feder-ated learning, we reformulate the analysis of multi-party graphs into adecentralization process. The new federation framework consists of ashared module that is responsible for joint modeling and analysis, anda set of local modules that run on respective graph data. Specifically,we propose a federated representation model that is iteratively learnedfrom the encrypted characteristics of multi-party graphs in local mod-ules. We design multiple visualization views for joint visualization,exploration, and analysis of multi-party graphs. The contributions ofthis paper include:• a federated graph representation model to represent and extractdistinctive features from multi-party graphs; and• a federated visual analysis approach to support privacy-preserving graphs analysis.The rest of this paper is organized as follows. Related work is dis-cussed in Section 2. Section 3 introduces the problem formulation.Section 4 explains our design goals and the overview of our approach.Our federated graph representation model is introduced in Section 5.Section 6 presents the visual interface. The evaluation is discussed inSection 7. Discussions and the conclusion are given in Section 9 andSection 10, respectively.
ELATED W ORK
A graph analysis task is usually defined as the analysis of entities or as-sociated properties [37]. Here, entities denote nodes, links, paths, andnetworks, while the properties include structures and derived features.Graph analysis tasks [19] can be classified into four groups: topology-based tasks, attribute-based tasks, browsing tasks, and overview tasks.Complex tasks can be decomposed into a set of basic tasks. Alterna-tively, tasks can be represented as a combination of two fundamen-tal tasks [33]: analyzing topology for given attributes, and analyzingattributes for a given topological structure. These two tasks are sup-ported with respect to topological structures, including nodes, edges,clusters, node neighbors, paths, and substructures. A recent study [3] proposes a multi-level typology to facilitate spe-cific task classifications. Likewise, 29 group-level graph visualizationtasks [40] are classified into four groups: group only tasks, group-nodetasks, group-link tasks, and group-network tasks. From the viewpointof graph-based sensemaking, four categories of graph visualizationtasks [36] are introduced: visualization and exploration; global, local,and hybrid views; subgraph mining and interaction. Without loss ofgenerality, this paper follows their notations and design specified tasks.A typical graph analysis system named Network Repository [39] pro-vides users with the ability to explore, visualize, and compare dataalong many different dimensions interactively and in real-time. Bycombining global network statistics, local node-level network statis-tics, and features, users can easily discover key insights into the data.
Machine learning is benefited from the ability to train increasinglysophisticated models with the unprecedented growth of data collec-tion [47]. To overcome the problem of high computational cost for an-alyzing large-scale data, parallel or distributed computing has becomepopular [5]. Similarly, decentralized machine learning approaches canbe reformulated from centralized versions. There are two basic build-ing blocks of distributed learning algorithms: matrix multiplicationand data shuffling [20]. Much effort is paid to improve communicationefficiency. For instance, minimizing the number of rounds of commu-nication works well for cases where data is unevenly distributed overan extremely large number of nodes [15]. A new framework is pro-posed to manage asynchronous data communications between clientsand servers with flexible consistency, elastic scalability, and fault tol-erance [21].To protect the privacy data during the process of communication, aHomomorphic Encryption (HE) scheme [38] is designed to preservestructures of the original message space. HE can be leveraged toprivacy-preserving training or prediction of linear regression, linearclassifiers, decision trees, matrix factorization, and neural networks.To address the problem of non-linear activation functions, a HE-basedneural network scheme [34] is proposed with an interactive protocolbetween the data owner and the model owner. The calculated trans-formation of the data owner is swapped in an encrypted form with theresult from the model.As a pioneering work on privacy-preserving computing, securemulti-party computation (MPC) [13] guarantees that clients can onlyget the final cumulative model weight. Recently, federated learningemerged as a new privacy-protection scheme to construct a global ma-chine learning model with distributed multiple clients [1,15,16,29,32,54]. To improve the training efficiency while protecting the privacy ofmultiple parties, local training data is kept from the central server. Itis trained in a decentralized manner on multiple remote clients with-out transferring raw data. By integrating parameters from clients, theserver can compose a global model. For large-scale graph data, dis-tributed processing is needed. A distributed implementation of the Di-jkstra algorithm [8] can handle various graph problems like the depthfirst search in an undirected graph.
TUX [53] is a distributed enginethat supports the layout and computation of graphs with billions ofedges and outperforms other state-of-the-art graph engines by using aseries of graph-based optimizations.To our best knowledge, research on using federated learning forgraph analysis is rare. A distributed learning algorithm on graph gen-eralizes the previous work on federated learning [18] and provides afully decentralized framework with localized data of individual nodeskept from one another. The entire learning process over nodes does notuse a central server and hence is a peer-to-peer method. A distributedgraph neural network is constructed by following the scheme of feder-ated learning [31]. The algorithm uses a similarity matrix to capturethe high-distance structure of nodes precisely in graph neural network. ROBLEM F ORMULATION
For the reason of data privacy, directly analyzing graphs in multipleclients is difficult. Our solution is to extract joint representations fromulti-party graphs and use joint representations for analysis. How-ever, existing federated learning frameworks are inapplicable for multi-party graphs, because the data characteristics and analysis tasks ofmulti-party graphs are quite diverse. Designing a model that extractsjoint representations with privacy-persevering is a nontrivial issue.Given K parties, we define G k as a graph in the k -th party, where G k = ( V k , E k ) and V k , E k represent the node set and edge set, respec-tively. Each graph G k owns the identical attributes A k = a , a ,.., a J ,where J denotes the dimension of attributes. Each node v has a publicID. Nodes from different graphs may have the same public ID. Ourgoal is to construct a federated graph representation model that gener-ates the joint representations R of nodes V k ∈ G k . Sensitive informa-tion should be kept locally: attributes of each node, such as age, yearand salary; links among nodes; and personal privacy information, likename, e-mail, and address. PPROACH O VERVIEW
We work closely with five domain experts. Two of them are professorswhose research focus is on federated learning and privacy analysis,respectively. The other three include one professor and two Ph.D. stu-dents. Their research interests are all related to graph representationlearning. We also consulted two experts of an online game publisherand provider. Both of them have experience in federated learning andgraph analysis research. Through discussions with these experts, weidentify the following design goals:
G Global representation learning model for multi-party graphsdata.
Considering that the data distribution can be distinctivefor different parties, a globalized criterion is needed to ensure allcomputations are secured and generating satisfying results.
R Representations learning for each party graph.
The main goalof multi-party graphs representation learning is to derive the lo-calized graph representation of each party. In particular, local-ized representations should be:R1
Privacy-preserving:
Parties are not allowed to share ortransmit their raw data, which should be kept locally toavoid the leakage of sensitive information.R2
High-quality:
Local representations learned by our decen-tralized scheme should achieve comparable performancewith the ones learned in a centralized manner.R3
Information-diverse:
Learned local representations arecomprehensive and can enlighten insights about graphsfrom various aspects. Such representations should be ex-tracted on the basis of rich and diverse characteristics of agraph, such as the structure information, the side informa-tion (node attributes), and graph embedding results.
C Customizations support.
Analysis tasks vary with differentusers, and thus a flexible scheme is needed. Users should beallowed to customize analysis methods and control relevant pa-rameters of different steps upon their preferences.
Our general approach is shown in Figure 2. The central component ofour approach is a server that runs a global federated graph representa-tion model (FGRM). The server communicates with individual clients,each of which owns its local graph data and runs its local FGRM. Andwith the user interface to provide data for various views.FGRM is designed to extract multiple graph representations ( G ).It runs in a server-client mode, and contains two components: graphrepresentation and federated computing.The graph representation of each local graph is computed based onthree components ( R2 , R3 ): the embedding component, the structurecomponent, and the attribute component. The federated computing isused to federate graph representations from multi-party graphs.The federated computing ( R1 , C ) includes three parts: federated ini-tiation, client-side update, and server-side update. In the federated ini-tiation, the server distributes FGRM, predefined encryption schemes,and related rules to multiple remote clients. The client-side module updates FGRM with the local graph and sends extracted graph repre-sentations to the server. The server-side module collects and handlesthese representations. Next, the server sends back the representationsto each client. The update process is iterated until the specified num-ber of rounds is reached. Finally, the server receives federated graphrepresentations from multi-party graphs.We design and implement a visual interface to visualize differentfederated graph representations. Users configure and build FGRM bycustom schemes in the server view for extracting federated representa-tions from multi-party graphs ( C ). The embedding view, the attributeview, and the structure view display the federated embedding represen-tation, the federated attribute representation, and the federated struc-ture representation, respectively. EDERATED G RAPH R EPRESENTATION M ODEL
FGRM supports the construction of three different types of graph rep-resentations: graph embedding, node attribute, and structure informa-tion. These representations depict graph information from differentaspects, making FGRM suitable for various visual analysis tasks. Forthe purpose of simplicity, the federated graph representations are de-noted as R = ( W emb , W att , W struc ) . Here, the federated embedding rep-resentation is denoted as W emb . The federated atrribute representationis denoted as W att = ( W att , W att ,..., W att P ) , and P denotes the dimen-sion of attributes. The federated structure representation is denoted as W struc . The embedding component is used to construct a graph embeddingrepresentation of a graph. This representation converts the node setinto low dimensional vectors in a canonical space [4].
Input:
The input includes a graph G = ( V , E ) with attributes A , anembedding model, and corresponding parameters. Extraction:
For a graph G , its embeddings are generated from agraph embedding model. Some models only use the topology to ex-tracted representations, while some require node features. For embed-ding learning models that require topology and features as the input,the final representation is extracted directly by the model with G and A . For those models that only require topology as the input, basic em-bedding vectors are extracted by the model with G . Then, features ofnodes are extracted by following these steps. First, one-hot vectors areextracted to represent categorical attributes and text attributes. Then,normalized vectors are extracted to represent the numerical attributes.Feature vectors are then concatenated by these two vectors. Thereafter,feature vectors are reduced to the same number of dimensions as basicembedding vectors. The embedding representation is concatenated byreduced feature vectors and basic embedding vectors, yielding a highdimensional vector for each node. Output:
A set of vectors of nodes is generated.
The structure component is used to construct the connection rela-tionships of a graph. The raw connection relationships are sensitive.Therefore, the structures are restructured from the embeddings of thegraph.
Input:
The input includes a graph G = ( V , E ) with A , an embeddingmethod, and corresponding parameters. Extraction:
The embedding can be extracted by the embeddingcomponent or by specified embedding models. Then, the distance ma-trix is calculated by node embeddings. The edges among nodes are ex-tracted by the user-specified reconstruction method with the distancematrix. The reconstruction is controlled by predefined parameters toavoid privacy disclosure. Here, different embedding methods can beused to support different analysis tasks. Some methods are strong inlink prediction and graph reconstruction, while some are good at nodeclustering. This component and embedding component can use differ-ent embedding methods for specified analysis tasks.
Output:
Reconstructed graph structures are output. lobal
FGRM
Embedding Component Client-side UpdateStructure Component Federated InitializationServer-side Update Attribute Component
Server
Federated Representation Federated Computation
Visual Interface
Embedding ViewAttribute ViewStructure ViewServer ViewClient ViewLocal Graph 1 Local
FGRM
Client 1
Local Graph 2 Local
FGRM
Client 2
Local Graph N Local
FGRM
Client N Figure 2: Overview of our approach. The server distributes FGRM to clients. Each client updates FGRM and sends graph representationsto the server. The server collects and handles graph representations and sends them back to each client. After a certain number of iterations,federated graph representations of multi-party graphs are generated. The visual interface is used to configure FGRM and illustrate federatedgraph representations.
The attribute component is used to construct the attribute distributionof a graph. Attribute distributions are extracted to avoid privacy dis-closure.
Input:
The input includes a graph G = ( V , E ) with attributes A , thebin size of the attribute distribution and filter condition. Extraction:
For numeric attributes, the bin size of the distributionis specified by users. For categorical attributes, the bin size of thedistribution is the dimensions of attributes. Those data types that areless meaningful for counting (e.g., name ) are not used. Moreover, thetopology attributes can also be extracted. These attributes are impor-tant to the analysis of the nodes of a graph, such as identifying thesocial influence in the social media network. To avoid privacy expo-sure, the extraction only constructs topology distributions rather thantopology values. The distribution of the topology attribute of one nodecharacterizes the node. The supported topology metrics include: (1)
Degree , (2)
Betweenness , (3)
Eigenvector , (4)
PageRank , (5)
Cluster-ing Coefficient , (6)
Average Nearest Neighbors Degree (KNN) . Thiscomponent will filter out specific nodes according to the filter condi-tions, and count the attribute distribution of these nodes (e.g., extract-ing the attribute distribution of players whose age is between 10 and30).
Output:
The attribute distribution of each node is extracted.Note that other graph representation components can also be de-signed for specified graph visual analysis tasks.
Federated computation is used to federate graphs and generate jointrepresentations. The process contains three steps: federated initializa-tion, server-side update, and client-side update.
Federated initialization sets the configuration of the federation model:encryption schemes, rules for computing feature vectors, and the dis-tribution of the model.
Encryption:
We provide different encryption schemes to protectthe transmission of data, although the transmitted data contains no sen-sitive information. Sometimes, the data owner still has concerns aboutdata privacy, so our model employs encrypted federated average andencrypted model training [54]. Specifically, our model is implementedwith the federated functions of TensorFlow (e.g., federated mean and federated max ). TensorFlow uses homomorphic encryption [38] to se-cure the transmission process.Attributes of nodes can be used to specify an individual’s identityuniquely. A traditional way of protecting privacy is to transfer onlyattribute distributions of nodes. Unfortunately, the identity can be re-identified by exploiting the side information or schematic meaning of data [41,51]. Thus, FGRM employs multiple attribute distribution pro-tection models to improve results: (1) syntactic anonymization mod-els: k -anonymity [42] and l -diversity [27] [22]. (2) differential privacymodels: Laplace mechanism [11] and exponential mechanism [30]. Computing feature vectors:
The server counts fields of categori-cal attributes from all clients, and sets the corresponding one-hot vec-tor for each field. Then, the server counts the maximum and mini-mum values of each numerical attribute and formulate the normaliza-tion standard of each attribute. Finally, each client calculates featurevectors of each node based on computing rules.
The distribution of FGRM:
The server distributes our FGRM andrelevant settings to each client. The weights of the model (the hiddenlayer) are the embedding results of the graph. The weights composean N × M matrix, where the row number N is the number of differentnodes of all graphs, and the column number M is the dimensions ofthe vector of a node. M can be configured by users. A row presentsthe embedding representation of a node. Models in both clients andthe server have the same weight. However, the nodes of local graphsare different. To find the embedding of the row corresponding to eachnode, the server unifies the row index of each node of all clients. Itshould be noted that the node counts of graphs in clients are differ-ent. The model makes statistics of the number of graph nodes from allclients and the server, and then sets the index of the row of the matrixcorresponding to each node. Client-side update runs on each client. For the embedding componentand the structure component, the server distributes a graph represen-tation model and the initial weights of the model to each client inthe federated initiation. Each client executes the model with an ini-tial weight. Then, each client learns the graph representation from thelearning model with the local graph per round, and calculates the gra-dients that encode the differences between weight pairs between twoweights. Thereafter, each client sends gradients to the server. This pro-cess does not transmit the raw data, but only transfers the gradients ofthe learning model.Each client calculates W att by a user-specified attribute distributionprotection model. Then, W att is encrypted and transmitted to the serverby means of TensorFlow. Note that, secure aggregation protocols andhomomorphic encryption algorithms [2] are supported by TensorFlow. The server randomly generates the weights of the embedding learn-ing model and distributes the model with weights to each client. Theserver collects gradients of models from clients to fulfill federated av-erage per round. Then the server computes the weighted average ofgradients according to the node number of each client. Next, newweights are computed based on weights and averaged gradients, andare sent back to each client. The server executes the weights updating lgorithm 1
The update in the k th client Input: W : weights sent by the server-side, η : the learning rate of thelearning model; Model : the embedding model.
Output: ∆ : gradients of FGRM, n k : the updated weight of the client. function C LIENT U PDATING ( W , Model , η ) W ← S ERVER E XECUTES () β ← (Local data is divided into minibatches) n k ← | β | // The updated weight is used to do the weightedaverage W init ← W for batch b ∈ β do W ← W − η∇ Model ( W ; b ) end for ∆ k ← W − W init return ∆ k , n k end functionAlgorithm 2 The attribute extraction in the k th client Input:
ADPM : the attribute distribution protection model.
Output: E att k : the encrypted attribute representation. function CLIENT A TTRIBUTE ( ADPM ) W att k ← ADPM ( G k , A k ) E att k ← TTF . FEDERATED . MAP ( W att k ) // TensorFlow transmitsit to the server based on encryption schemes. return E att k end function process iteratively until the specified number of rounds is reached. Theweights of the model, W emb , are learned from graphs in clients. Thistransmission process does not transmit raw data.The federated structure representations W struc is generated based on W emb with link prediction and graph restructuration algorithms. Forinstance, the distance matrix of nodes is calculated. Edges can begenerated by calculating the nearest | E | node pairs. The federatedattribute representation W att is calculated by means of TensorFlow. ISUAL I NTERFACE
Figure 1 shows the interface of our system. It consists of five views: aserver view (Figure 1 (A)) that provides FGRM configuration (Figure 1(A1)), data selection (Figure 1 (A2)) and model monitoring (Figure 1(A3)); a client view (Figure 1 (B)) that shows the information and theprocess state of clients; an embedding view (Figure 1 (C)) that showsfederated embedding representations; a structure view (Figure 1 (D))that shows federated structure representations: and an attribute view(Figure 1 (E)) that visualizes federated attribute representations.
Within the server view (Figure 1 (A)), users can configure graph learn-ing models, multiple parameters and encryption schemes (Figure 1(A1)), and monitor the running process of FGRM (Figure 1 (A3)). Theloss and the accuracy of FGRM are shown in a line chart (Figure 1(A3)). Users can choose the federated graph representation anytime ina training process and visualize them in other views (Figure 1 (F1)).Through these visual graphs, users can explore the training representa-tion, verify the accuracy of the representation, and evaluate the modelsand parameters.
In the client view (Figure 1 (B)), users can observe the general in-formation, the running state, and the training process of each client.Users can select a client to examine its process state and visualize itsfederated graph representations in other views.
The embedding view contains (Figure 1 (C)) multiple visualizationcomponents associated with federated graph embedding representa-
Algorithm 3
Server-side update clients per round.
Input: K : the amount of clients sampled per update, T : the iterationcount, ADPM : the attribute distribution protection model, M emb :the embedding model of the embedding component, η M emb : thelearning rate of M emb , M struc : the embedding model of the struc-ture component, η M struc : the learning rate of M struc . Output: W emb : the federated embedding representation of FGRM, W att : the federated attribute representation of FGRM, W struc : thefederated structure representation of FGRM. function S ERVER E XECUTES ( K , T , M emb , M struc ) W emb ← G ET E MBEDDING ( K , T , M emb , η M emb ) W M struc ← G ET E MBEDDING ( K , T , M struc , η M struc ) W struc ← R ESTRUCTURE ( W M struc ) for each client k = , ,..., K do E att k ← CLIENT A TTRIBUTE ( ADPM ) end for W att ← TTF . FEDERATED . MEAN ( E att , E att ,..., E att K ) // Ten-sorFlow calculates the average distributions of clients based onencryption schemes. return W emb , W att , W struc end function function G ET E MBEDDING ( K , T , Model , η ) initialize W randomly. for each round t = , ,..., T do ( ∆ kt , n kt ) ← C LIENT U PDATING ( W t , Model , η ) from the k thclient W t ← ∑ k ∆ kt // Sum the weighted average n t ← ∑ k N kt ∆ t ← W t n t // Updating the weighted average based on F W t + ← W t + ∆ t end for return W end function tions. Following visual graphs are supported: the projection view (Fig-ure 1 (C1)), the clustering view (Figure 1 (C2)), and the anomaly view((Figure 1 (C3))). Projection view:
This view shows the distance of nodes of multi-party graphs in high-dimensional embedding space. Each point in theview represents a node (Figure 1 (C1)). The node color encodes clus-tering information or anomaly information. Interactive actions such aspanning, zooming, and lasso-based selection are supported. Users canselect nodes to study attribute distributions in the attribute view andtheir reconstructed structures in the structure view.
Cluster view:
This view uses a table to visualize the clusteringresult of nodes from multi-client graphs according to federated embed-ding representations(Figure 1 (C2)). Users can select nodes from oneof the clusters for visualizing their attribute distributions and recon-structed structures in other views (Figure 1 (D2) (E2)).
Anomaly view : This view lists the anomaly detection results ofmulti-party graphs based on federated embedding representations (Fig-ure 1 (C3)). When users select some nodes, their attribute distributionsand reconstructed structures are shown in other views.
Control panel:
With the control panel, users can configure param-eters or methods in each view (in the top right corner). In the pro-jection view, different projection methods including MDS [10], and t -SNE [26] can be chosen. In the cluster view, different node cluster-ing algorithms and corresponding parameters, including K -Means [17]and DBSCAN [12] can be set. In the anomaly view, One-ClassSVM [9] or IsolationForest [23] can be used for nodes’ anomaly de-tection. The structure view (Figure 1 (D)) depicts reconstructed structures ofselected nodes from other views. To make structure exploration clearand avoiding heavy overlapping, this view hides nodes without anyedge. Different layout methods in the view can be chosen. .5 The Attribute View
In the attribute view (Figure 1 (E)), multiple histograms are used tovisualize the attribute distribution of multi-party graphs. Users caninteract with bins of histograms (Figure 1 (F3)) to specify attributedistributions in the corresponding attribute interval. The y -axis in eachhistogram can be either linear or logarithmic. VALUATION
In implementing GraphFederator, the front-end client was developedwith the React framework and D3.js. Our in-house graph visualizationengine is employed with rich user interactions, and flexible customiza-tions. TensorFlow is used to compute the federated average of theattribute component. Pytorch is used to execute the graph embeddinglearning model.
Datasets.
DBLP Dataset:
This is a paper citation graph dataset.Each node represents a paper, and an edge represents the citation rela-tionship between two papers, which has 5 attributes. Papers and theircitation graphs in four areas are extracted to form a graph: AI, System,Theory, and Interdisciplinary. The total number of papers in each areaare 64,232, 62,020, 14,430, and 62,708, respectively.
NetEase-Game-Player Dataset (NEGP):
This is a game playertransaction graph dataset provided by NetEase Co. It is collectedin five servers of a massively multi-player online role playing game.In each graph, a node represents a player, and an edge represents atransaction between two players. Each player has 36 attributes relatedto the player’s role information and account status, such as role level , role class and create date .Dataset Client ID We conducted several experiments to evaluate our approach. Experi-ments were conducted in a PC with a single Intel (R) Xeon (R) Gold5218 CPU (a basic frequency of 2.3GHz and 16 cores), 128GB inter-nal memory, and single RTX 2080TI GPU.
Considering that the federated scheme may have a lower quality of em-bedding representations, the performance and computing cost with ourapproach and its centralized counterpart were tested on two datasetsDBLP [43] and NEGP.
Configurations.
We ran three configurations to evaluate the per-formance of different models: embedding learning in a single client(ELSC), centralized embedding learning (CEL), and our federated em-bedding component (FEC).
Data processing.
DBLP Dataset [43]:
The citation graphs of years2014-2018 were used in our experiments with CEL and FEC, each ofwhich located in a client. The abstract of each paper was encoded asa fix-length vector based on word frequency count. For ELSC, theembedding generated from the graph of the year 2016, which is thelargest graph among them, was used to generate the performance base-line (ELSC). The sub-filed information was employed as the groundtruth to evaluate embedding representations with three configurations. http://game.163.com Dataset Model Configuration Accuracy TimeDBLP DeepWalk ELSC 89.04 2.3hCEL 91.87 8.1hFEC 90.13 4.4hGAT ELSC 86.74 0.4hCEL - -FEC 88.35 1.1hNEGP DeepWalk ELSC 85.26 2.6hCEL 85.62 8.7hFEC 85.86 5.1hGAT ELSC 86.11 0.9hCEL - -FEC 87.59 1.7hTable 2: The performance (in hours) with three configurations onthe DBLP and NEGP datasets, with two embedding learning models:DeepWalk and GAT.
NEGP Dataset : The game player transaction graphs from fiveservers were evaluated with CEL and FEC. The graph from gameserver 1 was used to derive ELSC. Players were classified into two cat-egories: high and low levels of in-game consumption. The consump-tion information was used as the ground truth to evaluate embeddingrepresentations with three configurations.
Settings.
Two popular embedding learning methods were em-ployed: DeepWalk [35], an unsupervised approach for learning noderepresentation without utilizing attribute vectors; and GAT [46], oneof the most popular neural graph architectures of graphs, which cap-tures both the structure and the feature of each node. The random walkstep of DeepWalk was firstly applied to generate samples by setting thenumber of walk times to be 80, and the length of walks to be 40. Deep-Walk learned node representations by using the skip-gram model. Weset the dimension of representation and the length of the sliding win-dow to be 128 and 10, respectively. The GAT model was configuredas follows: the number of layers was 3; the intermediate dimensionof representations was 256; three heads were used in GAT. To trainGAT, we employed the stochastic gradient descent (SGD) algorithm.The learning rate and the coefficient of L2 regularization were set tobe 0.001 and 0.0001, respectively. DeepWalk and GAT of all clientswere updated with 300 rounds.We tested three configurations on the DBLP and NEGP datasets.GAT has high computational complexity and memory consumption.The CEL was not conducted on GAT. All evaluations were derived bya 5-fold cross-validation.
Results.
The accuracy and time consumption are shown in Table 2.The accuracy of FEC has a similar performance with that of CEL, andthe accuracy of FEC or CEL is better than that of SCEL. The time con-sumption of FEC is more than that of SCEL, and less than that of CEL.Line charts of the loss of our approach are shown in Figure 3. Projec-tions of test dataset embeddings extracted by our approach are shownin Figure 4 by using t -SNE. In the first half of the training process, theembedding already has good adequate performance. Users can earlyterminate the training by observing the visualization of the results inreal-time. We tested the computing cost of the federated average in the attributecomponent. For the attribute protection model, the time consumptionis very short. The federated average supported by TensorFlow wasused to encrypt attributes distributions of all clients. The time consum-ing of the encrypt could not be ignored.
Settings.
The computing cost of three aspects was collected: thenumber of clients (1-30), the number of attributes (1-30), and the nodenumber in each client (1-50,000). We evaluated one aspect while fix-ing the other two. Each evaluation was repeated for five times.
Results.
The influence of each variable on the computing cost is re-ported in Figure 5. The computing cost of the extaraction of federatedattribute representations shows a linear complexity over the number of
10 30 50 70 90 120 150 180 210 240 270 300 011 11 11 11 0.25 0.5 0.75 1 10 30 50 70 90 120 150 180 210 240 270 3000255075100 10 30 50 70 90 120 150 180 210 240 270 30000.250.50.751 10 30 50 70 90 120 150 180 210 240 270 3000255075100 10 30 50 70 90 120 150 180 210 240 270 30000.250.50.751
DBLP GAT DBLP DeepWalkDBLP DeepWalkDBLP GAT NEGP GATNEGP GAT NEGP DeepWalkNEGP DeepWalk
10 30 50 70 90 120 150 180 210 240 270 300025507510010 30 50 70 90 120 150 180 210 240 270 3000255075100 10 30 50 70 90 120 150 180 210 240 270 30000.40.81.5
Loss Round Round Round RoundRound Round Round RoundAccuracy Accuracy Accuracy AccuracyLoss Loss Loss Client 1Client 2Client 3Client 4Client 5Federated
Figure 3: The loss and the accuracy of each client of federated embedding representations with FEC in each client on the DBLP and NEGPdatasets, with two embedding learning models: GAT and DeepWalk.
Round =240 Round =300Rround = 30 Round = 60 Round = 180DBLP GATNEGP GATRound = 120 Round =240 Round =300Rround = 30 Round = 60 Round = 180Round = 120 LabelsLabels
Figure 4: Projections ( t -SNE) of federated embedding representationswith FEC in different training rounds on test datasets of DBLP andNEGP with GAT. The color encodes ground truth labels.clients and attributes (Figure 5 (A) (B)) but has little relevance withthe node number in each client (Figure 5 (C)). The number of attributesComputing time in seconds
The number of clientsComputing time in seconds
Computing time in seconds (A) (B) (C)
The number of attributes = 10 The number of attributes = 10The node number in each client = 500 The node number in each client = 500 The node number in each clientThe number of clients = 10 The number of clients = 10
Figure 5: The computing cost of creating federated attribute repre-sentations. The computing cost increases linearly with the number ofclients (A) and the number of attributes (B), but has little relevancewith the node number in each client (C).
We evaluated the performance of federated structure representation bylink prediction evaluation metrics: the Area Under Curve (AUC) scoreand Precision [25]. Deepwalk was selected because it precisely cap-tures the linkages among nodes from the node sequence generated byrandom walk. Supervised models such as GAT seek to minimize dif-ference among intra-class nodes, and may lead to a result that node em- beddings of same category are concentrative in space, which is prob-lematic for linkage reconstruction.
Setting.
We used training representations of the entire dataset totest the performance. We selected 10,000,000 edge pairs to test AUCscore and set L of Precision as 1000. Results.
Given G = ( V , E ) and W emb , the distance matrix of nodesis calculated, and the time complexity is O ( | V | ) . Edges can be gen-erated by calculating the nearest | E | node pairs. Its time complex-ity is O ( | V | log ( | E | )) by using the heap sorting technique. The totaltime complexity O ( | V | + | V | log ( | E | )) . Results are show in Table 3.AUC score is high on two datasets. Precision is not satisfactory on theNEGP dataset, probably because the dataset has a high data compli-cacy. Dataset Model AUC PrecisionDBLP DeepWalk 99.89 94.7NEGP DeepWalk 98.98 76.5Table 3: The performance of federated structure representations forthe DBLP and NEGP datasets. We invited an expert to use our GraphFederator to analyze the gamedata. He works in NetEase and is skilled at game data analysis. Weintroduced our system and showed how it works. Then, he usedGraphFederator to analyze and explore NEGP datasets freely. His in-terest was in verifying the validity of the learned model and anomaloustrades with GraphFederator. He took graphs from four different gameservers as the input data (clients) of FGRM in the server view (Figure 1(A2)). Then, he configured FGRM to build federated graph representa-tions (Figure 1 (A1)). The general information of the graph from eachclient is shown in the client view (Figure 1 (B)). The running statusand the progress of FGRM are shown in the monitoring view (Figure 1(A3)). The monitor information indicates that the model runs well.With the obtained representations of all components after 300rounds of training, the expert saw clear clusters of the embedding inthe projection view (Figure 1 (C1)). The structure view showed clearand diverse structures. He concluded that FGRM performed well. Hestudied the distributions of the main attributes of players (Figure 7(a1-5)) and found that the distributions of several attributes are inter-esting. While most
Role total score (players’ scores) are low, and a nomaly detection
Game server 1
Anomaly detection
Game server 2
Anomaly detection
Game server 3
Anomaly detection
Game server 4(A) (B) (C) (D) (E) (F) (G)
Interactions with a lasso tool
Game servers 1-4
Anomaly detection
Detection rate of anomalous trades:
Detection rate of anomalous trades:
Detection rate of anomalous trades: 26.3% Detection rate of anomalous trades: 40.2% Detection rate of anomalous trades: 40.03% Detection rate of anomalous trades: 32.23%Interactions with the attribute view
Game servers 1-4 Game servers 1-4
Figure 6: Anomaly detection with GraphFederator for the NEGP dataset. (A): reconstructed structures of anomaly detection by using federatedgraph representations from game servers 1-4; (B): reconstructed structures are interactively modified in the attribute view; (C): the selectedstructures are highlighted with a lasso tool; (D)-(G): raw structures of anomaly detection by using the graph representation from a single gameserver. By checking with the ground truth, detection rates of anomalous trades of FGRM detection and GraphFederator with interactions aremuch higher than only using a single game server graph.small number of players have scores distributed in the highest interval.This result indicates that a small number of players’ game progressleads to the vast majority. The distribution of
Role equip score (play-ers’ equipment score) and the distribution of
Degree (the number oftrades of a player) are generally in a relatively low range (Figure 7(a2) (a5)), and imply that most players have low equipment scores andtrading records. These distributions helped the expert better under-stand the upgrading speed of the game, the consumption preferencesof the players, and the preferences of players in different activities.He selected 20,526 detected anomalous players (Figure 1 (F2)) tostudy anomalies in in-game trading behavior from the anomaly view.He found that many anomalous players had no trading record in thestructure view (Figure 1 (D1)), and many players’ accounts werebanned (Figure 1 (E1)) in the attribute view. Apparently, these play-ers behaved differently from the rest of all players (Figure 7 (a1)). Itshould be noted that the status of being banned is one of the most im-portant attributes for game analysis because it indicates that a playermay have an illegal plugin or other behaviors that violate game fairnesspolicies [44, 45]. He selected banned players from the correspond-ing histogram (Figure 1 (F3)), and the structure view showed tradesamong 1,122 players who were banned and detected as anomalies (Fig-ure 1 (D2)). There are 462 trades among 465 players. It indicates thatsome banned players knew each other and may belong to a studio thatcontrols the accounts of many plugin players. In addition, these play-ers may accumulate their virtual wealth in the game onto a specificplayer’s account. He selected three structures (Figure 1 (D2)) becausehe believed that the selected structures were typical anomalous trad-ing patterns based on his domain knowledge. The short and intensivetrading chain is the unique characteristic of anomalous trades.The
Degree distribution showed that some banned players mademany trades (Figure 1 (E2)). Interestingly, he found that kjjf (cumu-lative score of an activity) distributions were distributed in the lowestinterval (Figure 1 (E2)), compared with those of all players (Figure 7(a4)). He gave two possible explanations. The first one is that bannedplayers were banned earlier, and they had no chance to accumulatescores. The second is that some plugin players seek to make moneyor upgrade the level. They made no contribution to kjjf , which is anindicator of entertainment activities.The three structures with 66 players and 168 trades (Figure 6 (C))were also presented to the expert. 113 trades are anomalous trades.The expert was surprised that our system achieved a high detectionrate of anomalous trades. Raw graph data of each game server wasused to learn embeddings and detect anomalous trades. We found thattrades among these players are very intensive (Figure 6 (D)-(G)). Werecorded structures of each step analysis in this case study. These re-sults were presented to the expert. Using FGRM to detect anomaloustrades from four game servers, yields detection rate of 50.90% (Fig-ure 6 (A)). However, when a single game server data is employed, the detection rate are much lower (Figure 6 (D)-(G)). This indicatesthat federated graph representations extracted from multi-partygraphs can capture more underlying insights .In this case study, the expert achieved a detection rate of 67.26%(Figure 6 (C)). GraphFederator empowers him to improve the detec-tion rate of anomalous trades and discover anomaly trading patterns.
Role_equip_score . . . . . . . . E + 41010010
Ban_status
Kjjf Degree
E + 4 . . . . . . a1 a2 a3 a4 a5 Role_total_score
E + 3 . . . . . . . Figure 7: Attribute distributions of the NEGP dataset.
We invited a professor to use our GraphFederator to analyze the DBLPdataset. His research area is soical network analysis. He wanted toanalyze the clustering result of the paper citation data.He chose data from 2014 to 2018 and configured the embeddingcomponent with DeepWalk. He selected the training results of the lastround and observed federated graph representations in other views. Hemodulated different parameters of the K -Means algorithm from 3 to 6and selected 5 (Figure 8 (B)). Then, he selected different clusters. Byobserving the attribute view (Figure 8 (C)), he found differences of at-tributes: Ref num (the number of references),
N citation (the numberof citations), and
Average neighbor degree , the distributions of whichare shown in Figure 8 (C1). Clusters (a) and (d) have similar distri-butions with distributions of the entire set of papers. He inferred thatmost papers belong to these two clusters, so they have similar distribu-tions. However, the distributions of three clusters (b) (c) and (d) lie inthe lower interval. This indicates that those papers falling into theseclusters are rarely cited. He inferred that these papers were publishedrecently. The distributions in clusters (c) and (e) lie in the lowest in-terval. Clusters (c) and (e) have fewer papers than other clusters. The
Ref num and
N citation distributions indicate that the numbers of ci-tations and references of papers in these two clusters are small. The
Average neighbor degree implies that the number of citations and ref-erences of cited papers and reference papers is also small. He con-cluded that papers in two clusters are not attractive.We used raw data of each year from 2014-2016 to learnembeddings(Figure 8 (A2)-(A4)). Figure 8 (A1) shows the projectionof federated embedding representations of years 2014-2016. The colorencodes ground truth labels. We found that papers with the green la-bel are separated in 2014 and 2015, but clustered together in 2016.Papers in red circles (Figure 8 (A1)-(A4)) are clustered in terms of
A1) Year 2014-2018
Cluster a Cluster (A3)
Year 2015 (A2)
Year 2014 Year 2014-2018 KMeans = 5(A) (B) (C1) (C) (A4)
Year 2016
ClustersLabels b d c a2 N_citation a3 average_neighbor_degree10100 a1 Ref_num a e
N_citation average_neighbor_degree
Ref_numKMeans=5 b b1 Ref_num b2 N_citation b3 average_neighbor_degree c c1 Ref_num c2 N_citation c3 average_neighbor_degree Cluster d d1 Ref_num d2 N_citation d3 average_neighbor_degree Cluster e e1 Ref_num e2 N_citation e3 average_neighbor_degree Cluster (A2) ( )
Y 2015
Figure 8: Clustering results with GraphFederator for the DBLP dataset case study. (A1) and (B) show the same projection of federatedembedding representations on the DBLP dataset in 2014-2018 with DeepWalk. (A2), (A3) and (A4) show projections of representations byusing the data of 2014, 2015, 2016, respectively. Colors of projections in (A) encode 4 ground truth labels. Colors in (B) encode 5 clusterscalculated by the K -Means algorithm. (C) shows attribute distributions of the entire set (C1) and clusters. Areas of green and red circles showdistributions of selected papers from different representations extracted by different data.both data representations. It is regarded as a cluster, even though theyhave different labels. It indicates that federated graph representationsextracted from multi-party graphs can capture more features and infor-mation, and help find clusters with unique features. We interviewed the expert involved in our first case study (Sec-tion 7.2.1). The expert thought that the findings could help to improvestrategies of the anomalous player detection. He also believed thatthe clustering result of players of different clients might be used tostimulate more strategies for studying players of different clusters withunique features. The professor in the second case study (Section 7.2.2)suggested that our system supports the extraction and visualizations ofkeywords or abstracts of papers. He hoped that analysts should be ableto control privacy standards and analyze more data or dimensions; oth-erwise, some interesting insights would be missed.To evaluate the effectiveness of our approach, we also conductedone-on-one interviews with five additional domain experts fromNetEase Co. They are all skilled at game data analysis, graph analy-sis, and federated learning. With a live, hands-on demonstration forapproximately 10 minutes, we showed them case studies. We dis-cussed the feasibility of FGRM and findings and solicited feedbackfrom them. They all confirmed that our approach could help them an-alyze features and information of players without touching raw data,and GraphFederator empowers them to explore multiple aspects andfeatures of graphs. They liked our intuitive user interface for visualanalysis of large-scale graphs. One expert commented on our systemby saying: “ ...The system can help me validate training results of thefederated model by the visual interface and help accomplish variousanalysis tasks of multiple graphs. ” Another expert claimed: “ ...WhenI want to analyze graphs which are distributed in multiple clients, theaccessibility of graphs limits me. FGRM can solve these problems andgive wonderful visualizations of features from graphs. ” ISCUSSIONS
Privacy.
In the computing of federated representations, the servercomputes the weighted average of gradients of each client model andsends weights to each client. By jointly averaging gradients, the modelcan be trained by using multi-party graphs without switching rawdata. The federated average algorithm computes the attribute repre-sentations from each client graph with the encryption of TensorFlow.The encrypted algorithm prevents transmission data from being inter-cepted. Although the transmitted data contains no sensitive informa-tion, it is still possible that privacy-related data can be inferred from non-sensitive information. At the same time, to prevent the identifi-cation of individuals from attribute distributions, our model employsmultiple strategies like syntactic anonymization models and differen-tial privacy models. Structure representations are reconstructed by em-bedding representations, and reconstructed parameters could be usedto adjust accuracy. In fact, a small difference between raw structuresand reconstructed structures can protect privacy [50]. In the fields ofsecure multiple computing and homomorphic encryption, there are var-ious strategies to handle different privacy and security issues. Ourapproach is fully compatible with them and supports interactive con-figurations.
Expansibility.
Our approach can accomplish various graph visualanalysis tasks. Three components are employed for constructing differ-ent types of graph representations: embedding representation, attributerepresentation, and structure representation. Users can freely config-ure strategies, methods, and parameters of each component. As shownin case studies, users accomplished different analysis tasks for multi-party graphs, including anomaly detection, clustering, and comparison.Experts highly rated our approach in gaining and identifying patterns.Our approach also supports to design new components for construct-ing distinctive graph representations. Various visualization styles fordifferent graph representations can also be employed to fulfill a varietyof complex tasks.
Scalability.
Our approach constructs federated graph representa-tions from multi-party graphs with reasonable scalability. FGRMis compatible with different models and encrypt strategies for differ-ent tasks and requirements. We conducted multiple experiments tomeasure the performance of the federated graph representation (Sec-tion 7.1.1). The efficiency of FGRM in terms of data size depends onthe selected model. For the embedding component, GAT can only han-dle a moderate-sized data due to the use of matrix, and DeepWalk cansupport large-sized data because it uses the skip-gram technique. Theother two components also support extracting representations fromlarge-scale graphs. Our in-house visualization engine is amenable forvisualizing large-scaled graphs with rich user interactions.
Performance.
Our model indeed extracted high-quality federatedgraph representations from multi-party graphs. Federated representa-tions improve the efficiency of anomaly detection and clustering re-sults compared with using the representation extracted from singledata. GraphFederator with rich interactions empowers experts to accel-erate the process of detecting anomalous trades and comparing clustersof papers. There are three conclusions drawn from experiments.• Compared with the centralized counterpart, our approach cangenerate results with a similar quality, and achieve a better run-ing performance.• The graph can be reconstructed well with federated structure rep-resentations in terms of AUC score and precision, and the recon-structed structures keep differences to prevent privacy leaks.• The federated attribute representation can be constructed withrelatively low computing costs.
ONCLUSION
This paper presents GraphFederator, a federation approach that con-structs joint representations of multi-party graphs, and supportsprivacy-preserving visual analysis of multi-party graphs. In the fu-ture, we plan to explore various encryption strategies. We alsp planto extend our approach to other graph data. Currently, we assume thatmulti-party graphs have identical attributes. We will improve FGRMto support heterogeneous graphs. R EFERENCES [1] E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov. Howto backdoor federated learning.
Computing Research Repository ,abs/1807.00459, 2018.[2] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan,S. Patel, D. Ramage, A. Segal, and K. Seth. Practical secure aggrega-tion for privacy-preserving machine learning. In
Proceedings of ACMConference on Computer and Communications Security , pp. 1175–1191,2017.[3] M. Brehmer and T. Munzner. A multi-level typology of abstract visualiza-tion tasks.
IEEE Transactions on Visualization and Computer Graphics ,19(12):2376–2385, 2013.[4] H. Cai, V. W. Zheng, and K. C.-C. Chang. A comprehensive survey ofgraph embedding: Problems, techniques, and applications.
IEEE Trans-actions on Knowledge and Data Engineering , 30(9):1616–1637, 2018.[5] K. Canini, T. Chandra, E. Ie, J. McFadden, K. Goldman, M. Gunter,J. Harmsen, K. LeFevre, D. Lepikhin, T. L. Llinares, et al. Sibyl: A sys-tem for large scale supervised machine learning.
Technical Talk , 1:113,2012.[6] N. Cao, Y.-R. Lin, L. Li, and H. Tong. g -Miner: Interactive visual groupmining on multivariate graphs. In Proceedings of ACM Conference onHuman Factors in Computing Systems , pp. 279–288, 2015.[7] N. Cao, C. Shi, W. S. Lin, J. Lu, Y. Lin, and C. Lin. Targetvue: Visualanalysis of anomalous user behaviors in online communication systems.
IEEE Transactions on Visualization and Computer Graphics , 22(1):280–289, 2015.[8] K. M. Chandy and J. Misra. Distributed computation on graphs: Shortestpath algorithms.
Communications of the ACM , 25(11):833–837, 1982.[9] Y. Chen, X. S. Zhou, and T. S. Huang. One-class SVM for learning inimage retrieval. In
Proceedings of International Conference on ImageProcessing , pp. 34–37, 2001.[10] M. A. Cox and T. F. Cox. Multidimensional scaling. In
Handbook of datavisualization , pp. 315–347. Springer, 2008.[11] C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise tosensitivity in private data analysis. In
Proceedings of Theory of Cryptog-raphy Conference , pp. 265–284, 2006.[12] M. Ester, H. Kriegel, J. Sander, and X. Xu. A density-based algorithm fordiscovering clusters in large spatial databases with noise. In
Proceedingsof International Conference on Knowledge Discovery and Data Mining ,pp. 226–231, 1996.[13] O. Goldreich. Secure multi-party computation.
Manuscript. Preliminaryversion , 78, 1998.[14] S. Hong, S. Depner, T. Manhardt, J. Van Der Lugt, M. Verstraaten, andH. Chafi. PGX.D: A fast distributed graph processing engine. In
Pro-ceedings of International Conference for High Performance Computing,Networking, Storage and Analysis , pp. 1–12, 2015.[15] J. Koneˇcn`y, H. B. McMahan, D. Ramage, and P. Richt´arik. Federated op-timization: Distributed machine learning for on-device intelligence.
Com-puting Research Repository , abs/1610.02527, 2016.[16] J. Koneˇcn`y, H. B. McMahan, F. X. Yu, P. Richt´arik, A. T. Suresh, andD. Bacon. Federated learning: Strategies for improving communicationefficiency.
Computing Research Repository , abs/1610.05492, 2016.[17] K. Krishna and M. N. Murty. Genetic K-means algorithm.
IEEE Trans-actions on Systems, Man, and Cybernetics , 29(3):433–439, 1999. [18] A. Lalitha, O. C. Kilinc, T. Javidi, and F. Koushanfar. Peer-to-peer federated learning on graphs.
Computing Research Repository ,abs/1901.11173, 2019.[19] B. Lee, C. Plaisant, C. S. Parr, J.-D. Fekete, and N. Henry. Task taxonomyfor graph visualization. In
Proceedings of AVI Workshop on BEyond Timeand Errors: Novel Evaluation Methods for Information Visualization , pp.1–5, 2006.[20] K. Lee, M. Lam, R. Pedarsani, D. Papailiopoulos, and K. Ramchandran.Speeding up distributed machine learning using codes.
IEEE Transac-tions on Information Theory , 64(3):1514–1529, 2017.[21] M. Li, L. Zhou, Z. Yang, A. Li, F. Xia, D. G. Andersen, and A. Smola.Parameter server for distributed machine learning. In
Proceedings of BigLearning NIPS Workshop , p. 2, 2013.[22] N. Li, T. Li, and S. Venkatasubramanian. t -closeness: Privacy beyond k -anonymity and l -diversity. In Proceedings of International Conferenceon Data Engineering , pp. 106–115, 2007.[23] F. T. Liu, K. M. Ting, and Z.-H. Zhou. Isolation forest. In
Proceedings ofIEEE International Conference on Data Mining , pp. 413–422, 2008.[24] J. Liu, E. Bier, A. Wilson, J. A. Guerra-Gomez, T. Honda, K. Sricharan,L. Gilpin, and D. Davies. Graph analysis for detecting fraud, waste, andabuse in healthcare data.
AI Magazine , 37(2):33–46, 2016.[25] L. L¨u and T. Zhou. Link prediction in complex networks: A survey.
Physica A: statistical mechanics and its applications , 390(6):1150–1170,2011.[26] L. v. d. Maaten and G. Hinton. Visualizing data using t -SNE. Journal ofmachine learning research , 9(11):2579–2605, 2008.[27] A. Machanavajjhala, D. Kifer, J. Gehrke, and M. Venkitasubramaniam. l -diversity: Privacy beyond k -anonymity. ACM Transactions on KnowledgeDiscovery from Data , 1(1):3, 2007.[28] R. R. McCune, T. Weninger, and G. Madey. Thinking like a vertex: Asurvey of vertex-centric frameworks for large-scale distributed graph pro-cessing.
ACM Computing Surveys , 48(2):1–39, 2015.[29] H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas. Federatedlearning of deep networks using model averaging.
Computing ResearchRepository , abs/1602.05629, 2016.[30] F. McSherry and K. Talwar. Mechanism design via differential privacy.In
Proceedings of Symposium on Foundations of Computer Science , pp.94–103, 2007.[31] G. Mei, Z. Guo, S. Liu, and L. Pan. SGNN: A graph neural network basedfederated learning approach by hiding structure. In
Proceedings of IEEEInternational Conference on Big Data , pp. 2560–2568, 2019.[32] V. Mugunthan, A. Peraire-Bueno, and L. Kagal. PrivacyFL: A simula-tor for privacy-preserving and secure federated learning.
Computing Re-search Repository , abs/2002.08423, 2020.[33] C. Nobre, M. D. Meyer, M. Streit, and A. Lex. The state of the art invisualizing multivariate networks. In
Proceedings of Computer GraphicsForum , pp. 807–832, 2019.[34] C. Orlandi, A. Piva, and M. Barni. Oblivious neural network computingvia homomorphic encryption.
EURASIP Journal on Information Security ,2007:1–11, 2007.[35] B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning ofsocial representations. In
Proceedings of ACM International Conferenceon Knowledge Discovery and Data Mining , pp. 701–710, 2014.[36] R. Pienta, J. Abello, M. Kahng, and D. H. Chau. Scalable graph explo-ration and visualization: Sensemaking challenges and opportunities. In
Proceedings of International Conference on Big Data and Smart Comput-ing , pp. 271–278, 2015.[37] J. Pretorius, H. C. Purchase, and J. T. Stasko. Tasks for multivariatenetwork analysis. In
Proceedings of Multivariate Network Visualization ,pp. 77–95, 2013.[38] R. L. Rivest, L. Adleman, M. L. Dertouzos, et al. On data banks andprivacy homomorphisms.
Foundations of secure computation , 4(11):169–180, 1978.[39] R. Rossi and N. Ahmed. The network data repository with interactivegraph analytics and visualization. In
Proceedings of AAAI Conference onArtificial Intelligence , 2015.[40] B. Saket, P. Simonetto, and S. Kobourov. Group-level graph visualizationtaxonomy. In
Proceedings of Eurographics Conference on Visualization ,2014.[41] L. Sweeney. Simple demographics often identify people uniquely.
Health ,671:1–34, 2000.[42] L. Sweeney. k -anonymity: A model for protecting privacy. Interna-tional Journal of Uncertainty, Fuzziness and Knowledge-Based Systems ,0(5):557–570, 2002.[43] J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. ArnetMiner: ex-traction and mining of academic social networks. In
Proceedings of ACMInternational Conference on Knowledge Discovery and Data Mining , pp.990–998, 2008.[44] J. Tao, J. Lin, S. Zhang, S. Zhao, R. Wu, C. Fan, and P. Cui. MVAN:multi-view attention networks for real money trading detection in onlinegames. In
Proceedings of ACM International Conference on KnowledgeDiscovery and Data Mining , pp. 2536–2546, 2019.[45] J. Tao, J. Xu, L. Gong, Y. Li, C. Fan, and Z. Zhao. NGUARD: A gamebot detection framework for netease mmorpgs. In
Proceedings of ACMInternational Conference on Knowledge Discovery and Data Mining , pp.811–820, 2018.[46] P. Veliˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Ben-gio. Graph attention networks. In
Proceedings of International Confer-ence on Learning Representations , 2018.[47] J. Verbraeken, M. Wolting, J. Katzy, J. Kloppenburg, T. Verbelen, and J. S.Rellermeyer. A survey on distributed machine learning.
ACM ComputingSurveys , 53(2):33, 2020.[48] T. von Landesberger, A. Kuijper, T. Schreck, J. Kohlhammer, J. J. vanWijk, J. Fekete, and D. W. Fellner. Visual analysis of large graphs: State-of-the-art and future research challenges. In
Proceedings of ComputerGraphics Forum , pp. 1719–1749, 2011.[49] H. Wang, Y. Lu, S. T. Shutters, M. Steptoe, F. Wang, S. Landis, and R. Ma-ciejewski. A visual analytics framework for spatiotemporal trade networkanalysis.
IEEE Transactions on Visualization and Computer Graphics ,25(1):331–341, 2018.[50] X. Wang, W. Chen, J. Chou, C. Bryan, H. Guan, W. Chen, R. Pan, andK. Ma. Graphprotector: A visual interface for employing and assessingmultiple privacy preserving graph algorithms.
IEEE Transactions on Vi-sualization and Computer Graphics , 25(1):193–203, 2019. doi: 10.1109/TVCG.2018.2865021[51] X. Wang, J.-K. Chou, W. Chen, H. Guan, W. Chen, T. Lao, and K.-L.Ma. A utility-aware visual approach for anonymizing multi-attribute tab-ular data.
IEEE Transactions on Visualization and Computer Graphics ,24(1):351–360, 2017.[52] Y. Wu, N. Cao, D. Gotz, Y. Tan, and D. A. Keim. A survey on vi-sual analytics of social media data.
IEEE Transactions on Multimedia ,18(11):2135–2148, 2016.[53] W. Xiao, J. Xue, Y. Miao, Z. Li, C. Chen, M. Wu, W. Li, and L. Zhou.Tux : Distributed graph computation for machine learning. In Proceed-ings of USENIX Symposium on Networked Systems Design and Imple-mentation , pp. 669–682, 2017.[54] Q. Yang, Y. Liu, T. Chen, and Y. Tong. Federated machine learning:Concept and applications.
ACM Transactions on Intelligent Systems andTechnology , 10(2):1–19, 2019.[55] T. Zhang, X. Wang, Z. Li, F. Guo, Y. Ma, and W. Chen. A surveyof network anomaly visualization.
Science China Information Sciences ,60(12):121101:1–121101:17, 2017.[56] J. Zhao, M. Glueck, F. Chevalier, Y. Wu, and A. Khan. Egocentric anal-ysis of dynamic networks with egolines. In