SDGNN: Learning Node Representation for Signed Directed Networks
SSDGNN: Learning Node Representation for Signed Directed Networks
Junjie Huang, Huawei Shen ∗ , Liang Hou, Xueqi Cheng CAS Key Laboratory of Network Data Science and Technology, Institute of Computing Technology,Chinese Academy of Sciences, Beijing, ChinaUniversity of Chinese Academy of Sciences, Beijing, China { huangjunjie17s, shenhuawei, houliang17z, cxq } @ict.ac.cn Abstract
Network embedding is aimed at mapping nodes in a networkinto low-dimensional vector representations. Graph NeuralNetworks (GNNs) have received widespread attention andlead to state-of-the-art performance in learning node repre-sentations. However, most GNNs only work in unsigned net-works, where only positive links exist. It is not trivial totransfer these models to signed directed networks, which arewidely observed in the real world yet less studied. In thispaper, we first review two fundamental sociological theories(i.e., status theory and balance theory) and conduct empiricalstudies on real-world datasets to analyze the social mecha-nism in signed directed networks. Guided by related socio-logical theories, we propose a novel Signed Directed GraphNeural Networks model named SDGNN to learn node em-beddings for signed directed networks. The proposed modelsimultaneously reconstructs link signs, link directions, andsigned directed triangles. We validate our model’s effective-ness on five real-world datasets, which are commonly usedas the benchmark for signed network embeddings. Experi-ments demonstrate the proposed model outperforms existingmodels, including feature-based methods, network embed-ding methods, and several GNN methods.
Introduction
With the growing popularity of online social media, manyinteractions between people are generated and recorded onthe web. Modeling and understanding these interactions isa useful perspective on a range of social computing studies.Most of these interactions are positive relationships, suchas friend, trust, like, support, and approval. Meanwhile, theconflicts on the web (Kumar et al. 2018) are everywhere,forming negative links that indicate hostility, distrust, hate,disagree, and disapproval. Since the interactions are gen-erated by people and reflect people’s attitude and psychol-ogy, related researches on social psychology (e.g., statustheory and balance theory) shed light on the social mech-anism of signed networks. These two theories are widelyused in signed network analysis (Leskovec, Huttenlocher,and Kleinberg 2010a). In signed network analysis, link signprediction is to predict the sign of a given link. Leskovec,Huttenlocher, and Kleinberg carefully extract features based on social theories and achieve good performances. Link signprediction is the main downstream machine learning task forevaluating signed network embedding. Signed network em-bedding learning, which aims to map nodes in signed net-work to low-dimensional vector representations, also relieson these sociological theories (Wang et al. 2017; Kim et al.2018; Chen et al. 2018a; Islam, Prakash, and Ramakrishnan2018; Chen et al. 2018b). For example, Chen et al. mathe-matically model “bridge” edges based on balance and statustheory and achieve state-of-the-art performances.Recently, Graph Neural Networks (GNNs) have been re-ceiving more and more attention. Modern GNNs mostly fol-low Message Passing Neural Networks (MPNNs) (Gilmeret al. 2017) manner, which consist of message aggrega-tors and update functions. The common aggregators includemean aggregator (Hamilton, Ying, and Leskovec 2017),attention aggregator (Veliˇckovi´c et al. 2018), and max-pooling aggregator (Hamilton, Ying, and Leskovec 2017).GNNs have achieved good results in many machine learn-ing tasks (e.g., semi-supervised node classification, learninglow-dimensional representations, and link prediction) (Kipfand Welling 2019, 2016; Wang, Cui, and Zhu 2016). Anumber of recent researches have pivoted to learn the nodeembeddings using GNNs, which aim to aggregate informa-tion from neighbors for node embeddings (Kipf and Welling2016; Duran and Niepert 2017; Pan et al. 2018). TheseGNN-based network embedding methods have revolution-ized the field of network embedding and achieved state-of-the-art performances. Specifically, Graph Auto-Encoder(GAE) uses GCNs to process node features jointly with thegraph structure to produce a set of hidden representations(Kipf and Welling 2016). It uses a scoring function to recon-struct the adjacency matrix of the graph from hidden repre-sentations. GAE is designed for the unsigned networks andcan’t directly be applied in the signed networks.For signed network, SGCN generalizes GCN to signednetworks and designs a new information aggregator basedon balance theory (Derr, Ma, and Tang 2018). SNEA lever-ages the self-attention mechanism to enhance the perfor-mance of signed network embeddings (Li et al. 2020). Al-though signed GNNs discussed above are proposed to modelsigned networks with balance theory, they don’t take direc-tion (i.e., status theory) into consideration, which is impor-tant for signed graph modeling (Cui et al. 2020). In this pa- a r X i v : . [ c s . S I] M a r er, we try to model signed directed networks with a newSigned Directed Graph Neural Networks model (SDGNN).In comparison with traditional GNNs, we take related so-cial theories into account and redesign our aggregators andloss functions. For signed directed networks, we define fourdifferent signed directed relations. And we propose a layer-by-layer signed relation GNN to aggregate and propagatethe information of nodes in signed networks. For trainingour model, we reconstruct three important parts of signeddirected network: signs, directions, and triangles.The major contributions of this paper are as follows:• After reviewing the two fundamental social psychologytheories of signed network analysis, we conduct an empir-ical analysis of these theories on five real-world datasets.• We introduce a new layer-by-layer Signed Directed Rela-tion GNN model. It aggregates and propagates informa-tion between nodes under different signed directed rela-tion definitions. Guided by two sociological theories, ourloss functions consist of reconstructing signs, directions,and triangles.• We conduct link sign prediction experiments on five real-world signed social network datasets to demonstrate theeffectiveness of our proposed model. Related Work
Signed Network Embedding
Signed social networks are such social networks insigned social relations having both positive and negativesigns (Easley and Kleinberg 2010). To mine signed net-works, many algorithms have been developed for lots oftasks, such as community detection (Traag and Brugge-man 2009), node classification (Tang, Aggarwal, and Liu2016), link prediction (Leskovec, Huttenlocher, and Klein-berg 2010b), spectral graph analysis (Li et al. 2018), andgroup partition (Huang and Luo 2018). Recently, with thedevelopment of network representation learning (Perozzi,Al-Rfou, and Skiena 2014; Grover and Leskovec 2016; Tanget al. 2015), researchers begin to learn low-dimensional rep-resentations for signed networks. For signed network em-bedding methods, SNE (Yuan, Wu, and Xiang 2017) adoptsthe log-bilinear model and incorporates two signed-typevectors to capture the positive or negative relationship ofeach edge along the path. SiNE (Wang et al. 2017) de-signs an objective function guided by social theories to learnsigned network embeddings. SiNE proposes that social the-ories can provide a fundamental understanding of signed so-cial networks. For directed signed network, SIDE (Kim et al.2018) provides a linearly scalable method that leveragesbalance theory along with random walks. SIGNet (Islam,Prakash, and Ramakrishnan 2018) combines balance theorywith specialized random and new sampling techniques in di-rected signed networks. BESIDE (Chen et al. 2018a) mathe-matically models “bridge” edges based on balance and statustheory and achieves state-of-the-art performances.These methods are devoted to defining an objective func-tion that incorporates sociological theory and then using ma-chine learning techniques (e.g., sampling and random walks)to optimize look-up embeddings.
Graph Neural Networks
Today’s GNNs can be summarized as Message Passing Neu-ral Networks (MPNNs), including message functions andvertex update functions (Gilmer et al. 2017). Because of thenon-Euclidean data structure of the graph, traditional RNNsand CNNs are not easy to be used in the graph domains. Byusing GNNs, researchers have successfully applied convo-lution (Kipf and Welling 2019), attention (Veliˇckovi´c et al.2018), LSTM (Hamilton, Ying, and Leskovec 2017), andother mechanisms into the graph data. Specifically, GraphAuto-Encoders (Kipf and Welling 2016; Pan et al. 2018)are a family of models aimed at mapping (encoding) eachnode to low-dimensional vectors which reconstructing (de-coding) the graph. Compared to the network embeddingmethods, GNNs have a partial intersection but use the deeplearning methods instead of matrix factorization and ran-dom walk and can better describe the network structure andnode characteristics (Wu et al. 2020). A lot of GNN mod-els show a better performance than the shadow look-up em-beddings (Kipf and Welling 2019; Veliˇckovi´c et al. 2018;Hamilton, Ying, and Leskovec 2017).Most Graph Neural Networks (Kipf and Welling 2019;Veliˇckovi´c et al. 2018; Hamilton, Ying, and Leskovec 2017;Xu et al. 2019, 2020) are designed for unsigned social net-works whose links are only positive. How to apply graphneural networks to signed directed networks faces somechallenges (e.g., how to model the negative links and thedirections). SGCN (Derr, Ma, and Tang 2018) designs anew information aggregation and propagation mechanismfor the undirected signed networks according to balance the-ory. SGCN applies a mean-pooling strategy that is close toGraphSAGE (Hamilton, Ying, and Leskovec 2017) to learn-ing node embeddings. SiGAT (Huang et al. 2019) intro-duces GAT (Huang et al. 2019) to directed signed networksand designs a motif-based graph neural network modelbased on social theories. However, SiGAT uses 38 motifs,which is expensive for large graphs, and its objective func-tions can only model the sign, ignoring other vital features(e.g., directions and triangles).
Problem Definition
We define a signed directed network as G = ( V , E , s ) , where V is the set of nodes in a graph G , and E is the edge listwith signs s and directions. E consist of E + and E − while E + (cid:84) E − = ∅ ; E + and E − denote the sets of positive andnegative links, respectively. It can be denoted as the adja-cency matrix of the signed network A , where A ij = 1 meansthere exists a positive link from u i to u j , A ij = − denotesa negative link from u i to u j , and A ij = 0 means there isno link from u i to u j . Given a signed graph G = ( V , E , s ) ,our purpose is to map the nodes u ∈ V to low-dimensionalvectors z u ∈ R d as: f ( A ) → Z, (1)where Z ∈ R |V|× d is d -dimensional representations for the |V| nodes of the signed network, f is a learned transforma-tion function. ociological Theory Two sociological theories (i.e., Balance Theory and Sta-tus Theory) play an essential role in analyzing and mod-eling signed directed networks (Leskovec, Huttenlocher,and Kleinberg 2010b). In this section, we will briefly intro-duce these two theories and compare them in five real-worlddatasets. 𝑖 𝑗 𝑘 𝑖 𝑗 𝑘 𝑖 𝑗 𝑘 𝑖 𝑗 𝑘 (a) Balance theory ! " (b) Status theory
Figure 1: Illustrations of two sociological theories.
Balance Theory
Balance theory originated in the 1950s (Heider 1946) is ini-tially intended as a model for undirected signed networks.Triads with an even number of negative edges as balanced.In Figure 1(a), the triangles with three positive signs andthose with one positive sign (i.e., the first two triads inFigure 1(a)) are balanced. Balance theory posits that bal-anced triads are more plausible — and hence should be moreprevalent in real-world networks — than unbalanced triads.It exemplifies the principle that the friend of my friend is myfriend and the enemy of my enemy is my friend . Balance the-ory is widely used in the field of signed networks (Leskovec,Huttenlocher, and Kleinberg 2010a).
Status Theory
Status theory is another critical sociological theory in signednetwork analysis, which provides a different organizingprinciple for directed networks of signed links. It supposesthat a positive directed link “+” indicates that the creatorof the link views the recipient as having higher status; anda negative directed link “-” indicates that the recipient isviewed as having lower status (Leskovec, Huttenlocher,and Kleinberg 2010b). The status may denote the relativeprestige, ranking, or reputation. For example, a positive linkfrom A to B means not only
B is A’s friend but also
B has ahigher status than A . For the triangles in Figure 1(b), the firsttwo triads satisfy the status order, but the last two do not sat-isfy it. For the first triads, when
Status(j) > Status(i) and
Status(k) > Status(j) , we have
Status(k) > Status(i) . Comparison of Balance Theory and Status Theory
Balance theory focuses on undirected signed networks, al-though it has been applied to directed networks by simplydisregarding the directions (Wasserman, Faust et al. 1994).While, status theory normally reflects relations between twousers, which is based on directions. In some cases, thesetwo theories can be consistent with predictions. This phe-nomenon has received a lot of attention from researchers(Leskovec, Huttenlocher, and Kleinberg 2010b; Chen et al.2018a). Table 1: The ratio of triads satisfying balance and/or statustheory.
Dataset Both Only Balance Only Status NeitherBitcoin-Alpha 0.673 0.208 0.094 0.025Bitcoin-OTC 0.686 0.208 0.083 0.023Wikirfa 0.686 0.059 0.189 0.066Slashdot 0.751 0.167 0.066 0.016Epinions 0.769 0.156 0.066 0.009
We follow the related works and examine the percentageof triads satisfying balance and/or status theory on five real-world datasets used in this paper.From Table 1, we can find that only a tiny fraction of tri-angles satisfies neither of two theories. About 70% of triadscan be consistent with both theories. We think these trianglesthat satisfy neither theory maybe the noise for the signed net-work embedding representation. In addition, balance theorymodels the relationship among three vertices, and status the-ory capture the relationship between two vertices with thetransitivity property (Chen et al. 2018a). The complementa-tion of both theories can be the key point for signed directednetwork representation learning.
Proposed Method
Based on the previous discussion on sociological theory insigned directed networks, we will introduce how to designour new SDGNN model in this section.
Signed Directed GNNs
For unsigned networks, neighborhood nodes have thesame semantic relations. GraphSAGE (Hamilton, Ying, andLeskovec 2017) uses a mean aggregator to aggregate infor-mation from neighborhoods. As the weights in GraphSAGEonly determined by network structure, GAT (Veliˇckovi´cet al. 2018) learns the weights by structure masked self-attention mechanism. We adapt the above GNNs aggregatorsto signed directed networks.Figure 2 depicts the overall architecture of the proposedSDGNN model. For a signed directed graph, the directionsand signs between nodes reflect different relations and se-mantics. Different neighborhoods with different relationsshould be distinguished. We first define 4 different signeddirected relation (i.e., u → + v, u → − v, u ← + v, u ← − v ).Based on a signed directed relation (SDR) r i , we can getthe neighborhoods N r i . After that, we use different SDRGNN aggregators to aggregate the information from differ-ent neighborhoods and use an MLP to encode these mes-sages into node embeddings. GNN aggregators can be meanaggregators or attention aggregators. For a mean aggrega-tor, we get the information X lr i ( u ) and concatenate z lu with X l ( u ) by: X lr i ( u ) = σ ( W lr i · MEAN( { z lu } ∪ { z lv , ∀ v ∈ N r i ( u ) } )) z l +1 u = MLP(CONCAT( X l ( u ) , X lr ( u ) , ..., X lr i ( u ))) , (2)where z u is the embedding of node u , N r i ( u ) is the neigh-borhoods of u under the definition r i , W is the parameter, lu Positive LinkNegative Link S D R A g g S D R A g g S D R A gg S D R A gg X lr ( u ) X lr ( u ) z lu X lr ( u ) X lr ( u ) CONCAT MLP z l z l +1 u Figure 2: An Illustration of SDGNN model. We use Signed Directed Relation (SDR) aggregators to aggregate and propagatethe information of nodes.and σ is the activation function.For attention aggregators, we first compute α r i uv for thenode u and node v by the attention mechanism (cid:126) a r i andLeakyReLU nonlinearity activation function (with negativeinput slope α = 0.2) as: α r i uv = exp (cid:0) LeakyReLU (cid:0) (cid:126) a (cid:62) r i [ W lr i z lu (cid:107) W lr i z lv ] (cid:1)(cid:1)(cid:80) k ∈N ri ( u ) exp (cid:0) LeakyReLU (cid:0) (cid:126) a (cid:62) r i [ W lr i z lu (cid:107) W lr i z lk ] (cid:1)(cid:1) , (3)where (cid:107) is the concatenation operation, N r i ( u ) is the neigh-borhoods of node u under the definition of r i , W r i isthe weight matrix parameter. Then we get the information X lr i ( u ) and concatenate X lr i ( u ) with z u to get new embed-dings: X lr i ( u ) = (cid:88) v ∈N ri ( u ) α r i uv W r i z lv ,z l +1 u = MLP(CONCAT( X l ( u ) , X lr ( u ) , ..., X lr i ( u ))) . (4)In Figure 2, we use a layer-by-layer Signed Directed GNNto cover the direction and the topology of the triangle whenwe use multi-layers instead of one layer. Loss Function
For training our SDGNN model, we design three loss func-tions to reconstruct three critical features of signed net-works: sign, direction, and triangle.For sign, we use the following cross entropy loss functionto model the sign between two nodes: L sign ( u, v ) = − y u,v log (cid:16) σ ( z (cid:62) u z v ) (cid:17) − (1 − y u,v ) log (cid:16) − σ ( z (cid:62) u z v ) (cid:17) , L sign = (cid:88) e u,v ∈E L sign ( u, v ) , (5) where σ is the sigmoid function, y u,v is the sign groundtruth, and E is the edge list with signs.As we discussed before, status theory relies on the di-rections in signed networks. We denote the status rankingscore of node u , v as s ( z u ) , s ( z v ) . We use the followingsquare loss function to measure the difference between the predicted status relationship value s ( z u ) − s ( z v ) from theedge e uv and the “ground truth” value q uv : L direction ( u → v ) = ( q uv − ( s ( z u ) − s ( z v ))) q uv = (cid:26) max( s ( z u ) − s ( z v ) , γ ) u → v : − min( s ( z u ) − s ( z v ) , γ ) u → v : + L direction = (cid:88) e u,v ∈E L direction ( u → v ) , (6)where s ( z ) = sigmoid( W · z + b ) is a score function formapping embedding z to a score, γ is a threshold of statusrelationship value ( γ = 0 . in this paper), and E is the edgelist with signs.For triangles, we hope our model can learn from thetrue triangles distribution. For a triangle (e.g., (cid:52) i,j,k , i → + j, i → + k, k → + j ), we maximize the likelihood by: J (cid:52) i,j,k = P (+ | e ij ) ∗ P (+ | e ik ) ∗ P (+ | e kj ) . (7)Further, triangles objective function J tri as follows: J tri = (cid:89) (cid:52)∈ T J (cid:52) , L triangle = − log J tri = (cid:88) (cid:52)∈ T − log J (cid:52) = (cid:88) (cid:52)∈ T L (cid:52) , (8)where T is the set of triangles based on balanced and sta-tus theories. We can construct a triangle by three vertices( i, j, k ) by: L (cid:52) i,j,k = L ij + L ik + L kj , L ij = − y i,j log P (+ | e ij ) − (1 − y i,j )log(1 − P (+ | e ij ))= − y i,j log (cid:0) σ ( z (cid:62) i z j ) (cid:1) − (1 − y i,j ) log(1 − σ ( z (cid:62) i z j )) , (9)where y i,j is the sign ground truth for edge e ij . With Equa-tion 8 and Equation 9, we can reconstruct triangles by edgebinary cross entropy loss function. The weight of the edgeloss function is the number of edges in triangles.Based on sign, direction and triangle loss function, theoverall objective function is written as: L loss = L sign + λ L direction + λ L triangle , (10)where λ and λ are the weight of different loss functions.Equation 10 shows that our loss functions are designed toreconstruct the various properties of signed networks. xperiments In this section, we conduct link sign prediction to checkwhether our model improves the performance of signed net-work embeddings. Link sign prediction is the task of predict-ing the unobserved sign of existing edges in the test datasetgiven training dataset (Leskovec, Huttenlocher, and Klein-berg 2010a; Chen et al. 2018a). We follow their experimen-tal settings and compare our method against some state-of-the-art embedding methods.
Experimental Settings
Datasets
We do experiments on five real-world signedsocial network datasets (i.e., Bitcoin-Alpha , Bitcoin-otc ,Wikirfa , Slashdot and Epinions ). Bitcoin-Alpha andBitcoin-OTC (Kumar et al. 2016) are the who-trust-whomnetworks of people who trade using Bitcoin on platforms.In these datasets, members rate other members on a scaleof -10 (total distrust) to +10 (total trust) in steps of 1. Wetreat the scores greater than 0 as positive and others as neg-ative. Wikirfa (Kumar et al. 2016) is a signed network inwhich nodes represent Wikipedia members and edges repre-sent votes. It records the voting results for “request for ad-minship (RfA)”, where the community member can cast asupporting, neutral, or opposing vote for a Wikipedia ad-minship election. We remove the neural votes and constructa signed network (Chen et al. 2018a). Slashdot (Leskovec,Huttenlocher, and Kleinberg 2010b) is from a technology-related news website with user communities. The websiteintroduced Slashdot Zoo features that allow users to tageach other as friends or foes. The dataset is a commonsigned social network with friends and enemies labels. Epin-ions (Leskovec, Huttenlocher, and Kleinberg 2010b) is awho-trust-whom online social network of a consumer re-view site Epinions.com. Members of the site can indicatetheir trust or distrust of the reviews of others. The networkreflects people’s opinions on others.Table 2: Statistics of five datasets. Dataset
The statistics of five datasets are summarized in Table 2.For these five datasets, positive and negative links are imbal-anced (i.e., nearly 80% links are positive edges).
Baselines
To validate the effectiveness of SDGNN, wecompare it with a number of baselines including unsignednetwork embedding methods, signed embedding methodsand sigend graph neural networks. http://snap.stanford.edu/data/soc-sign-bitcoin-alpha.html http://snap.stanford.edu/data/soc-sign-bitcoin-otc.html http://snap.stanford.edu/data/wiki-RfA.html http://snap.stanford.edu/data/soc-sign-Slashdot090221.html http://snap.stanford.edu/data/soc-sign-epinions.html • Random: It generates d dimensional random values, z =( x , x , ..., x d ) , x i ∈ [0 . , . . It can be used to show theability of the downstream logistic regression.• Unsigned Network Embedding: We use several classi-cal unsigned network embedding methods to validate theeffectiveness of the structure: DeepWalk (Perozzi, Al-Rfou, and Skiena 2014), Node2vec (Grover and Leskovec2016), and LINE (Tang et al. 2015). Since these meth-ods cannot distinguish positive and negative edges, we re-move the negative links in the training stage.• Signed Network Embedding: In order to show the effec-tiveness of modeling signed directed edges, we use somesigned network embedding methods (e.g., , SiNE (Wanget al. 2017), SIGNet (Islam, Prakash, and Ramakrishnan2018), BESIDE (Chen et al. 2018a)). Specifically, SiNEis designed for undirected signed networks. SIGNet andBESIDE can model signed directed networks.• FeExtra (Leskovec, Huttenlocher, and Kleinberg 2010a):This method extracts two parts, a total of 23 features fromsigned directed social network. For each pair ( v i , v j ), thefirst one is the degree based, such as the number of in-coming positive and negative links of v i , the number ofoutgoing positive and negative links of v j and so on. Theother one is the structure information of the triad that con-tains v i and v j .• Graph Neural Network: For the baselines of GNNs,we choose SGCN (Derr, Ma, and Tang 2018) andSiGAT (Huang et al. 2019). SGCN makes a dedicated andprincipled effort that utilizes balance theory to correctlyaggregate and propagate the information across layers ofa undirected signed GCN model. SiGAT uses 38 motifsand GAT aggregators to model signed directed networks.For a fair comparison, the embedding dimension d is setto 20 for all methods except FeExtra. It is a common set-ting used in SiNE (Wang et al. 2017), BESIDE (Chen et al.2018a) and SiGAT (Huang et al. 2019). We use the authors’released code for DeepWalk , Node2vec , LINE , SiNE ,SIGNet , BESIDE , and SiGAT . For SGCN, we use thecode from github . We follow the authors’ suggested hyper-parameter settings. Like previous works (Kim et al. 2018;Wang et al. 2017; Derr, Ma, and Tang 2018), we first usethese methods to get node representations. For edge e ij , weconcatenate these two learned representation z i and z j tocompose an edge representation z ij . After that, we train alogistic regression classifier on the training set and use itto predict the edge sign in the test set. We randomly select80% edges as training set and the remaining 20% as the testset. We run with different train-test splits for 5 times to getthe average scores. Each training set is used to train both https://github.com/phanein/deepwalk https://github.com/aditya-grover/node2vec https://github.com/tangjianpku/LINE https://github.com/raihan2108/signet https://github.com/yqc01/BESIDE https://github.com/huangjunjie95/SiGAT https://github.com/benedekrozemberczki/SGCN able 3: The results of link sign prediction on five datasets. Random Unsigned Network Embedding Signed Network Embedding FeatureEngineering Graph Neural NetworkDataset Metric Random Deepwalk Node2vec LINE SiNE SIGNet BESIDE FeExtra SGCN SiGAT SDGNNBitcoin-Alpha Micro-F1 0.9367 0.9367 0.9355 0.9352 0.9458 0.9422 0.9489 0.9486 0.9256 0.9456
Binary-F1 0.9673 0.9673 0.9663 0.9664 0.9716 0.9696
AUC 0.6146 0.6409 0.7576 0.7114 0.8728 0.8908 0.8981 0.8882 0.8469 0.8872
Bitcoin-OTC Micro-F1 0.9000 0.8937 0.9089 0.8911 0.9095 0.9229 0.9320
AUC 0.6145 0.6596 0.7643 0.7248 0.8571 0.8935
Binary-F1 0.8762 0.8779 0.8719 0.8827 0.8972 0.9001 0.9117 0.8987 0.9069 0.9042
Macro-F1 0.4381 0.4666 0.5626 0.5738 0.7319 0.7384 0.7803 0.7235 0.7527 0.7535
AUC 0.5423 0.5876 0.6930 0.6772 0.8602 0.8682
Binary-F1 0.8728 0.8724 0.8528 0.8525 0.8918 0.8983 0.9105 0.9070 0.8926 0.9055
Macro-F1 0.4364 0.4384 0.5390 0.5052 0.7273 0.7554
AUC 0.5370 0.5408 0.6709 0.6145 0.8409 0.8752
Binary-F1 0.9204 0.9005 0.9170 0.9175 0.9525 0.9489 0.9615 0.9561 0.9486 0.9593
Macro-F1 0.4602 0.5131 0.6862 0.6305 0.8160 0.8060 0.8601 0.8130 0.8105 0.8454
AUC 0.5589 0.6702 0.8081 0.6835 0.8872 0.9095 0.9351 embedding vectors and logistic regression classifiers. Ourmodels are implemented by PyTorch with the Adam opti-mizer (
LearningRate = 0 . , WeightDecay = 0 . ).We use the 2-layer-GAT aggregators to build our model andset λ = 1 and λ = 1 . All experiments run on a computerwith Intel Xeon E5-2640 CPU and 128GB RAM, which in-stalls Linux CentOS 7.1. Experiment Results
We report the average Micro-F1, Binary-F1, Macro-F1, andAUC in Table 3 (Pedregosa et al. 2011). We have bolded thehighest value of each row and underlined the second value.From Table 3, we can find that:• For signed networks, link sign prediction is a positive andnegative imbalance classificiation problem. Even givenrandom embedding, logistic regression can be used as thedownstream machine learning methods.• After using unsigned network embedding methods, eventhe only positive links are used, the metrics have been in-creased. It means that the structural information matters.Node2vec has made the best results in the unsigned net-work embeddings methods.• Signed network embedding methods (i.e., SiNE, SIGNet,and BESIDE) are designed for signed network. The re-sults are significantly higher than other unsigned net-work embedding methods. These algorithms model re-lated sociological theories and had achieved good resultsfor signed network analysis. SiNE is designed for undi-rected signed networks based on balance theory. Its re-sults are not as good as SIGNet and BESIDE. AlthoughSIGNet can be applied to signed directed networks, itdoes not model status theory. This makes it is less effec-tive than BESIDE. BESIDE shows some good results in our experiments, which demonstrates that modeling twotheories is the key for the problem.• FeExtra method is a very classic method in the early daysand performs well because of its successful use of relevantsociological theories. However, it should be pointed outthat this method relies on feature engineering to extractfeatures manually and only model edge features withoutnode embeddings, so its generalization ability is weak.• SGCN shows a performance close to SiNE, but it can-not effectively model signed directed networks becausethe algorithm did not consider the direction. Besides, ituses the
MEAN aggregators which can not model the ef-fects of different neighborhoods. SiGAT is designed fordirected networks and uses the attention aggregators. Itsexperimental results are better than SGCN. But its de-coder design limits its expressiveness.• SDGNN outperforms all baseline methods in most termsof metrics. It demonstrates the ability of learning nodeembeddings using our SDGNN model. It is worth men-tioning that the experimental results depend on the down-stream classifier, and there may exist some inconsistentbetween F1 and AUC.
Parameter Analysis and Ablation Study
In this section, we investigate the effects of hyperparametersand do some ablation studies to analyze our model architec-ture design. We choose the Bitcoin-Alpha as our dataset andselect 80% training edges and 20% test edges as the previoussubsection does. We also use logistic regression function asour downstream machine learning classifier.
Parameter Analysis
In this subsection, we analyze thehyperparameters about the number of epoch and the embed-ding dimension d .able 4: Ablation study on different aggregators. Metric 2-Layer-GAT-AGG 1-Layer-GAT-AGG 2-Layer-MEAN-AGG 1-Layer-MEAN-AGGMicro-F1 0.9446 0.9442 0.9415 0.9399Binary-F1 0.9706 0.9703 0.9689 0.9679Macro-F1 0.7510 0.7516 0.7417 0.7411AUC 0.9154 0.9095 0.9041 0.9000
Table 5: Ablation study on loss functions.
Metric L sign L sign + L direction L sign + L triangle L sign + L direction + L triangle Micro-F1 0.9386 0.9438 0.9415 0.9475Binary-F1 0.9677 0.9702 0.9690 0.9721Macro-F1 0.6738 0.7414 0.7210 0.7585AUC 0.8883 0.9082 0.9030 0.9109
Epoch M a c r o - F Lo ss LossMacro-F1 (a) Epoch
Embedding Dimension d M a c r o - F (b) d Figure 3: Parameter analysis for sign prediction.When analyzing epoch, we set d = 20 , and record theMacro-F1 performance of different epoch generated repre-sentations and the loss value during training. Figure 3(a)shows that SDGNN converges quickly and keeps relativelystable performances. When discussing the dimension d , weset the corresponding Epoch to 100 to discuss the robust-ness of different dimensions. Figure 3(b) shows that the per-formance increases first and decreases in 100 (We decreasethe learning rate for the large d ). Even with a small embed-ding dimension like d = 5 , our model has already achievedpretty good performance which is close to BESIDE (Chenet al. 2018a). We find that too many parameters will causethe difficulties of training embeddings, which may be thereason for the decrease in large dimensions. Ablation Study
In this subsection, we do some abla-tion studies to discuss the design of aggregators and lossfunctions. As the same as previous subsection, we chooseBitcoin-Alpha as the dataset to analyze different architec-ture designs. For the aggregators, as we mentioned, mean ag-gregator and attention aggregator are common GNNs mod-els. We analyze whether the attention mechanism can per-form better than mean strategy. In addition, we also discussethe number of GNN layers to verify our layer-by-layer de-sign. Table 4 demonstrates that the attention mechanism per-forms better than the mean strategy. And the number of lay-ers can capture higher-order information, which can increasethe performance.For our loss functions, we can control λ and λ in Equa-tion 10 to discuss the effects of different objectives on the re- sults. When λ = 0 and λ = 0 , it degenerates to just recon-struct the sign of signed networks, which is close to SiGAT.In Table 5, we can find that only reconstructing signs us-ing L sign performs poorly. When considering direction andtriangle, the results are improved. It demonstrates that ourtraining objectives should take both directions and trianglesinto consideration. Signs, directions, and triangles are vitalfeatures for signed directed networks. Conclusion
In this paper, we investigate the signed directed network rep-resentations learning. We firstly analyze two fundamentalsocial theories (i.e., Balance Theory and Status Theory) inthe signed directed network. Guided by sociological theo-ries, we propose SDGNN to encode a signed network intonetwork embeddings. SDGNN aggregates messages fromdifferent signed directed relation definitions. It can applymulti-layers to capture high order structure information. Totrain our SDGNN, we introduce combined loss functions toreconstruct not only the signs but also other important fea-tures for signed directed networks (i.e., directions and tri-angles). We perform experiments on five real-world signednetworks and demonstrate that our proposed SDGNN per-forms better than other state-of-art baselines. We analyzethe hyperparameters and do some ablation studies to ana-lyze our design for aggregators and loss functions. In futurework, we will further generalize this method to heteroge-neous networks to incorporate more complex semantic in-formation (Schlichtkrull et al. 2018).
Acknowledgments
The authors would like to thank the AAAI reviewers for theirinsightful suggestions to improve the manuscript. This workis funded by the the National Key R&D Program of China2020AAA0105200 and National Natural Science Founda-tion of China under Grant No. 91746301 and 61802371.This work is supported by Beijing Academy of ArtificialIntelligence (BAAI) under Grant No. BAAI2019QN0304.Huawei Shen is also funded by K.C. Wong Education Foun-dation. eferences
Chen, Y.; Qian, T.; Liu, H.; and Sun, K. 2018a. ” Bridge”Enhanced Signed Directed Network Embedding. In
CIKM ,773–782.Chen, Y.; Qian, T.; Zhong, M.; and Li, X. 2018b. BASSI:Balance and Status Combined Signed Network Embedding.In
DASFAA , 55–63. Springer.Cui, J.; Zhuang, H.; Liu, T.; and Wang, H. 2020. Semi-Supervised Gated Spectral Convolution on a DirectedSigned Network.
IEEE Access
8: 49705–49716.Derr, T.; Ma, Y.; and Tang, J. 2018. Signed graph convolu-tional networks. In
ICDM , 929–934. IEEE.Duran, A. G.; and Niepert, M. 2017. Learning graph rep-resentations with embedding propagation. In
NIPS , 5119–5130.Easley, D.; and Kleinberg, J. 2010.
Networks, crowds, andmarkets: Reasoning about a highly connected world . Cam-bridge University Press.Gilmer, J.; Schoenholz, S. S.; Riley, P. F.; Vinyals, O.; andDahl, G. E. 2017. Neural message passing for Quantumchemistry. In
ICML , 1263–1272.Grover, A.; and Leskovec, J. 2016. node2vec: Scalable fea-ture learning for networks. In
KDD , 855–864. ACM.Hamilton, W.; Ying, Z.; and Leskovec, J. 2017. Inductiverepresentation learning on large graphs. In
NIPS , 1025–1035.Heider, F. 1946. Attitudes and cognitive organization.
TheJournal of psychology
FiCloudW , 95–101. IEEE.Huang, J.; Shen, H.; Hou, L.; and Cheng, X. 2019. Signedgraph attention networks. In
ICANN , 566–577. Springer.Islam, M. R.; Prakash, B. A.; and Ramakrishnan, N. 2018.Signet: Scalable embeddings for signed networks. In
PAKDD , 157–169. Springer.Kim, J.; Park, H.; Lee, J.-E.; and Kang, U. 2018. Side: rep-resentation learning in signed directed networks. In
WWW ,509–518.Kipf, T. N.; and Welling, M. 2016. Variational graph auto-encoders. In
NIPS Workshop on Bayesian Deep Learning .Kipf, T. N.; and Welling, M. 2019. Semi-supervised classi-fication with graph convolutional networks. In
ICLR .Kumar, S.; Hamilton, W. L.; Leskovec, J.; and Jurafsky, D.2018. Community interaction and conflict on the web. In
WWW , 933–943.Kumar, S.; Spezzano, F.; Subrahmanian, V.; and Faloutsos,C. 2016. Edge weight prediction in weighted signed net-works. In
ICDM , 221–230. IEEE.Leskovec, J.; Huttenlocher, D.; and Kleinberg, J. 2010a. Pre-dicting positive and negative links in online social networks.In
WWW , 641–650. ACM. Leskovec, J.; Huttenlocher, D.; and Kleinberg, J. 2010b.Signed networks in social media. In
CHI , 1361–1370. ACM.Li, Y.; Tian, Y.; Zhang, J.; and Chang, Y. 2020. LearningSigned Network Embedding via Graph Attention. In
AAAI ,4772–4779.Li, Y.; Yuan, S.; Wu, X.; and Lu, A. 2018. On spectral analy-sis of directed signed graphs.
International Journal of DataScience and Analytics
IJCAI .Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.;Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss,R.; Dubourg, V.; Vanderplas, J.; Passos, A.; Cournapeau, D.;Brucher, M.; Perrot, M.; and Duchesnay, E. 2011. Scikit-learn: Machine Learning in Python.
JMLR
12: 2825–2830.Perozzi, B.; Al-Rfou, R.; and Skiena, S. 2014. Deepwalk:Online learning of social representations. In
KDD , 701–710.ACM.Schlichtkrull, M.; Kipf, T. N.; Bloem, P.; Van Den Berg, R.;Titov, I.; and Welling, M. 2018. Modeling relational datawith graph convolutional networks. In
European SemanticWeb Conference , 593–607. Springer.Tang, J.; Aggarwal, C.; and Liu, H. 2016. Node classifica-tion in signed social networks. In
SIAM , 54–62. SIAM.Tang, J.; Qu, M.; Wang, M.; Zhang, M.; Yan, J.; and Mei, Q.2015. Line: Large-scale information network embedding. In
WWW , 1067–1077.Traag, V. A.; and Bruggeman, J. 2009. Community detec-tion in networks with positive and negative links.
PhysicalReview E
ICLR
URL https://openreview.net/forum?id=rJXMpikCZ.Wang, D.; Cui, P.; and Zhu, W. 2016. Structural deep net-work embedding. In
KDD , 1225–1234. ACM.Wang, S.; Tang, J.; Aggarwal, C.; Chang, Y.; and Liu, H.2017. Signed network embedding in social media. In
SIAM ,327–335. SIAM.Wasserman, S.; Faust, K.; et al. 1994.
Social network anal-ysis: Methods and applications , volume 8. Cambridge uni-versity press.Wu, Z.; Pan, S.; Chen, F.; Long, G.; Zhang, C.; and Philip,S. Y. 2020. A comprehensive survey on graph neural net-works.
IEEE Transactions on Neural Networks and Learn-ing Systems .Xu, B.; Huang, J.; Hou, L.; Shen, H.; Gao, J.; and Cheng, X.2020. Label-Consistency based Graph Neural Networks forSemi-supervised Node Classification. In
SIGIR , 1897–1900.Xu, B.; Shen, H.; Cao, Q.; Qiu, Y.; and Cheng, X. 2019.Graph wavelet neural network. In
ICLR .Yuan, S.; Wu, X.; and Xiang, Y. 2017. Sne: signed networkembedding. In
PAKDD , 183–195. Springer. ppendix
Comparison of Different Model Architectures
In general, our model is a layer-by-layer signed relationGNN model with social theories guided loss functions. Welist the difference between the proposed method with exist-ing baselines in Table 6.Table 6: Summary of selected methods.Method SDGNN BESIDE SiGATGNN Aggregator (cid:51) (cid:55) (cid:51)
Layer by Layer (cid:51) (cid:55) (cid:55)
Sign Loss (cid:51) (cid:55) (cid:51)
Direction Loss (cid:51) (cid:51) (cid:55)
Triangle Loss (cid:51) (cid:51) (cid:55)
Algorithm Detail
When our model work on a large signed directed graphwhose number of nodes is more than 100,000, we could notput the whole graph into our memory with L -layer GNN ag-gregators ( L ≥ ).To train our SGDNN model in such large networks, weuse mini-batch stochastic gradient descent to update the pa-rameters SDGNN. It needs to recombine the neighborhoodsof the nodes in the batch to achieves batch calculation. Inthis paper, the batch size is 500. The training procedure issummarized in Algorithm 1. Algorithm 1:
SDGNN Algorithm
Input:
Signed Directed Graph G ( V , E , s ) ;Encoder Aggregators Enc ;GNN Layer Number L ;Epoch T ; Output:
Node representation Z Prepare Original node embeddings z u , ∀ u ∈ V . Initialize the parameters of neural networks. for epoch = 1 , ..., T do for each mini-batch node-set B from V do Get neighborhoods N ( v ) , ∀ v ∈ B for u ∈ B do for l = 1 ...L do Get neighborhoods embeddings Z l N ( v ) , ∀ v ∈ B z l +1 u ← Enc l ( z lu , Z l N ( u ) ) end for end for Z B ← z Lv , ∀ v ∈ B Compute loss (cid:80) v ∈ B L loss ( v ) with Z B Back propagation, update parameters. end for end for return Z = Enc L ( V ) From Algorithm 1, we can find that the number of com-puted neighbors increases exponentially, which is consistentwith GraphSAGE (Hamilton, Ying, and Leskovec 2017). Inthis paper, we set L = 2 , which can capture triangles struc-ture information. Experiment Detail
As we said in Experimental Settings section, the dataset usedin this paper is public and can be downloaded from the corre-sponding links. Data preprocessing is also discussed in Ex-periment Setting section, including how to binarize the net-works. In wikirfa dataset, there exits duplicate links (i.e., A → + B and A → − B ), we use the last record in the edgelist. That’s why it is slightly different from Chen et al.. Be-sides, we number the nodes to make string id into a numberid, which is eaiser to be vectorized. These numbers are com-pleted in the data preprocessing and are same to all embed-ding methods.
Complexity Analysis
We give complexity analysis in this section. Firstly, in thedataset given in Dataset section, no node attribute informa-tion is provided. For graph neural network methods (i.e.,SGCN, SiGAT and SDGNN), there are usually two typesof methods: using id embedding or using spectral analy-sis methods. For our SDGNN, we use a learnable id em-beddings for the initial node representation. It’s O ( |V| ∗ d ) ,where d is the embedding dimensions. Then, when we have L -layer GNN aggregators, we have the W whose complexityis O ( L ∗ d ∗ d ) . Besides, we have a two-layer MLP to encoderthe memssage for neighborhoods N ( v ) and node represen-tation z v . The complexity of this part is O ( L ∗ ∗ d ∗ d ) . Intotal, the complexity of SGNN is O ( |V| ∗ d + 11 ∗ L ∗ d ∗ d ) .For attention aggregators, it will include an additional atten-tion vector, whose complexity is O ( d ) . Various Values of λ and λ We show the results of various value of λ and λ in Table 7.From Table 7, we can find that λ and λ are not zero, show-ing better performance. Since reconstructing the sign is themost fundamental task, we don’t set it to zero.Table 7: Various Values of λ and λ . Metric λ = 0 λ = 1 λ = 5 λ = 10 λ = 0 Micro-F1 0.9386 0.9415 0.9411 0.9413Binary-F1 0.9677 0.9690 0.9688 0.9689Macro-F1 0.6738 0.7210 0.7267 0.7272AUC 0.8883 0.9030 0.8991 0.9008 λ = 1 Micro-F1 0.9438 0.9475 0.9452 0.9440Binary-F1 0.9702 0.9721 0.9709 0.9703Macro-F1 0.7414 0.7585 0.7484 0.7392AUC 0.9082 0.9109 0.9070 0.9014 λ = 5 Micro-F1 0.9456 0.9440 0.9461 0.9436Binary-F1 0.9711 0.9702 0.9714 0.9700Macro-F1 0.7553 0.7495 0.7531 0.7476AUC 0.9139 0.9152 0.9123 0.9088 λ = 10= 10