New complex network building methodology for High Level Classification based on attribute-attribute interaction
NN EW COMPLEX NETWORK BUILDING METHODOLOGY FOR H IGH L EVEL C LASSIFICATION BASED ON ATTRIBUTE - ATTRIBUTEINTERACTION
Esteban Wilfredo Vilca Zuñiga
Dept. of Computing and MathematicsFFCLRP-USPRibeirão Preto, Brasil [email protected]
September 16, 2020 A BSTRACT
High-level classification algorithms focus on the interactions between instances. These produce anew form to evaluate and classify data. In this process, the core is the complex network buildingmethodology because it determines the metrics to be used for classification. The current methodolo-gies use variations of kNN to produce these graphs. However, this technique ignores some hiddenpattern between attributes and require normalization to be accurate. In this paper, we propose anew methodology for network building based on attribute-attribute interactions that do not requirenormalization and capture the hidden patterns of the attributes. The current results show us that couldbe used to improve some current high-level techniques.
The machine learning classification algorithms are low level when they use just physical features to classify usuallydistance measures like euclidean distance. However, high-level classification algorithms focus on the interaction betweenthe data. Using metrics that evaluate the behavior of each node concerning others [Christiano Silva and Zhao 2016].These interactions between instances are usually represented as complex networks. There is a variety of tech-niques to build networks but usually, they use kNN as the core [Carneiro and Zhao 2018] [Colliri et al. 2018][Fadaee and Haeri 2019]. These metrics produce a network where each node represents an instance and each edgerepresents a neighbor in kNN.A complex network is defined as a non-trivial graph [Albert and Barabási 2002]. Usually, the quantity of instanceson the dataset and the interactions generates a large graph with numerous edges. This large graphs presents specialcharacteristics that are exploited in many techniques to classify data like Betweenness Centrality, Clustering Coefficient,Assortativity, HLNB_BC and so on.In order to produce the best classification, we need to present structures that capture all the interactions in the dataset.The current techniques are based on kNN that exploits the relationship between instances. Nevertheless, they justcapture instance-instance interactions but there are some hidden patterns on attribute-attribute interaction that areomitted.In this paper, we will present a new methodology that captures these attribute-attribute interactions and how the currenthigh-level prediction techniques are affected by this new paradigm. a r X i v : . [ c s . L G ] S e p PREPRINT - S
EPTEMBER
16, 2020
A complex network presents two main parts, the nodes V and the links E . Many methodologies are based on kNN tobuild the graphs. They use each instance X i as a node V i and the links are the connections between this node and its kneighbors. These neighbors in the graph are represented as the neighborhood N of each node. If we are going to usethe network for supervised learning, we will need to add the label y i as a parameter to remove all the links were theneighbors labels are different from the evaluated node [Silva and Zhao 2012]. N ( X i ) = kN N ( X i , y i ) (1)Where X i is the instance and y i is the label of the instance. The neighborhood N will give us the nodes connected tothe main node V i following the next rule { V j , ( V i , V j ) ∈ V : V j ∈ N ( X i ) } . When the graph is sparse this methodologycaptures good relations, but in dense graphs ignore close relationships. There is another variation adding (cid:15) - radius algorithm to capture this dense regions [Silva and Zhao 2012]. N ( X i ) = (cid:26) kN N ( X i , y i ) , otherwise (cid:15) - radius ( X i , y i ) , if | (cid:15) - radius ( X i , y i ) | > k (2)Where (cid:15) - radius ( X i , y i ) returns the set of nodes { V j , V j ∈ V : distance ( X i , X j ) < (cid:15) ∧ y i = y j } . The distance measure could be a similarity function like euclidean distance. The (cid:15) value is determinated according to the sparsity ofthe network. A common value is the median value of the kN N distances [Colliri et al. 2018].This methodology need the normalization of the data to be used by current high-level prediction methodologies andconsidering a node V i as an entire instance X i ignores some hidden-patterns between attributes. There are some problems related to the core of these methodologies.
In figure 2.1.1, we can observe how two instance are reduced to just two nodes omitting possible patterns in the sameattribute. Figure 1: Image of two related instances transformed in two linked nodes.
Due to the reduction of the attributes in one instance, we need normalization because each attribute could presentdifferent scale values.
By cause of the reduction of dimensions, if we do not have a high number of neighbors the networks could be discon-nected. Some methodologies must introduce an extra node class to avoid this problem like
HLNB_BC classification technique.
Our methodology, has three main parts. First, we will use each attribute-attribute interaction as an independent networklike in figure 2. As a result, we will capture possible hidden patterns for each attribute. Also, this will reduce the2
PREPRINT - S
EPTEMBER
16, 2020normalization dependence because each attribute present the same scale. These networks will follow the same neighborhood equation (1) in one dimension.Figure 2: Image with of dataset with 4 graph representation for each attribute.Second, we will combine all the current networks following the same node index in each network. Hence, each nodewill become in a meta node that absorbs all the links from each attribute. Then, we will use the current techniques likethe equation (2) for these meta nodes to preserve the instance-instance interaction.Third, this combined graph could be used for current high-level classification techniques. The new testing instancesinserted must follow the same process. It has to be divided for each attribute, combine in a unique node, and use thismeta node in the equation (2).This building approach reduces the probability of non-connected graphs, capture the relations on the same scale reducingthe dependency of normalization, and acquiring the hidden attribute-attribute patterns. We can observe the differencebetween the current technique described in equation (2), and our methodology on figure (3) and figure (4) respectively.Figure 3: Image of disconnected complex network using the current techniques, kNN (k=1) and (cid:15) -radius ( (cid:15) = median of kN N distances ) on Wine UCI Dataset with three attributes red, blue and green nodes.Figure 4: Image of complex network using our methodology, kNN (k=1) and (cid:15) -radius ( (cid:15) = median of kN N distances )on Wine UCI Dataset with three attributes red, blue and green nodes.3
PREPRINT - S
EPTEMBER
16, 2020
In table 1, we present the results in normalized UCI datasets [Dua and Graff 2017] with the current technique and ourmethodology. The dataset Wine presents attributes on different scales. For this reason, the current building methodologypresents a reduced accuracy. However, our new methodology presents a better performance 95.56% against 75.84%. Inthe Iris dataset, the performance was similar in both technique due to this data set presents attribute with the same scale.In the last dataset zoo, our methodology reduces its performance drastically. This dataset has 17 attributes and 7 classesbut just 100 instances. Theses characteristics could create noise in the final graph.Results of 10 times using 10-folds cross validationDataset Prediction Building (k) AccuracyIris HLNB_BC kNN+ (cid:15) - radius (1) 95.33 ± ± (cid:15) - radius (1) 75.84 ± ± (cid:15) - radius (1) 96.36 ± ± We introduce a new technique focused on capture the hidden patterns between attribute-attribute interactions. Thecurrent results show us that capturing these patterns could improve the classification of high-level algorithms. However,this technique reduces its efficiency in small datasets with a high quantity of attributes and classes.
The proposed algorithm presents problems when the attribute-attribute relations do not provide additional informationbecause they introduce noise to the final graph. Thus, it is needed a form to capture these attributes and remove theirconnections to avoid the noise.
References [Albert and Barabási 2002] Albert, R. and Barabási, A.-L. (2002). Statistical mechanics of complex networks.
Rev.Mod. Phys. , 74:47–97.[Carneiro and Zhao 2018] Carneiro, M. and Zhao, L. (2018). Organizational data classification based on the importanceconcept of complex networks.
IEEE Transactions on Neural Networks and Learning Systems , 29:3361–3373.[Christiano Silva and Zhao 2016] Christiano Silva, T. and Zhao, L. (2016).
Machine Learning in Complex Networks .Springer International Publishing.[Colliri et al. 2018] Colliri, T., Ji, D., Pan, H., and Zhao, L. (2018). A network-based high level data classificationtechnique. In , pages 1–8.[Dua and Graff 2017] Dua, D. and Graff, C. (2017). UCI machine learning repository.[Fadaee and Haeri 2019] Fadaee, S. A. and Haeri, M. A. (2019). Classification using link prediction.
Neurocomputing ,359:395 – 407.[Silva and Zhao 2012] Silva, T. C. and Zhao, L. (2012). Network-based high level data classification.