Philip Tee

Towards an approximate graph entropy measure for identifying incidents in network event data

A key objective of monitoring networks is to identify potential service threatening outages from events within the network before service is interrupted. Identifying causal events, Root Cause Analysis (RCA), is an active area of research, but current approaches are vulnerable to scaling issues with high event rates. Elimination of noisy events that are not causal is key to ensuring the scalability of RCA. In this paper, we introduce vertex-level measures inspired by Graph Entropy and propose their suitability as a categorization metric to identify nodes that are a priori of more interest as a source of events. We consider a class of measures based on Structural, Chromatic and Von Neumann Entropy. These measures require NP-Hard calculations over the whole graph, an approach which obviously does not scale for large dynamic graphs that characterise modern networks. In this work we identify and justify a local measure of vertex graph entropy, which behaves in a similar fashion to global measures of entropy when summed across the whole graph. We show that such measures are correlated with nodes that generate incidents across a network from a real data set.

Vertex Entropy As a Critical Node Measure in Network Monitoring

Understanding which node failures in a network have more impact is an important problem. Current understanding, motivated by the scale free models of network growth, places emphasis on the degree of the node. This is not a satisfactory measure; the number of connections a node has does not capture how redundantly it is connected into the whole network. Conversely, the structural entropy of a graph captures the resilience of a network well, but is expensive to compute, and, being a global measure, does not attribute any specific value to a given node. This lack of locality prevents the use of global measures as a way of identifying critical nodes. In this paper, we introduce local vertex measures of entropy which do not suffer from such drawbacks. In our theoretical analysis, we establish the possibility that our local vertex measures approximate global entropy, with the advantage of locality and ease of computation. We establish properties that vertex entropy must have in order to be useful for identifying critical nodes. We have access to a proprietary event, topology, and incident dataset from a large commercial network. Using this dataset, we demonstrate a strong correlation between vertex entropy and incident generation over events.

Constraints and entropy in a model of network evolution

Abstract Barabási–Albert’s “Scale Free” model is the starting point for much of the accepted theory of the evolution of real world communication networks. Careful comparison of the theory with a wide range of real world networks, however, indicates that the model is in some cases, only a rough approximation to the dynamical evolution of real networks. In particular, the exponent γ of the power law distribution of degree is predicted by the model to be exactly 3, whereas in a number of real world networks it has values between 1.2 and 2.9. In addition, the degree distributions of real networks exhibit cut offs at high node degree, which indicates the existence of maximal node degrees for these networks. In this paper we propose a simple extension to the “Scale Free” model, which offers better agreement with the experimental data. This improvement is satisfying, but the model still does not explain why the attachment probabilities should favor high degree nodes, or indeed how constraints arrive in non-physical networks. Using recent advances in the analysis of the entropy of graphs at the node level we propose a first principles derivation for the “Scale Free” and “constraints” model from thermodynamic principles, and demonstrate that both preferential attachment and constraints could arise as a natural consequence of the second law of thermodynamics.

Relating vertex and global graph entropy in randomly generated graphs

Combinatoric measures of entropy capture the complexity of a graph but rely upon the calculation of its independent sets, or collections of non-adjacent vertices. This decomposition of the vertex set is a known NP-Complete problem and for most real world graphs is an inaccessible calculation. Recent work by Dehmer et al. and Tee et al. identified a number of vertex level measures that do not suffer from this pathological computational complexity, but that can be shown to be effective at quantifying graph complexity. In this paper, we consider whether these local measures are fundamentally equivalent to global entropy measures. Specifically, we investigate the existence of a correlation between vertex level and global measures of entropy for a narrow subset of random graphs. We use the greedy algorithm approximation for calculating the chromatic information and therefore Körner entropy. We are able to demonstrate strong correlation for this subset of graphs and outline how this may arise theoretically.

The application of Neural Networks to predicting the root cause of service failures

The principal objective when monitoring compute and communications infrastructure is to minimize the Mean Time To Resolution of service-impacting incidents. Key to achieving that goal is determining which of the many alerts that are presented to an operator are likely to be the root cause of an incident. In turn this is critical in identifying which alerts should be investigated with the highest priority.