Damien Fay

Weighted spectral distribution for internet topology analysis: theory and applications

Comparing graphs to determine the level of underlying structural similarity between them is a widely encountered problem in computer science. It is particularly relevant to the study of Internet topologies, such as the generation of synthetic topologies to represent the Internets AS topology. We derive a new metric that enables exactly such a structural comparison: the weighted spectral distribution. We then apply this metric to three aspects of the study of the Internets AS topology. i) We use it to quantify the effect of changing the mixing properties of a simple synthetic network generator. ii) We use this quantitative understanding to examine the evolution of the Internets AS topology over approximately seven years, finding that the distinction between the Internet core and periphery has blurred over time. iii) We use the metric to derive optimal parameterizations of several widely used AS topology generators with respect to a large-scale measurement of the real AS topology.

24-h electrical load data—a sequential or partitioned time series?

Variations in electrical load are, among other things, hour of the day dependent, introducing a dilemma for the forecaster: whether to partition the data and use a separate model for each hour of the day (the parallel approach), or use a single model (the sequential approach). This paper examines which approach is appropriate for forecasting hourly electrical load in Ireland. It is found that, with the exception of some hours of the day, the sequential approach is superior. The final solution however, uses a combination of linear sequential and parallel neural models in a multi-time scale formulation.

On the Influence of Weather Forecast Errors in Short-Term Load Forecasting Models

Weather information is an important factor in load forecasting models. Typically, load forecasting models are constructed and tested using actual weather readings. However, online operation of load forecasting models requires the use of weather forecasts, with associated weather forecast errors. These weather forecast errors inevitably lead to a degradation in model performance. This is an important factor in load forecasting but has not been widely examined in the literature. The main aim of this paper is to present a novel technique for minimizing the consequences of this degradation. In addition, a supplementary technique is proposed to model weather forecast errors to reflect current accuracy. The proposed technique utilizes a combination of forecasts from several load forecasting models (sub-models). The parameter estimation may thus be split into two parts: sub-model and combination parameter estimation. It is shown that the lowest PMSE corresponds to training the sub-models with actual weather but training the combiner with forecast weather.

Network growth and the spectral evolution model

We introduce and study the spectral evolution model, which characterizes the growth of large networks in terms of the eigenvalue decomposition of their adjacency matrices: In large networks, changes over time result in a change of a graphs spectrum, leaving the eigenvectors unchanged. We validate this hypothesis for several large social, collaboration, authorship, rating, citation, communication and tagging networks, covering unipartite, bipartite, signed and unsigned graphs. Following these observations, we introduce a link prediction algorithm based on the extrapolation of a networks spectral evolution. This new link prediction method generalizes several common graph kernels that can be expressed as spectral transformations. In contrast to these graph kernels, the spectral extrapolation algorithm does not make assumptions about specific growth patterns beyond the spectral evolution model. We thus show that it performs particularly well for networks with irregular, but spectral, growth patterns.

Tuning Topology Generators Using Spectral Distributions

An increasing number of synthetic topology generators are available, each claiming to produce representative Internet topologies. Every generator has its own parameters, allowing the user to generate topologies with different characteristics. However, there exist no clear guidelines on tuning the value of these parameters in order to obtain a topology with specific characteristics. In this paper we optimize the parameters of several topology generators to match a given Internet topology. The optimization is performed either with respect to the link density, or to the spectrum of the normalized Laplacian matrix. Contrary to approaches in the literature that rely only on the largest eigenvalues, we take into account the set of all eigenvalues. However, we show that on their own the eigenvalues cannot be used to construct a metric for optimizing parameters. Instead we present a weighted spectral method which simultaneously takes into account all the properties of the graph.

Per flow packet sampling for high-speed network monitoring

We present a per-flow packet sampling method that enables the real-time classification of high-speed network traffic. Our method, based upon the partial sampling of each flow (i.e., performing sampling at only early stages in each flows lifetime), provides a sufficient reduction in total traffic (e.g., a factor of five in packets, a factor of ten in bytes) as to allow practical implementations at one Gigabit/s, and, using limited hardware assistance, ten Gigabit/s.

Mixing biases: structural changes in the AS topology evolution

In this paper we study the structural evolution of the AS topology as inferred from two different datasets over a period of seven years. We use a variety of topological metrics to analyze the structural differences revealed in the AS topologies inferred from the two different datasets. In particular, to focus on the evolution of the relationship between the core and the periphery, we make use of a recently introduced topological metric, the weighted spectral distribution. We find that the traceroute dataset has increasing difficulty in sampling the periphery of the AS topology, largely due to limitations inherent to active probing. Such a dataset has too limited a view to properly observe topological changes at the AS-level compared to a dataset largely based on BGP data. We also highlight limitations in current measurements that require a better sampling of particular topological properties of the Internet. Our results indicate that the Internet is changing from a core-centered, strongly customer-provider oriented, disassortative network, to a soft-hierarchical, peering-oriented, assortative network.

Graph metrics as summary statistics for Approximate Bayesian Computation with application to network model parameter estimation

In this paper, we investigate Approximate Bayes Computation as a technique for estimating the parameters of graph generators relative to an observed graph. Specifically, we investigate six spectral graph metrics with a view to evaluating their suitability as summary statistics. The overall findings are that Approximate Bayesian Computation can result in reasonable estimates of the parameter posteriors, if the rank of the metrics is sufficiently high. For some graph metrics, biases can exist in the estimated parameters though these appear, empirically, to be small. We demonstrate that combining metrics to form a new summary statistic provides more robust estimates. Given these results, the authors then create two, somewhat arbitrary, graph generators and show how the parameters for these may be estimated with ease. In addition, we show how to apply model selection to determine which generator best explains the observed graph.

Discriminating graphs through spectral projections

This paper proposes a novel non-parametric technique for clustering networks based on their structure. Many topological measures have been introduced in the literature to characterize topological properties of networks. These measures provide meaningful information about the structural properties of a network, but many networks share similar values of a given measure [1]. Furthermore, strong correlation between these measures occur on real-world graphs [2], so that using them to distinguish arbitrary graphs is difficult in practice [3]. Although a very complicated way to represent the information and the structural properties of a graph, the graph spectrum [4] is believed to be a signature of a graph [5]. A weighted form of the distribution of the graph spectrum, called the weighted spectral distribution (WSD), is proposed here as a feature vector. This feature vector may be related to actual structure in a graph and in addition may be used to form a metric between graphs; thus ideal for clustering purposes. To distinguish graphs, we propose to rely on two ways to project a weighted form of the eigenvalues of a graph into a low-dimensional space. The lower dimensional projection, turns out to nicely distinguish different classes of graphs, e.g. graphs from network topology generators [6-8], Internet application graphs [9], and dK-random graphs [10]. This technique can be used advantageously to separate graphs that would otherwise require complex sets of topological measures to be distinguished [9].

Diversity dynamics in online networks

Diversity is an important characterization aspect for online social networks that usually denotes the homogeneity of a networks content and structure. This paper addresses the fundamental question of diversity evolution in large-scale online communities over time. In doing so, we study different established notions of network diversity, based on paths in the network, degree distributions, eigenvalues, cycle distributions, and control models. This leads to five appropriate characteristic network statistics that capture corresponding aspects of network diversity: effective diameter, Gini coefficient, fractional network rank, weighted spectral distribution, and number of driver nodes of a network. Consequently, we present and discuss comprehensive experiments with a broad range of directed, undirected, and bipartite networks from several different network categories -- including hyperlink, interaction, and social networks. An important general observation is that network diversity shrinks over time. From the conceptual perspective, our work generalizes previous work on shrinking network diameters, putting it in the context of network diversity. We explain our observations by means of established network models and introduce the novel notion of eigenvalue centrality preferential attachment.