John Brennan

Combining AiG Agents with Unicore Grid for Improvement of User Support

Grid computing has, in recent history, become an invaluable tool for scientific research. As grid middleware has matured, considerations have extended beyond the core functionality, towards greater usability. The aim of this paper is to consider how resources that are available to the users across the Queens gate Grid (QGG) at the University of Huddersfield (UoH), could be accessed with the help of an ontology-driven interface. The interface is a part of the Agent in Grid (AiG) project under development at the Systems Research Institute Polish Academy of Sciences (SRIPAS). It is to be customized and integrated with the UoH computing environment. The overarching goal is to help users of the grid infrastructure. The secondary goals are: (i) to improve the performance of the system, and (ii) to equalize the distribution of work among resources. Results presented in this paper include the new ontology that is being developed for the grid at the UoH, and the description of issues encountered during the development of a scenario when user searches for an appropriate resource within the Unicore grid middleware and submits job to be executed on such resource.

CDES: An Approach to HPC Workload Modelling

Computational science and complex system administration relies on being able to model user interactions. When it comes to managing HPC, HTC and grid systems user workloads - their job submission behaviour, is an important metric when designing systems or scheduling algorithms. Most simulators are either inflexible or tied in to proprietary scheduling systems. For system administrators being able to model how a scheduling algorithm behaves or how modifying system configurations can affect the job completion rates is critical. Within computer science research many algorithms are presented with no real description or verification of behaviour. In this paper we are presenting the Cluster Discrete Event Simulator (CDES) as an strong candidate for HPC workload simulation. Built around an open framework, CDES can take system definitions, multi-platform real usage logs and can be interfaced with any scheduling algorithm through the use of an API. CDES has been tested against 3 years of usage logs from a production level HPC system and verified to a greater than 95% accuracy.

GFP-X: A parallel approach to massive graph comparison using spark

The problem of how to compare empirical graphs is an area of great interest within the field of network science. The ability to accurately but efficiently compare graphs has a significant impact in such areas as temporal graph evolution, anomaly detection and protein comparison. The comparison problem is compounded when working with massive graphs containing millions of vertices and edges. This paper introduces a parallel feature extraction based approach for the efficient comparison of large unlabelled graph datasets using Apache Spark. The approach acts by producing a ‘Graph Fingerprint’ which represents both vertex level and global level topological features from a graph. By using Spark we are able to efficiently compare graphs considered unmanageably large to other approaches. The runtime of the approach is shown to scale sub-linearly with the size and complexity of the graphs being fingerprinted. Importantly, the approach is shown to not only be comparable to existing approaches, but on when comparing topology and size, more sensitive at detecting variation between graphs.

Evaluating the quality of graph embeddings via topological feature reconstruction

In this paper we study three state-of-the-art, but competing, approaches for generating graph embeddings using unsupervised neural networks. Graph embeddings aim to discover the ‘best’ representation for a graph automatically and have been applied to graphs from numerous domains, including social networks. We evaluate their effectiveness at capturing a good representation of a graphs topological structure by using the embeddings to predict a series of topological features at the vertex level. We hypothesise that an ‘ideal’ high quality graph embedding should be able to capture key parts of the graphs topology, thus we should be able to use it to predict common measures of the topology, for example vertex centrality. This could also be used to better understand which topological structures are truly being captured by the embeddings. We first review these three graph embedding techniques and then evaluate how close they are to being ‘ideal’. We provide a framework, with extensive experimental evaluation on empirical and synthetic datasets, to assess the effectiveness of several approaches at creating graph embeddings which capture detailed topological structure.

Exploring the Evolution of Big Data Technologies

This chapter explores the rise of “big data” and the computational strategies, both hardware and software, that have evolved to deal with this paradigm. Starting with the concept of data-intensive computing, the different facets of data processing like Map/Reduce, Machine Learning, and Streaming data are explored. The evolution of different frameworks such as Hadoop and Spark are outlined and an assessment of the modular offerings within the frameworks is compared with a detailed analysis of the different functionalities and features. The hardware considerations required to move from compute-intensive to data-intensive are outlined along with the impact of cloud computing on big data. The chapter concludes with the upcoming developments in the near future for big data and how this computing paradigm fits into the road to exascale.

Deep topology classification: A new approach for massive graph classification

The classification of graphs is a key challenge within many scientific fields using graphs to represent data and is an active area of research. Graph classification can be critical in identifying and labelling unknown graphs within a dataset and has seen application across many scientific fields. Graph classification poses two distinct problems: the classification of elements within a graph and the classification of the entire graph. Whilst there is considerable work on the first problem, the efficient and accurate classification of massive graphs into one or more classes has, thus far, received less attention. In this paper we propose the Deep Topology Classification (DTC) approach for global graph classification. DTC extracts both global and vertex level topological features from a graph to create a highly discriminate representation in feature space. A deep feed-forward neural network is designed and trained to classify these graph feature vectors. This approach is shown to be over 99% accurate at discerning graph classes over two datasets. Additionally, it is shown to be more accurate than current state of the art approaches both in binary and multi-class graph classification tasks.