Harris Papadakis

Peer-to-Peer resource discovery in Grids: Models and systems

Resource location or discovery is a key issue for Grid systems in which applications are composed of hardware and software resources that need to be located. Classical approaches to Grid resource location are either centralized or hierarchical and will prove inefficient as the scale of Grid systems rapidly increases. On the other hand, the Peer-to-Peer (P2P) paradigm emerged as a successful model that achieves scalability in distributed systems. One possibility would be to borrow existing methods from the P2P paradigm and to adopt them to Grid systems taking into consideration the existing differences. Several such attempts have been made during the last couple of years. This paper aims to serve as a review of the most promising Grid systems that use P2P techniques to facilitate resource discovery in order to perform a qualitative comparison of the existing approaches and to draw conclusions about their advantages and weaknesses. Future research directions are also discussed.

Interactive Image Segmentation Based on Synthetic Graph Coordinates

In this paper, we propose a framework for interactive image segmentation. The goal of interactive image segmentation is to classify the image pixels into foreground and background classes, when some foreground and background markers are given. The proposed method minimizes a min-max Bayesian criterion that has been successfully used on image segmentation problem and it consists of several steps in order to take into account visual information as well as the given markers, without any requirement of training. First, we partition the image into contiguous and perceptually similar regions (superpixels). Then, we construct a weighted graph that represents the superpixels and the connections between them. An efficient algorithm for graph clustering based on synthetic coordinates is used yielding an initial map of classified pixels. This method reduces the problem of graph clustering to the simpler problem of point clustering, instead of solving the problem on the graph data structure, as most of the known algorithms from literature do. Finally, having available the data modeling and the initial map of classified pixels, we use a Markov Random Field (MRF) model or a flooding algorithm to get the image segmentation by minimizing a min-max Bayesian criterion. Experimental results and comparisons with other methods from the literature are presented on LHI, Gulshan and Zhao datasets, demonstrating the high performance and accuracy of the proposed scheme.

ITA: Innocuous Topology Awareness for Unstructured P2P Networks

One of the most appealing characteristics of unstructured P2P overlays is their enhanced self-* properties, which results from their loose, random structure. In addition, most of the algorithms which make searching in unstructured P2P systems scalable, such as dynamic querying and 1-hop replication, rely on the random nature of the overlay to function efficiently. The underlying communications network (i.e., the Internet), however, is not as randomly constructed. This leads to a mismatch between the distance of two peers on the overlay and the hosts they reside on at the IP layer, which in turn leads to its misuse. The crux of the problem arises from the fact that any effort to provide a better match between the overlay and the IP layer will inevitably lead to a reduction in the random structure of the P2P overlay, with many adverse results. With this in mind, we propose ITA, an algorithm which creates a random overlay of randomly connected neighborhoods providing topology awareness to P2P systems, while at the same time has no negative effect on the self-* properties or the operation of the other P2P algorithms. Using extensive simulations, both at the IP router level and autonomous system level, we show that ITA reduces communication latencies by as much as 50 percent. Furthermore, it not only reduces by 20 percent the number of IP network messages which is critical for ISPs carrying the burden of transporting P2P traffic, but also distributes the traffic load more evenly on the routers of the IP network layer.

Distributed community detection: Finding neighborhoods in a complex world using synthetic coordinates

In this paper, we propose an algorithm that finds the entire community structure of a network, based on local interactions between neighboring nodes and on an unsupervised centralized clustering algorithm. The novelty of the proposed approach is the fact that the algorithm is based on the use of network coordinates computed by a distributed algorithm. The current paper not only presents an efficient distributed community finding algorithm, but also demonstrates that network coordinates could be used to derive efficient solutions to a variety of problems. Experimental results and comparisons with other methods from literature are presented for a variety of benchmark graphs with known community structure, derived by varying a number of graph parameters.

Local Community Finding Using Synthetic Coordinates

A fundamental problem in social networking and computing is the community finding problem that can be used in a lot of social networks’ applications. In this paper, we propose an algorithm that finds the entire community structure of a network, based on interactions between neighboring nodes (distributed method) and on an unsupervised centralized clustering algorithm. Experimental results and comparisons with another method found in the literature are presented for a variety of benchmark graphs with known community structure, derived by varying a number of graph parameters. The experimental results demonstrate the high performance of the proposed algorithm to detect communities.

Imbuing unstructured P2P systems with non-intrusive topology awareness

The random nature of unstructured P2P overlays imbues them with enhanced self-* properties. Most of the algorithms which make searching in unstructured P2P systems scalable, such as dynamic querying and 1-hop replication, rely on the random nature of the overlay to function efficiently. However, they do not take into account the structure of the underlying physical communications network, which is anything but random. Efforts to provide topology awareness to unstructured P2P systems often result to clustered graphs which affect negatively algorithms that rely on random overlays. In this paper, we propose ITA, an algorithm which creates a random overlay of randomly connected neighborhoods providing topology awareness to P2P systems, while at the same time has no negative effect on the self-* properties or the operation of the other P2P algorithms. Using extensive simulations, we demonstrate that ITA reduces communication latency by as much as 50% which is important for P2P users. Furthermore, it reduces by 20% the number of IP network messages which is critical for ISPs carrying the burden of transporting P2P traffic. Finally, ITA is shown to reduce significantly the load imposed on the routers of the IP network layer.

COOPERATIVE SELF-COMPOSITION AND DISCOVERY OF GRID SERVICES IN P2P NETWORKS

The desirable global scalability of Grid systems has steered the research towards the employment of the peer-to-peer (P2P) paradigm for the development of new resource discovery systems. As Grid systems mature, the requirements for such a mechanism have grown from simply locating the desired service to compose more than one service to achieve a goal. In Semantic Grid, resource discovery systems should also be able to automatically construct any desired service if it is not already present in the system, by using other, already existing services. In this paper, we present a novel system for the automatic discovery and composition of services, based on the P2P paradigm, having in mind (but not limited to) a Grid environment for the application. The paper improves composition and discovery by exploiting a novel network partitioning scheme for the decoupling of services that belong to different domains and an ant-inspired algorithm that places co-used services in neighbouring peers.

Divide et Impera: Partitioning Unstructured Peer-to-Peer Systems to Improve Resource Location

Unstructured P2P systems exhibit a great deal of robustness and self-healing at the cost of reduced scalability. Resource location is performed using a broadcast-like process called flooding. The work presented in this paper comprises an effort to reduce the overwhelming volume of traffic generated by flooding, thus increasing the scalability of unstructured P2P systems. Using a simple hash-based content categorization method the Ultrapeer overlay network is partitioned into a relatively small number of distinct subnetworks. By employing a novel index splitting technique each leaf peer is effectively connected to each different subnetwork. The search space of each individual flooding is restricted to a single partition, and is thus considerably limited. This reduces significantly the volume of traffic produced by flooding without affecting at all the accuracy of the search method. Experimental results demonstrate the efficiency of the proposed method.

Distributed detection of communities in complex networks using synthetic coordinates

Various applications like finding Web communities, detecting the structure of social networks, and even analyzing a graph’s structure to uncover Internet attacks are just some of the applications for which community detection is important. In this paper, we propose an algorithm that finds the entire community structure of a network, on the basis of local interactions between neighboring nodes and an unsupervised distributed hierarchical clustering algorithm. The novelty of the proposed approach, named SCCD (standing for synthetic coordinate community detection), lies in the fact that the algorithm is based on the use of Vivaldi synthetic network coordinates computed by a distributed algorithm. The current paper not only presents an efficient distributed community finding algorithm, but also demonstrates that synthetic network coordinates could be used to derive efficient solutions to a variety of problems. Experimental results and comparisons with other methods from the literature are presented for a variety of benchmark graphs with known community structure, derived from varying a number of graph parameters and real data set graphs. The experimental results and comparisons to existing methods with similar computation cost on real and synthetic data sets demonstrate the high performance and robustness of the proposed scheme.

METADATA RANKING AND PRUNING FOR FAILURE DETECTION IN GRIDS

The objective of Grid computing is to make processing power as accessible and easy to use as electricity and water. The last decade has seen an unprecedented growth in Grid infrastructures which nowadays enables large-scale deployment of applications in the scientific computation domain. One of the main challenges in realizing the full potential of Grids is making these systems dependable. In this paper we present FailRank, a novel framework for integrating and ranking information sources that characterize failures in a grid system. After the failing sites have been ranked, these can be eliminated from the job scheduling resource pool yielding in that way a more predictable, dependable and adaptive infrastructure. We also present the tools we developed towards evaluating the FailRank framework. In particular, we present the FailBase Repository which is a 38GB corpus of state information that characterizes the EGEE Grid for one month in 2007. Such a corpus paves the way for the community to systematically uncover new, previously unknown patterns and rules between the multitudes of parameters that can contribute to failures in a Grid environment. Additionally, we present an experimental evaluation study of the FailRank system over 30 days which shows that our framework identifies failures in 93% of the cases and can achieve this by only fetching 65% of the available information sources. We believe that our work constitutes another important step towards realizing adaptive Grid computing systems.