Thorsten Schütt

Range queries on structured overlay networks

The efficient handling of range queries in peer-to-peer systems is still an open issue. Several approaches exist, but their lookup schemes are either too expensive (space-filling curves) or their queries lack expressiveness (topology-driven data distribution). We present two structured overlay networks that support arbitrary range queries. The first one, named Chord^#, has been derived from Chord by substituting Chords hashing function by a key-order preserving function. It has a logarithmic routing performance and it supports range queries, which is not possible with Chord. Its O(1) pointer update algorithm can be applied to any peer-to-peer routing protocol with exponentially increasing pointers. We present a formal proof of the logarithmic routing performance and show empirical results that demonstrate the superiority of Chord^# over Chord in systems with high churn rates. We then extend our routing scheme to multiple dimensions, resulting in SONAR, a Structured Overlay Network with Arbitrary Range queries. SONAR covers multi-dimensional data spaces and, in contrast to other approaches, SONARs range queries are not restricted to rectangular shapes but may have arbitrary shapes. Empirical results with a data set of two million objects show the logarithmic routing performance in a geospatial domain.

A structured overlay for multi-dimensional range queries

We introduce SONAR, a structured overlay to store and retrieve objects addressed by multi-dimensional names (keys). The overlay has the shape of a multi-dimensional torus, where each node is responsible for a contiguous part of the data space. A uniform distribution of keys on the data space is not necessary, because denser areas get assigned more nodes. To nevertheless support logarithmic routing, SONAR maintains, per dimension, fingers to other nodes, that span an exponentially increasing number of nodes. Most other overlays maintain such fingers in the key-space instead and therefore require a uniform data distribution. SONAR, in contrast, avoids hashing and is therefore able to perform range queries of arbitrary shape in a logarithmic number of routing steps--independent of the number of system- and query-dimensions. SONAR needs just one hop for updating an entry in its routing table: A longer finger is calculated by querying the node referred to by the next shorter finger for its shorter finger. This doubles the number of spanned nodes and leads to exponentially spaced fingers.

Key-based consistency and availability in structured overlay networks

Structured Overlay Networks (SONs) provide a promising platform for high performance applications since they are scalable, fault-tolerant and self-managing. SONs provide lookup services that map keys to nodes that can be used as processing or storage resources. In SONs, lookups for a key may return inconsistent results. Consequently, it is difficult to provide consistent data services on top of SONs that build on key-based search. In this paper, we study the frequency of occurrence of inconsistent lookups. We show that the affect of lookup inconsistencies can be reduced by using node responsibilities. We present our results as a trade-off between consistency and availability of keys.

On Adaptability in Grid Systems

With the increasing size and complexity, adaptability is among the most badly needed properties in today’s Grid systems. Adaptability refers to the degree to which adjustments in practices, processes, or structures of systems are possible to projected or actual changes of their environment.

Enhanced Paxos Commit for Transactions on DHTs

Key/value stores which are built on structured overlay networks often lack support for atomic transactions and strong data consistency among replicas. This is unfortunate, because consistency guarantees and transactions would allow a wide range of additional application domains to benefit from the inherent scalability and fault-tolerance of DHTs. The Scalaris key/value store supports strong data consistency and atomic transactions. It uses an enhanced Paxos Commit protocol with only four communication steps rather than six. This improvement was possible by exploiting information from the replica distribution in the DHT. Scalaris enables implementation of more reliable and scalable infrastructure for collaborative Web services that require strong consistency and atomic changes across multiple items.

Efficient synchronization of replicated data in distributed systems

We present nsync, a tool for synchronizing large replicated data sets in distributed systems. nsync computes nearly optimal synchronization plans based on a hierarchy of gossip algorithms that take the network topology into account. Our primary design goals were maximum performance and maximum scalability. We achieved these goals by exploiting parallelism in the planning and the synchronization phase, by omitting transfer of unnecessary metadata, by synchronizing at a block level rather than a file level, and by using sophisticated compression methods. With its relaxed consistency semantic, nsync neither needs a master copy nor a quorum for updating distributed replicas. Each replica is kept as an autonomous entity and can be modified with the usual tools.

ConPaaS: an integrated runtime environment for elastic cloud applications

Most Cloud applications are re-enactments of traditional enterprise applications such as Web applications, content delivery and e-commerce [1]. The advantages of the Cloud are well-known: access to a near-infinite number of resources, ability to adjust an applications capacity on demand, pay-as-you-go pricing model. However, Cloud application developers need to pay attention to new topics: building custom VM images, making applications elastic and scalable, controlling performance, fault-tolerance, etc.

On Consistency Of Data In Structured Overlay Networks

Data consistency can be violated in Distributed Hash Tables (DHTs) due to inconsistent lookups. In this paper, we identify the events leading to inconsistent lookups and inconsistent responsibilities for a key. We find the inaccuracy of failure detectors as the main reason for inconsistencies. By simulations with inaccurate failure detectors, we study the probability of reaching a system configuration which may lead to inconsistent data. We analyze majority-based algorithms for operations on replicated data. To ensure that concurrent operations do not violate consistency, they have to use non-disjoint sets of replicas. We analytically derive the probability of concurrent operations including disjoint replica sets. By combining the simulation and analytical results, we show that the probability for a violation of data consistency is negligibly low for majority-based algorithms in DHTs.

Remote partial file access using compact pattern descriptions

We present a method for the efficient access to parts of remote files. The efficiency is achieved by using a file format independent compact pattern description, that allows us to request several parts of a file in a single operation. This results in a drastically reduced number of remote operations and network latencies if compared to common solutions. We measured the time to access parts of remote files with compact patterns, compared it with normal GridFTP remote partial file access and observed a significant performance increase. Further we discuss how the presented pattern access can be used for an efficient read from multiple replicas and how this can be integrated into a data management system to support the storage of partial replicas for large scale simulations.

QoS-aware storage virtualization for cloud file systems

We present a reservation scheduler for object-based file systems. It supports storage virtualization for multi-tenant cloud environments with quality of service (QoS) guarantees. The reservation scheduler has been integrated into the XtreemFS cloud file system to maximize the resource utilization under the given QoS demands. Our simulation results obtained with a discrete event simulator indicate that a considerable number of active object stores can be saved while still ensuring the requested service guarantees (capacity, throughput, IOPS, etc.).