Akiyoshi Matono

RDFCube: a P2P-based three-dimensional index for structural joins on distributed triple stores

Today, RDF data/triples are scattered everywhere and their total size is rapidly increasing. Centralized RDF triple stores have limitations on both their failure tolerance and scalability. Therefore, RDF query processing in a P2P environment is an important issue. So far, several conventional P2P-based RDF triple stores have been proposed. They, however, are designed merely for triple retrieval rather than for triple join query processing. Consequently, they suffer from an unnecessary data transfer problem. This paper presents an RDF query processing technique based on a three-dimensional hash index. The triples are mapped into the index; then bit information that represents the presence or absence of triples in the index is introduced. We implemented our approach on the top of an emulated P2P environment. Evaluation results show that our approach can achieve good performance and scalability.

ADERIS: an adaptive query processor for joining federated SPARQL endpoints

Integrating distributed RDF data is facilitated by Linked Data and shared ontologies, however joins over distributed SPARQL services can be costly, time consuming operations. This paper describes the design and implementation of ADERIS, a query processing system for efficiently joining data from multiple distributed SPARQL endpoints. ADERIS decomposes federated SPARQL queries into multiple source queries and integrates the results utilising two techniques: adaptive join reordering, for which a cost model is defined, and the optimisation of subsequent queries to data sources to retrieve further data. The benefit of the approach in terms of minimising response time is illustrated by sample queries containing common SPARQL join patterns.

Adaptive integration of distributed semantic web data

The use of RDF (Resource Description Framework) data is a cornerstone of the Semantic Web. RDF data embedded in Web pages may be indexed using semantic search engines, however, RDF data is often stored in databases, accessible via Web Services using the SPARQL query language for RDF, which form part of the Deep Web which is not accessible using search engines. This paper addresses the problem of effectively integrating RDF data stored in separate Web-accessible databases. An approach based on distributed query processing is described, where data from multiple repositories are used to construct partitioned tables that are integrated using an adaptive query processing technique supporting join reordering, which limits any reliance on statistics and metadata about SPARQL endpoints, as such information is often inaccurate or unavailable, but is required by existing systems supporting federated SPARQL queries. The approach presented extends existing approaches in this area by allowing tables to be added to the query plan while it is executing, and shows how an approach currently used within relational query processing can be applied to distributed SPARQL query processing. The approach is evaluated using a prototype implementation and potential applications are discussed.

Extensions to the Pig data processing platform for scalable RDF data processing using Hadoop

In order to effectively handle the growing amount of available RDF data, a scalable and flexible RDF data processing framework is needed. We previously proposed a Hadoop-based framework, which takes advantages of scalable and fault-tolerant distributed processing technologies, originally proposed as Googles distributed file system and MapReduce parallel model. In this paper, we present a method extending the Pig data processing platform on top of the Hadoop infrastructure. Pig compiles programs written in a high level language, called Pig Latin, into MapReduce programs that can be executed by Hadoop. In order to support RDF, Pig was extended with the ability to load and store RDF data efficiently. Furthermore, as reasoning is an important requirement for most systems storing RDF data, support for inferring new triples using entailment rules was also added. In this paper, we describe these extensions and present an evaluation of their performance.

Dynamic Data Redistribution for MapReduce Joins

MapReduce has become a popular method for data processing, in particular for large scale datasets, due to its accessibility as a scalable yet convenient programming paradigm. Data processing tasks often involve joins, and the repartition and fragment-replicate joins are two widely-used join algorithms utilised within the MapReduce framework. This paper presents a multi-join supporting tuple redistribution, building on both the repartition and fragment-replicate joins. Hadoop is used to demonstrate how reduce tasks may improve performance by passing intermediate results to other reduce tasks that are better able to process them using Apache ZooKeeper as a means of communication and data transfer. A performance analysis is presented showing the technique has the potential to reduce response times when processing multiple joins in single MapReduce jobs.

Paragraph Tables: A Storage Scheme Based on RDF Document Structure

Efficient query processing for RDF graphs is essential, because RDF is one of the most important frameworks supporting the semantic web and linked data. The performance of query processing is based on the storage layout. So far, a number of storage schemes for RDF graphs have already been proposed. However most approaches must frequently perform costly join operations, because they decompose an RDF graph into a set of triples, store them separately, and need to connect them to reconstruct a graph that matchs the query graph, and this process requires join operations. In this paper, we propose a storage scheme that stores RDF graphs as they are connected, without decomposition. We focus on RDF documents, where adjacent triples have a high relationship and may be described for the same resource. So we define a set of adjacent triples that refer to the same resource as an RDF paragraph. Our approach constructs the table layout based on the RDF paragraphs. We evaluate the performance of our approach through experiments and demonstrate that our approach outperforms other approaches in query performance in most cases.

ADERIS: adaptively integrating RDF data from SPARQL endpoints

This paper describes the Adaptive Distributed Endpoint RDF Integration System (ADERIS), an adaptive, distributed query processor for integrating RDF data from multiple data resources supporting the SPARQL query language and protocol. The system allows a user to issue a federated query without any knowledge of the data contained in each endpoint and without specifying details of how the query should be executed. ADERIS relies on very limited information about each RDF data source to construct SPARQL source queries, the results of which are used to construct RDF predicate tables, which are integrated using pipelined index nested loop joins, the number and order of which may vary during query execution in order to reduce response time.

SPARQL-based set-matching for semantic grid resource selection

Grid is an emerging technology that enables the sharing of a wide variety of resources. However, effective and accurate grid resource matching is difficult because of the dynamic characteristics and heterogeneity of grid resources. Grid resource matching mechanisms that utilize semantic Web technologies have been proposed to deal with this issue, although none support set matching of grid resources based on their semantic description. This paper proposes a novel set-matching algorithm that uses standard RDF query language SPARQL [1] to semantically match a set of grid resources and SPARQL query features to efficiently perform set matching. We evaluated the efficiency and effectiveness of the algorithm by performing a set of experiments and present the results.

OGC catalog service for heterogeneous earth observation metadata using extensible search indices

In this paper, we propose an extensible information retrieval system based on data typed indices. The indices are constructed for various data types and are customized and extensible. Based on this system, we have implemented a catalog service of earth observation metadata. Using this system, it is possible to search through a large amount of metadata with heterogeneous schema. Fast response time is also achieved regardless of the number of individual query results.

Optimising Coverage, Freshness and Diversity in Live Exploration-based Linked Data Queries

Centralised indexes and distributed query federation-based approaches towards executing queries over distributed Linked Open Data are currently limited when it comes to providing complete coverage and up-to-date results. However, live exploration-based query execution, in accordance with the Linked Open Data publishing principles, dereferences Internationalised Resource Identifiers (IRI)s on the fly in order to provide results from Linked Data anywhere on the Web. We propose and investigate similarity search-based strategies for dereferencing IRIs during live exploration-based querying in order to maximise user criteria of coverage, freshness and diversity within a limited execution time, in contrast to existing approaches which may provide complete results but within response times that are too high to be useful within many practical applications. Results are presented from a set of sample queries comparing the IRI selection strategies with existing approaches showing that coverage, freshness and diversity can be improved by up to 30%.