Antonio Congiusta

Distributed data mining on grids: services, tools, and applications

Data mining algorithms are widely used today for the analysis of large corporate and scientific datasets stored in databases and data archives. Industry, science, and commerce fields often need to analyze very large datasets maintained over geographically distributed sites by using the computational power of distributed and parallel systems. The grid can play a significant role in providing an effective computational support for distributed knowledge discovery applications. For the development of data mining applications on grids we designed a system called KNOWLEDGE GRID. This paper describes the KNOWLEDGE GRID framework and presents the toolset provided by the KNOWLEDGE GRID for implementing distributed knowledge discovery. The paper discusses how to design and implement data mining applications by using the KNOWLEDGE GRID tools starting from searching grid resources, composing software and data components, and executing the resulting data mining process on a grid. Some performance results are also discussed.

Distributed data mining services leveraging WSRF

The continuous increase of data volumes available from many sources raises new challenges for their effective understanding. Knowledge discovery in large data repositories involves processes and activities that are computationally intensive, collaborative, and distributed in nature. The Grid is a profitable infrastructure that can be effectively exploited for handling distributed data mining and knowledge discovery. To achieve this goal, advanced software tools and services are needed to support the development of KDD applications. The Knowledge Grid is a high-level framework providing Grid-based knowledge discovery tools and services. Such services allow users to create and manage complex knowledge discovery applications that integrate data sources and data mining tools provided as distributed services on the Grid. All of these services are currently being re-designed and re-implemented as WSRF-compliant Grid Services. This paper highlights design aspects and implementation choices involved in such a process.

Service-oriented middleware for distributed data mining on the grid

Distribution of data and computation allows for solving larger problems and executing applications that are distributed in nature. The grid is a distributed computing infrastructure that enables coordinated resource sharing within dynamic organizations consisting of individuals, institutions, and resources. The grid extends the distributed and parallel computing paradigms allowing for resource negotiation and dynamical allocation, heterogeneity, open protocols, and services. Grid environments can be used both for compute-intensive tasks and data intensive applications by exploiting their resources, services, and data access mechanisms. Data mining algorithms and knowledge discovery processes are both compute and data intensive, therefore the grid can offer a computing and data management infrastructure for supporting decentralized and parallel data analysis. This paper discusses how grid computing can be used to support distributed data mining. Research activities in grid-based data mining and some challenges in this area are presented along with some promising future directions for developing grid-based distributed data mining.

A Data Mining Toolset For Distributed High-performance Platforms

Today a large number of scientific and commercial applications often require to analyse large data sets maintained over geographically distributed sites by using the computational power of distributed high-performance environments. Advances in networking technology and computational infrastructure made it possible to construct large-scale distributed computing platforms, called computational grids, that provide dependable, consistent, and pervasive access to high-end computational resources. Grids can play a significant role in providing an effective computational support for distributed data mining applications. Currently we are developing a software system for geographically distributed knowledge discovery applications called KNOWLEDGE GRID,which is designed on top of computational grid mechanisms, provided by grid environments such as Glob us. In this paper we present an integrated toolset named VEGA (Visual Environment for Grid Applications), which allows a Knowledge Grid user to develop and execute distributed data mining computations in a simple and effective way.

Grid-Based Data Mining and Knowledge Discovery

The increasing use of computers in all the areas of human activities is resulting in huge collections of digital data. Databases are common everywhere and are used as repositories of every kind of data. Knowledge discovery techniques and tools are used today to analyze those very large data sets to identify interesting patterns and trends in them. When data is maintained over geographically distributed sites the computational power of distributed and parallel systems can be exploited for knowledge discovery in databases. In this scenario the Grid can provide an effective computational support for distributed knowledge discovery on large data sets. To this purpose we designed a system called Knowledge Grid This chapter describes the Knowledge Grid architecture and discusses some related systems and models recently proposed for knowledge discovery on Grids. The chapter presents also how to design and implement distributed data mining applications by using the Knowledge Grid tools starting from searching Grid resources, composing software and data elements, and executing the resulting application on a Grid.

Developing distributed data mining applications in the knowledge grid framework

The development of data intensive and knowledge-based applications on Grids is a research area that today is receiving significant attention. One of the main topics in that area is the implementation of distributed data mining applications using Grid computing services and distributed resource management facilities. This paper describes the development process of distributed data mining applications on Grids by using the KNOWLEDGE GRID framework. After a quick introduction to the system principles and a description of tools and services it offers to users, the paper describes the design and implementation of two distributed data mining applications by using the KNOWLEDGE GRID features and tools and gives experimental results obtained by running the designed applications on real Grids.

PROTEUS: a Bioinformatics Problem Solving Environment on Grids

Bioinformatics can be considered as a bridge between life science and computer science, where high performance computational platforms and software are required to manage complex biological data. In this paper we present PROTEUS, a Grid-based Problem Solving Environment that integrates ontology and workflow approaches to enhance composition and execution of bioinformatics application on the Grid. Architecture and preliminary experimental results are reported.

Enabling Knowledge Discovery Services on Grids

The Grid is mainly used today for supporting high-performance compute intensive applications. However, it is going to be effectively exploited for deploying data-driven and knowledge discovery applications. To support these classes of applications, high-level tools and services are vital. The Knowledge Grid is a high-level system for providing Grid-based knowledge discovery services. These services allow professionals and scientists to create and manage complex knowledge discovery applications composed as workflows that integrate data sets and mining tools provided as distributed services on a Grid. This paper presents and discusses how knowledge discovery applications can be designed and deployed on Grids. The contribution of novel technologies and models such as OGSA, P2P, and ontologies is also discussed.

A data mining-based framework for grid workflow management

In this paper we investigate on the exploitation of data mining techniques to analyze data coming from the enactment of workflow-based processes in a service-oriented grid infrastructure. The extracted knowledge allows users to better comprehend the behavior of the enacted processes, and can be profitably exploited to provide advanced support to several phases in the life-cycle of workflow processes, including (re-)design, matchmaking, scheduling and performance monitoring. To this purpose, we focus on recent data mining techniques specifically aimed at enabling refined analyzes of workflow executions. Moreover, we introduce a comprehensive system architecture that supports the management of grid workflows by fully taking advantage of such mining techniques.

Wsrf-Based Services for Distributed Data Mining

Computational Grids can be effectively used as an infrastructure for distributed data mining and knowledge discovery in large data sets. To utilize Grids for high-performance knowledge discovery, software tools and mechanisms are needed. To this purpose we designed a system called Knowledge Grid and we are implementing its services as WSRF-compliant Grid Services. This chapter describes the composition of distributed knowledge discovery services, according to the service oriented architecture model, by using the Knowledge Grid environment. We discuss Grid Services for searching Grid resources, composing software and data elements, and executing the resulting data mining application on the Knowledge Grid. The chapter focuses in particular on the application modeling. Applications are designed using a UML model, which is translated into a BPEL representation, in turn processed by the Knowledge Grid services for its execution.