Taesang Huh

Implementation of AMGA GUI Client Toolkit : AMGA Manager

AMGA service, which is one of the EMI gLite middleware components, is widely used for analysis of distributed large scale experiments data as metadata repository by scientific and technological researchers and the use of AMGA is extended farther to include general industries needing metadata Catalogue as well. However AMGA, based unix and Grid UI, has the weakness of being absence of general-purpose user interfaces in comparison to other commercial database systems and that`s why it`s difficult to use and diffuse it although it has the superiority of the functionality. In this paper, we developed AMGA GUI toolkit to provide work convenience using object-oriented modeling language(UML). Currently, AMGA has been used as the main component among many user communities such as Belle II, WISDOM, MDM, and so on, but we expect that this development can not only lower the barrier to entry for AMGA beginners to use it, but lead to expand the use of AMGA service over more communities.

Directory search performance optimization of AMGA for the Belle II experiment

AMGA (ARDA Metadata Grid Application) is a grid metadata catalogue system that has been developed as a component of the EU FP7 EMI consortium based on the requirements of the HEP (High-Energy Physics) and the biomedical user communities. Currently, AMGA is exploited to manage the metadata in the gBasf2 framework at the Belle II experiment, one of the largest particle physics experiments in the world. In this paper, we present our efforts to optimize the metadata query performance of AMGA to better support the massive MC Campaign of the Belle II experiment. Although AMGA exhibits very outstanding performance for a relatively small amount of data, as the number of directories and the metadata size increase (e.g. hundreds of thousands of directories) during the MC Campaign, AMGA suffers from severe query processing performance degradation. To address this problem, we modified the query search mechanism and the database scheme of AMGA to provide dramatic improvements of metadata search performance and query response time. Throughout our comparative performance analysis of metadata search operations, we show that AMGA can be an optimal solution for a metadata catalogue in a large-scale scientific experimental framework

Implementation and Performance Analysis of Hadoop MapReduce over Lustre Filesystem

Hadoop is becoming widely adopted in scientific and commercial areas as an open-source distributed data processing framework. Recently, for real-time processing and analysis of data, an attempt to apply high-performance computing technologies to Hadoop is being made. In this paper, we have expanded the Hadoop Filesystem library to support Lustre, which is a popular high-performance parallel distributed filesystem, and implemented the Hadoop MapReduce execution environment over the Lustre filesystem. We analysed Hadoop MapReduce over Lustre by using Hadoop standard benchmark tools. We found that Hadoop MapReduce over Lustre execution has a performance 2-13 times better than a typical Hadoop MapReduce execution.

Improvement of AMGA Python Client Library for Belle II Experiment

This paper describes the recent improvement of the AMGA (ARDA Metadata Grid Application) python client library for the Belle II Experiment. We were drawn to the action items related to library improvement after in-depth discussions with the developer of the Belle II distributed computing system. The improvement includes client-side metadata federation support in python, DIRAC SSL library support as well as API refinement for synchronous operation. Some of the improvements have already been applied to the AMGA python client library as bundled with the Belle II distributed computing software. The recent mass Monte- Carlo (MC) production campaign shows that the AMGA python client library is reliably stable.