Yasuhiro Watashiba

Architecture of a High-Speed MPI_Bcast Leveraging Software-Defined Network

Collective communication is of great importance in MPI because the execution time of an MPI program is affected by the communication performance it can gain. Particularly these days, when a cluster system composed of multiple computing nodes has become dominant as a large-scale computing system, the execution time of collective communication affects the total execution time of the MPI program. However, in many implementations of MPI, collective communication is developed to make use of unicast-based communication in a repeated and combined way, which may result in inefficient communication. In this paper, we explore the use of a Software-Defined Network, which was originally expected to help network administrators operate networks through central control in a software-programming manner, to accelerate MPI_Bcast, a basic collective communication used in MPI. The evaluation in this paper indicates that our prototyped SDN_MPI_Bcast is superior to MPI_Bcast in OpenMPI in communication performance. Also, the evaluation implies that SDN_MPI_Bcast is feasible.

Efficacy Analysis of a SDN-enhanced Resource Management System through NAS Parallel Benchmarks

In the field of social science, a variety of high-performance computing simulations such as the Monte Carlo simulation and the Multi-agent simulation must be efficiently performed to deal with social scientific big data. To facilitate social scientists in performing their own analysis against such big data, the information infrastructure for social science must be equipped with a core technology that efficiently and effectively leverages limited resources available on the information infrastructure. From such a perspective, a new type of job management technology, which treats not only computational resources such as the Central Processing Unit (CPU) and memory, but also network resources unlike traditional job management, is investigated in this paper. A cluster system with a fat-tree topology interconnect is conventional cluster architecture these days. For this investigation, the National Aeronautics Space Administration Advanced Supercomputing, USA (NAS) Parallel Benchmarks, which contain computation patterns often observed in social scientific simulations, are used to assess the efficacy of the resource allocation by our proposed job management technology on a cluster system with a fat-tree topology interconnect.

Performance evaluation of SDN-enhanced MPI allreduce on a cluster system with fat-tree interconnect

Nowadays, supercomputers play an essential role in high-performance computing. In general, modern supercomuputers are built as a cluster system, which is a system of multiple computers interconnected on a network. In coding a parallel program on such a cluster system, MPI (Message Passing Interface) is utilized. In this paper, we aim to reduce the execution time of MPI Allreduce, a frequently used MPI collective communication in many simulation codes. To this end, we have integrated network programmability by Software Defined Networking into MPI Allreduce so that it effectively uses the bandwidth of the interconnect of the cluster system. An experiment conducted on a cluster system with fat-tree interconnect indicates that our proposed MPI Allreduce is superior to MPI Allreduce in OpenMPI implementations.

SAGE-based Tiled Display Wall enhanced with dynamic routing functionality triggered by user interaction

To empower scientists who are engaged in nation-wide or global-scale collaborative projects for scientific discovery, a large amount of scientific data needs to be visualized and then shared among the scientists. Tiled Display Wall?(TDW) has been widely accepted and used for visualization of large-scale scientific data. Scalable Adaptive Graphics Environment?(SAGE) has received attention from scientists as a middleware that organizes multiple display monitors into a network-aware large display monitor. Using a SAGE TDW, scientists can display multiple visualized contents on a single display monitor, each of which can be located at geographically distant site managed by other organizations. However, SAGE does not have a mechanism for managing multiple visualized data streams heading to a single TDW. In a conventional network, data flows for a same destination tend to share a same link, resulting in drop of packets and therefore poor visual quality. Moreover, because of the flexible nature of SAGE, rate of each visual data flow may change dynamically as a result of user interaction on a TDW, such as moving and resizing an application window. For the reason above, we propose and develop a dynamic route allocation method that switches packet flows onto network links where better performance is expected, in response to user interaction such as window movement and resizing. Technically, we have leveraged OpenFlow, an implementation of Software Defined Networking?(SDN), to integrate network programmability into SAGE. In this paper, we show how SAGE enhanced with the proposed method succeeded in avoiding network congestion and improving the quality of visualization on the TDW over the wide area OpenFlow network on the Internet. SAGE is a TDW middleware that can receive multiple streams from multi site.Dynamic changes of network traffic caused by user-interaction in SAGE.We modified SAGE with dynamic routing functionality triggered by user interaction.Dynamic routing functionality is implemented using OpenFlow.It aims to avoid network congestion for keeping high quality visualization.

Performance Characteristics of an SDN-Enhanced Job Management System for Cluster Systems with Fat-Tree Interconnect

In the era of cloud computing, data centers that accommodate a series of user-requested jobs with a diversity of resource usage pattern need to have the capability of efficiently distributing resources to each user job, based on individual resource usage patterns. In particular, for high-performance computing as a cloud service which allows many users to benefit from a large-scale computing system, a new framework for resource management that treats not only the CPU resources, but also the network resources in the data center is essential. In this paper, an SDN-enhanced JMS that efficiently handles both network and CPU resources and as a result accelerates the execution time of user jobs is introduced as a building block technology for such a HPC cloud. Our evaluation shows that the SDN-enhanced JMS efficiently leverages the fat-tree interconnect of cluster systems running behind the cloud to suppress the collision of communications generated by different jobs.

An Architectural Design of a Job Management System Leveraging Software Defined Network

Whether to be able to efficiently utilize the interconnect in a cluster system is a key factor in deciding the computational performance especially for a class of jobs that require intensive communications between processes. However, most of Job Management Systems (JMSs), which are deployed on cluster systems for load-balancing, do not have any mechanism that takes the pattern and requirements of communication occurred in a job into consideration, regardless of potential large impact on its performance. In this research, we explore a novel JMS that can assign computing resources to jobs submitted to the JMS from a standpoint of efficient use of both network and processor resource. More technically, in this paper, a JMS that integrates the network programming functionality of OpenFlow as Software Defined Network is proposed and discussed towards better and more efficient allocation of computing and network resources.

An Empirical Study of SDN-accelerated HPC Infrastructure for Scientific Research

High performance computing is required for Big Science application because the proliferation and huge amount of scientific data that needs to be analyzed is a serious problem. Traditionally, network resources were generally assumed as a static resource users cannot control on demand. By integrating network programmability to every stage of a scientific workflow, this study explores a next-generation high performance computing infrastructure where both computational and network resources are flexibly sliced and efficiently leveraged based on the resource requirements of the scientific applications. Technically, Software Defined Networking has been adopted as a key technology for this purpose. In this paper the concept and goals of a next-generation high performance computing infrastructure is introduced and the current status of our research is discussed.

Prototyping and evaluation of a network-aware Job Management System on a cluster system

Network performance in high-performance computing environments such as supercomputers and Grid systems takes a role of great importance in deciding the overall performance of computation. However, most Job Management Systems (JMSs) available today, which are responsible for managing multiple computing resources for distribution and balancing of a computational workload, do not consider network awareness for resource management and allocation. In this paper, the authors briefly overview our proposed and prototyped network-aware JMS that can allocate an appropriate set of computing and network resources to a job request. Also, we evaluate the usefulness and effectiveness of our proposal. Experiments conducted with the prototype implementation imply that our proposed network-aware JMS could reduce job execution time by 23.4 percent.

Hydra: A High-throughput Virtual Screening Data Visualization and Analysis Tool☆

Abstract Virtual high-throughput biochemical screening offers a cost-effective alternative to the empirical testing of millions of compounds. However, virtual screening data often requires some manual processing of the data to eliminate false positives, evaluate the ligand-macromolecule fit, and identify new molecular interactions. This analysis is generally hindered by highly specific software and hardware requirements and complex user interfaces. Hydra is an HTML5 and JavaScript based application, which ameliorates this issue by displaying ligand-macromolecule models calculated by virtual screening programs in a single, simple online interface. The application is capable of loading raw data sets from the DOCK virtual screening platform and utilizing pre-processed datasets from other software to display compounds side-by-side in a user-defined size grid of 3Dmol.js instances. It also searches databases for selected compound information to natively display within the interface. This tool provides a highly accessible platform for streamlined virtual screening results analysis.

Adaptive Network Resource Reallocation for Hot-Spot Avoidance on SDN-Based Cluster System

On a cluster system running behind the Cloud computing, multiple processes are generated from most applications and then executed on multiple computing nodes. Their processes communicate with each other during their execution. The communication performance among multiple processes plays an important role in the total execution performance of an application. The SDN-enhanced JMS, which we have developed in the previous work, offers the framework that allows administrators to prescribe resource provisioning way using the information on both computing and network resources on a cluster system. However, an advanced and effective method for providing an appropriate set of computing and network resources to jobs is still an issue to be tackled. In this paper, we aim to design and implement an adaptive network resource allocation method with which network resources can be reallocated to jobs in execution, in response to the change of process placement occurred at job dispatch and termination events. Our evaluation shows that the proposed method can reallocate network resources of jobs in execution, and suppress the degradation of network throughput of jobs.