Shujia Zhou

A lightweight I/O scheme to facilitate spatial and temporal queries of scientific data analytics

In the era of petascale computing, more scientific applications are being deployed on leadership scale computing platforms to enhance the scientific productivity. Many I/O techniques have been designed to address the growing I/O bottleneck on large-scale systems by handling massive scientific data in a holistic manner. While such techniques have been leveraged in a wide range of applications, they have not been shown as adequate for many mission critical applications, particularly in data postprocessing stage. One of the examples is that some scientific applications generate datasets composed of a vast amount of small data elements that are organized along many spatial and temporal dimensions but require sophisticated data analytics on one or more dimensions. Including such dimensional knowledge into data organization can be beneficial to the efficiency of data post-processing, which is often missing from exiting I/O techniques. In this study, we propose a novel I/O scheme named STAR (Spatial and Temporal AggRegation) to enable high performance data queries for scientific analytics. STAR is able to dive into the massive data, identify the spatial and temporal relationships among data variables, and accordingly organize them into an optimized multi-dimensional data structure before storing to the storage. This technique not only facilitates the common access patterns of data analytics, but also further reduces the application turnaround time. In particular, STAR is able to enable efficient data queries along the time dimension, a practice common in scientific analytics but not yet supported by existing I/O techniques. In our case study with a critical climate modeling application GEOS-5, the experimental results on Jaguar supercomputer demonstrate an improvement up to 73 times for the read performance compared to the original I/O method.

Accelerating Climate and Weather Simulations Through Hybrid Computing

Unconventional multi- and many-core processors (e.g., IBM® Cell B.E. ™ and NVIDIA® GPU) have emerged as effective accelerators in trial climate and weather simulations. Yet these climate and weather models typically run on parallel computers with conventional processors (e.g., Intel®, AMD®, and IBM) using Message Passing Interface (MPI). To address challenges involved in efficiently and easily connecting accelerators to parallel computers, we investigated using IBM’s Dynamic Application Virtualization (DAV) software in a prototype hybrid computing system with representative climate and weather model components. The hybrid system comprises 2 Intel blades and 2 IBM QS22 Cell B.E. blades, connected with both InfiniBand® (IB) and 1-Gigabit Ethernet. The system significantly accelerates a solar radiation model component by offloading compute-intensive calculations to the Cell blades. Systematic tests show that DAV can seamlessly offload compute-intensive calculations from Intel blades to Cell B.E. blades in a scalable, load-balanced manner. However, noticeable communication overhead was observed, mainly due to IP over IB protocol. Full utilization of IB Sockets Direct Protocol (SDP) and the lower latency production version of DAV will reduce this overhead.

Application controlled parallel asynchronous IO

Filesystems continue to be a major performance bottleneck for many applications across a variety of hardware architectures. Most existing attempts to address this issue, e.g., PVFS, rely upon system resources which are not typically tuned for any specific user application, whereas others rely on special hardware capabilities such as shared-memory.We have developed an MPI-based parallel asynchronous I/O (PAIO) software package which enables applications to balance compute and I/O resources directly. PAIO uses a queueing mechanism to stage the data, sent over in parallel from compute nodes, on the reserved I/O nodes. Because the bandwidth of the inter-processor network greatly exceeds that of the filesystem, significant performance improvements can be achieved under a bursty I/O load provided sufficient memory is available for the I/O nodes. The results of PAIO for typical weather applications on an SGI Altix and other architectures will be reported in a poster board.

Accelerating climate and weather simulations through hybrid computing

Unconventional multi- and many-core processors (e.g., IBM® Cell B.E. ™ and NVIDIA® GPU) have emerged as effective accelerators in trial climate and weather simulations. Yet these climate and weather models typically run on parallel computers with conventional processors (e.g., Intel®, AMD®, and IBM) using Message Passing Interface (MPI). To address challenges involved in efficiently and easily connecting accelerators to parallel computers, we investigated using IBM’s Dynamic Application Virtualization (DAV) software in a prototype hybrid computing system with representative climate and weather model components. The hybrid system comprises 2 Intel blades and 2 IBM QS22 Cell B.E. blades, connected with both InfiniBand® (IB) and 1-Gigabit Ethernet. The system significantly accelerates a solar radiation model component by offloading compute-intensive calculations to the Cell blades. Systematic tests show that DAV can seamlessly offload compute-intensive calculations from Intel blades to Cell B.E. blades in a scalable, load-balanced manner. However, noticeable communication overhead was observed, mainly due to IP over IB protocol. Full utilization of IB Sockets Direct Protocol (SDP) and the lower latency production version of DAV will reduce this overhead.