Takashi Shimokawabe

Peta-scale phase-field simulation for dendritic solidification on the TSUBAME 2.0 supercomputer

The mechanical properties of metal materials largely depend on their intrinsic internal microstructures. To develop engineering materials with the expected properties, predicting patterns in solidified metals would be indispensable. The phase-field simulation is the most powerful method known to simulate the micro-scale dendritic growth during solidification in a binary alloy. To evaluate the realistic description of solidification, however, phase-field simulation requires computing a large number of complex nonlinear terms over a fine-grained grid. Due to such heavy computational demand, previous work on simulating three-dimensional solidification with phase-field methods was successful only in describing simple shapes. Our new simulation techniques achieved scales unprecedentedly large, sufficient for handling complex dendritic structures required in material science. Our simulations on the GPU-rich TSUBAME 2.0 super- computer at the Tokyo Institute of Technology have demonstrated good weak scaling and achieved 1.017 PFlops in single precision for our largest configuration, using 4,000 CPUs along with 16,000 CPU cores.

An 80-Fold Speedup, 15.0 TFlops Full GPU Acceleration of Non-Hydrostatic Weather Model ASUCA Production Code

Regional weather forecasting demands fast simulation over fine-grained grids, resulting in extremely memory- bottlenecked computation, a difficult problem on conventional supercomputers. Early work on accelerating mainstream weather code WRF using GPUs with their high memory performance, however, resulted in only minor speedup due to partial GPU porting of the huge code. Our full CUDA porting of the high- resolution weather prediction model ASUCA is the first such one we know to date; ASUCA is a next-generation, production weather code developed by the Japan Meteorological Agency, similar to WRF in the underlying physics (non-hydrostatic model). Benchmark on the 528 (NVIDIA GT200 Tesla) GPU TSUBAME Supercomputer at the Tokyo Institute of Technology demonstrated over 80-fold speedup and good weak scaling achieving 15.0 TFlops in single precision for 6956 x 6052 x 48 mesh. Further benchmarks on TSUBAME 2.0, which will embody over 4000 NVIDIA Fermi GPUs and deployed in October 2010, will be presented.

Prospects for GRB Science with the Fermi Large Area Telescope

The Large Area Telescope (LAT) instrument on the Fermi mission will reveal the rich spectral and temporal gamma-ray burst (GRB) phenomena in the >100 MeV band. The synergy with Fermis Gamma-ray Burst Monitor detectors will link these observations to those in the well explored 10-1000 keV range; the addition of the >100 MeV band observations will resolve theoretical uncertainties about burst emission in both the prompt and afterglow phases. Trigger algorithms will be applied to the LAT data both onboard the spacecraft and on the ground. The sensitivity of these triggers will differ because of the available computing resources onboard and on the ground. Here we present the LATs burst detection methodologies and the instruments GRB capabilities.

145 TFlops Performance on 3990 GPUs of TSUBAME 2.0 Supercomputer for an Operational Weather Prediction

Abstract Numerical weather prediction is one of the major applications in high performance computing and demands fast and high-precision simulation over fine-grained grids. While utilizing hundreds of CPUs is certainly the most common way to get high performance for large scale simulations, we have another solution to use GPUs as massively parallel computing platform. In order to drastically shorten the runtime of a weather prediction code, we rewrite its huge entire code for GPU computing from scratch in CUDA. The code ASUCA is a high resolution meso-scale atmosphere model that is being developed by the Japan Meteorological Agency for the purpose of the next-generation weather forecasting service. The TSUBAME 2.0 supercomputer, which is equipped with 4224 NVIDIA Tesla M2050 GPUs, has started operating in November 2010 at the Tokyo Institute of Technology. A benchmark on the 3990 GPUs on TSUBAME 2.0 achieves extremely high performance of 145 TFlops in single precision for 14368×14284×48 mesh. This paper also describes the multi-GPU optimizations introduced into the ASUCA porting on TSUBAME 2.0.

GPU phase-field lattice Boltzmann simulations of growth and motion of a binary alloy dendrite

A GPU code has been developed for a phase-field lattice Boltzmann (PFLB) method, which can simulate the dendritic growth with motion of solids in a dilute binary alloy melt. The GPU accelerated PFLB method has been implemented using CUDA C. The equiaxed dendritic growth in a shear flow and settling condition have been simulated by the developed GPU code. It has been confirmed that the PFLB simulations were efficiently accelerated by introducing the GPU computation. The characteristic dendrite morphologies which depend on the melt flow and the motion of the dendrite could also be confirmed by the simulations.

Ultra-large-scale phase-field simulation study of ideal grain growth

Grain growth, a competitive growth of crystal grains accompanied by curvature-driven boundary migration, is one of the most fundamental phenomena in the context of metallurgy and other scientific disciplines. However, the true picture of grain growth is still controversial, even for the simplest (or ‘ideal’) case. This problem can be addressed only by large-scale numerical simulation. Here, we analyze ideal grain growth via ultra-large-scale phase-field simulations on a supercomputer for elucidating the corresponding authentic statistical behaviors. The performed simulations are more than ten times larger in time and space than the ones previously considered as the largest; this computational scale gives a strong indication of the achievement of true steady-state growth with statistically sufficient number of grains. Moreover, we provide a comprehensive theoretical description of ideal grain growth behaviors correctly quantified by the present simulations. Our findings provide conclusive knowledge on ideal grain growth, establishing a platform for studying more realistic growth processes.Grain growth: Simulations elucidate statistical behaviorGrain growth under ideal conditions is simulated by phase-field simulations in ultra-large time and space scales to elucidate the statistical behaviors. A team led by Tomohiro Takaki at Kyoto Institute of Technology in Japan performed large scale phase-field simulations to study ideal grain growth behavior. The time and space scales used in the simulations are more than ten times larger than those in previous reports, enabling them to reach a true steady-state with a statistically significant number of grains. A comprehensive theoretical description was derived to understand the ideal grain growth behavior based on the simulations. The knowledge provided by these findings may offer a model to understand the effects of complicated factors present in real materials and thus establish a platform to study more realistic grain growth phenomena in the future.

GPU-accelerated 3D phase-field simulations of dendrite competitive growth during directional solidification of binary alloy

Phase-field method has emerged as the most powerful numerical scheme to simulate dendrite growth. However, most phase-field simulations of dendrite growth performed so far are limited to two-dimension or single dendrite in three-dimension because of the large computational cost involved. To express actual solidification microstructures, multiple dendrites with different preferred growth directions should be computed at the same time. In this study, in order to enable large-scale phase-field dendrite growth simulations, we developed a phase-field code using multiple graphics processing units in which a quantitative phase-field method for binary alloy solidification and moving frame algorithm for directional solidification were employed. First, we performed strong and weak scaling tests for the developed parallel code. Then, dendrite competitive growth simulations in three-dimensional binary alloy bicrystal were performed and the dendrite interactions in three-dimensional space were investigated.

High-productivity framework on GPU-rich supercomputers for operational weather prediction code ASUCA

The weather prediction code demands large computational performance to achieve fast and high-resolution simulations. Skillful programming techniques are required for obtaining good parallel efficiency on GPU supercomputers. Our framework-based weather prediction code ASUCA has achieved good scalability with hiding complicated implementation and optimizations required for distributed GPUs, contributing to increasing the maintainability, ASUCA is a next-generation high resolution meso-scale atmospheric model being developed by the Japan Meteorological Agency. Our framework automatically translates user-written stencil functions that update grid points and generates both GPU and CPU codes. User-written codes are parallelized by MPI with intra-node GPU peer-to-peer direct access. These codes can easily utilize optimizations such as overlapping technique to hide communication overhead by computation. Our simulations on the GPU-rich supercomputer TSUBAME 2.5 at the Tokyo Institute of Technology have demonstrated good strong and weak scalability achieving 209.6 TFlops in single precision for our largest model using 4,108 NVIDIA K20X GPUs.

Spectral-Lag Relations in GRB Pulses Detected with HETE-2

Using a pulse-fit method, we investigated the spectral lags between the traditional gamma-ray band (50–400 keV) and the X-ray band (6–25 keV) for 8 GRBs with known redshifts (GRB 010921, GRB 020124, GRB 020127, GRB 021211, GRB 030528, GRB 040924, GRB 041006, and GRB 050408), detected with the WXM and FREGATE instruments aboard the HETE-2 satellite. We found several relations for individual GRB pulses between the spectral lag and other observables, such as the luminosity, pulse duration, and peak energy, Epeak. The obtained results are consistent with those for BATSE, indicating that the BATSE correlations are still valid at lower energies (6–25 keV). Furthermore, we found that the photon energy dependence for the spectral lags can be reconciled with the simple curvature effect model. We discuss the implications of these results from various points of view.

Spectral Evolution of GRB060904A Observed with Swift and Suzaku- Possibility of Inefficient Electron Acceleration

We observed an X-ray afterglow of GRB 060904A with the Swift and Suzaku satellites. We found rapid spectral softening during both the prompt tail phase and the decline phase of an X-ray flare in the BAT and XRT data. The observed spectra were fit by power-law photon indices which rapidly changed from