Irene Moulitsas

Architecture Aware Partitioning Algorithms

Existing partitioning algorithms provide limited support for load balancing simulations that are performed on heterogeneous parallel computing platforms. On such architectures, effective load balancing can only be achieved if the graph is distributed so that it properly takes into account the available resources (CPU speed, network bandwidth). With heterogeneous technologies becoming more popular, the need for suitable graph partitioning algorithms is critical. We developed such algorithms that can address the partitioning requirements of scientific computations, and can correctly model the architectural characteristics of emerging hardware platforms.

Multilevel Algorithms for Generating Coarse Grids for Multigrid Methods

Geometric Multigrid methods have gained widespread acceptance for solving large systems of linear equations, especially for structured grids. One of the challenges in successfully extending these methods to unstructured grids is the problem of generating an appropriate set of coarse grids. The focus of this paper is the development of robust algorithms, both serial and parallel, for generating a sequence of coarse grids from the original unstructured grid. Our algorithms treat the problem of coarse grid construction as an optimization problem that tries to optimize the overall quality of the resulting fused elements. We solve this problem using the multilevel paradigm that has been very successful in solving the related grid/graph partitioning problem. The parallel formulation of our algorithm incurs a very small communication overhead, achieves high degree of concurrency, and maintains the high quality of the coarse grids obtained by the serial algorithm.

A grid-free abstraction of the Navier-Stokes equations in Fortran 95/2003

Computational complexity theory inspires a grid-free abstraction of the Navier-Stokes equations in Fortran 95/2003. A novel complexity analysis estimates that structured programming time grows at least quadratically with the number of program lines. Further analysis demonstrates how an object-oriented strategy focused on mathematical objects renders the quadratic estimate scale-invariant, so the time required for the limiting factor in program development (debugging) no longer grows as the code grows. Compared to the coordinate-free C++ programming of Grant et al. [2000], grid-free Fortran programming eliminates a layer of procedure calls, eliminates a related need for the C++ template construct, and offers a shorter migration path for Fortran programmers. The grid-free strategy is demonstrated by constructing a physical-space driver for a Fourier-space Navier-Stokes solver. Separating the expression of the continuous mathematical model from the discrete numerics clarifies issues that are otherwise easily conflated. A run-time profile suggests that grid-free design substantially reduces the fraction of the procedures that significantly impact runtime, freeing more code to be structured in ways that reduce development time. Applying Amdahls law to the total solution time (development time plus run time) leads to a strategy that negligibly impacts development time but achieves 58% of the maximum possible speedup.

Dispersed-phase structural anisotropy in homogeneous magnetohydrodynamic turbulence at low magnetic Reynolds number

A new tensor statistic, the dispersed-phase structure dimensionality Dp, is defined to describe the preferred orientation of clusters of discrete bodies. The evolution of Dp is calculated via direct numerical simulations of passive, Stokesian particles driven by initially isotropic, decaying magnetohydrodynamic turbulence. Results are presented for five magnetic field strengths as characterized by magnetic interaction parameters, N, in the range 0–50. Four field strengths are studied at a grid resolution of 1283. The strongest field strength is also studied at 2563 resolution. In each case, the externally applied magnetic field was spatially uniform and followed a step function in time. Particles with initially uniform distributions were tracked through hydrodynamic turbulence for up to 2800 particle response times before the step change in the magnetic field. In the lower resolution simulation, the particle response time, τp, matched the Kolmogorov time scale at the magnetic field application time t0. Th...

MPI to Coarray Fortran: Experiences with a CFD Solver for Unstructured Meshes

High-resolution numerical methods and unstructured meshes are required in many applications of Computational Fluid Dynamics (CFD). These methods are quite computationally expensive and hence benefit from being parallelized. Message Passing Interface (MPI) has been utilized traditionally as a parallelization strategy. However, the inherent complexity of MPI contributes further to the existing complexity of the CFD scientific codes. The Partitioned Global Address Space (PGAS) parallelization paradigm was introduced in an attempt to improve the clarity of the parallel implementation. We present our experiences of converting an unstructured high-resolution compressible Navier-Stokes CFD solver from MPI to PGAS Coarray Fortran. We present the challenges, methodology, and performance measurements of our approach using Coarray Fortran. With the Cray compiler, we observe Coarray Fortran as a viable alternative to MPI. We are hopeful that Intel and open-source implementations could be utilized in the future.

Performance Evaluation of a Two-Dimensional Lattice Boltzmann Solver Using CUDA and PGAS UPC Based Parallelisation

The Unified Parallel C (UPC) language from the Partitioned Global Address Space (PGAS) family unifies the advantages of shared and local memory spaces and offers a relatively straightforward code parallelisation with the Central Processing Unit (CPU). In contrast, the Computer Unified Device Architecture (CUDA) development kit gives a tool to make use of the Graphics Processing Unit (GPU). We provide a detailed comparison between these novel techniques through the parallelisation of a two-dimensional lattice Boltzmann method based fluid flow solver. Our comparison between the CUDA and UPC parallelisation takes into account the required conceptual effort, the performance gain, and the limitations of the approaches from the application oriented developers’ point of view. We demonstrated that UPC led to competitive efficiency with the local memory implementation. However, the performance of the shared memory code fell behind our expectations, and we concluded that the investigated UPC compilers could not efficiently treat the shared memory space. The CUDA implementation proved to be more complex compared to the UPC approach mainly because of the complicated memory structure of the graphics card which also makes GPUs suitable for the parallelisation of the lattice Boltzmann method.

STEADY FLOW OF A TWO-DIMENSIONAL LIQUID CURTAIN UNDER PRESSURE

Abstract We use finite elements and the full-Newton iteration method to solve the steady, two-dimensional flow of a Newtonian planar film issuing from a slit under a pressure difference, in the presence of gravity and surface tension. The simulated film shapes agree with available experimental data within the range of the experimental error. The numerical calculations show that the shape of the film depends strongly on the imposed pressure difference, inertia and gravity, and is rather insensitive to surface tension.