Robert S. Blackmore

Improving the Scalability of Parallel Jobs by adding Parallel Awareness to the Operating System

A parallel application benefits from scheduling policies that include a global perspective of the applications process working set. As the interactions among cooperating processes increase, mechanisms to ameliorate waiting within one or more of the processes become more important. In particular, collective operations such as barriers and reductions are extremely sensitive to even usually harmless events such as context switches among members of the process working set. For the last 18 months, we have been researching the impact of random short-lived interruptions such as timer-decrement processing and periodic daemon activity, and developing strategies to minimize their impact on large processor-count SPMD bulk-synchronous programming styles. We present a novel co-scheduling scheme for improving performance of fine-grain collective activities such as barriers and reductions, describe an implementation consisting of operating system kernel modifications and run-time system, and present a set of empirical results comparing the technique with traditional operating system scheduling. Our results indicate a speedup of over 300% on synchronizing collectives.

Dicke narrowing of the polarized Stokes–Raman Q branch of the v=0→1 transition of D2 in He

Ab initio calculations were carried out for several Stokes–Raman Q branch line shapes of the v=0→1 transition of D2 in helium in the low pressure regime where Dicke narrowing is observed. Five different approximations were studied: a generalization of Hess’s method, which is based on the equivalent of a Bhatnagar–Gross–Krook model of the Waldmann–Snider kinetic equation; two high pressure asymptotes of Hess’s method; a moment method of the Chapman–Cowling type due to Corey and McCourt; and a collision kernel method that has found extensive use lately in rarefied gas dynamics and other highly nonequilibrium states. Using generalized cross sections incorporating S‐matrices calculated in the close coupled approximation with an ab initio potential energy surface, most of the methods agreed moderately well with the most recent line width determinations at most densities. Of the five methods, the generalized Hess and collision kernel methods agreed best with experiment over the whole range. At the lowest densit...

Comment on linewidths and shifts in the Stokes–Raman Q branch of D2 in He

Collision induced widths and shifts for Raman Q‐branch transitions of D2 in He were calculated from S matrices obtained with converged close coupling scattering calculations on an accurate theoretical interaction potential. Results agree well with experimental values. Discrepancies between experimental line shifts and theoretical values from an earlier study are traced to a computational error in that work. The effects of vibrational inelasticity and of centrifugal distortion on energy levels and on potential matrix elements, all of which were ignored in the earlier study, are explicitly considered here and found to be small.

Breaking the connection: RDMA deconstructed

The architecture, design and performance of RDMA (remote direct memory access) over the IBM HPS (high performance switch and adapter) are described. Unlike conventional implementations such as InfiniBand, our RDMA transport model is layered on top of an unreliable datagram interface, while leaving the task of enforcing reliability to the ULP (upper layer protocol). We demonstrate that our model allows a single MPI task to deliver bidirectional bandwidth of close to 3.0 GB/s across a single link and 24.0 GB/s when striped across 8 links. In addition, we show that this transport protocol has superior attributes in terms of a) being able to handle RDMA packets coming out of order; b) being able to use multiple routes between a source-destination pair and c) reducing the size of adapter caches.

Optimization of Message Passing Services on POWER8 InfiniBand Clusters

We present scalability and performance enhancements to MPI libraries on POWER8 InfiniBand clusters. We explore optimizations in the Parallel Active Messaging Interface (PAMI) libraries. We bypass IB VERBS via low level inline calls resulting in low latencies and high message rates. MPI is enabled on POWER8 by extension of both MPICH and Open MPI to call PAMI libraries. The IBM POWER8 nodes have GPU accelerators to optimize floating throughput of the node. We explore optimized algorithms for GPU-to-GPU communication with minimal processor involvement. We achieve a peak MPI message rate of 186 million messages per second. We also present scalable performance in the QBOX and AMG applications.