Stéphane Genaud

Single Node On-Line Simulation of MPI Applications with SMPI

Simulation is a popular approach for predicting the performance of MPI applications for platforms that are not at ones disposal. It is also a way to teach the principles of parallel programming and high-performance computing to students without access to a parallel computer. In this work we present SMPI, a simulator for MPI applications that uses on-line simulation, i.e., the application is executed but part of the execution takes place within a simulation component. SMPI simulations account for network contention in a fast and scalable manner. SMPI also implements an original and validated piece-wise linear model for data transfer times between cluster nodes. Finally SMPI simulations of large-scale applications on large-scale platforms can be executed on a single node thanks to techniques to reduce the simulations compute time and memory footprint. These contributions are validated via a large set of experiments in which SMPI is compared to popular MPI implementations with a view to assess its accuracy, scalability, and speed.

Toward Better Simulation of MPI Applications on Ethernet/TCP Networks

Simulation and modeling for performance prediction and profiling is essential for developing and maintaining HPC code that is expected to scale for next-generation exascale systems, and correctly modeling network behavior is essential for creating realistic simulations. In this article we describe an implementation of a flow-based hybrid network model that accounts for factors such as network topology and contention, which are commonly ignored by other approaches. We focus on large-scale, Ethernet-connected systems, as these currently compose 37.8 % of the TOP500 index, and this share is expected to increase as higher-speed 10 and 100GbE become more available. The European Mont-Blanc project, which studies exascale computing by developing prototype systems with low-power embedded devices, uses Ethernet-based interconnect. Our model is implemented within SMPI, an open-source MPI implementation that connects real applications to the SimGrid simulation framework. SMPI provides implementations of collective communications based on current versions of both OpenMPI and MPICH. SMPI and SimGrid also provide methods for easing the simulation of large-scale systems, including shadow execution, memory folding, and support for both online and offline (i.e., post-mortem) simulation. We validate our proposed model by comparing traces produced by SMPI with those from real world experiments, as well as with those obtained using other established network models. Our study shows that SMPI has a consistently better predictive power than classical LogP-based models for a wide range of scenarios including both established HPC benchmarks and real applications.

On the efficiency of several VM provisioning strategies for workflows with multi-threaded tasks on clouds

Cloud computing promises the delivery of on-demand pay-per-use access to unlimited resources. Using these resources requires more than a simple access to them as most clients have certain constraints in terms of cost and time that need to be fulfilled. Therefore certain scheduling heuristics have been devised to optimize the placement of client tasks on allocated virtual machines. The applications can be roughly divided in two categories: independent bag-of-tasks and workflows. In this paper we focus on the latter and investigate a less studied problem, i.e., the effect the virtual machine allocation policy has on the scheduling outcome. For this we look at how workflow structure, execution time, virtual machine instance type affect the efficiency of the provisioning method when cost and makespan are considered. To aid our study we devised a mathematical model for cost and makespan in case single or multiple instance types are used. While the model allows us to determine the boundaries for two of our extreme methods, the complexity of workflow applications calls for a more experimental approach to determine the general relation. For this purpose we considered synthetically generated workflows that cover a wide range of possible cases. Results have shown the need for probabilistic selection methods in case small and heterogeneous execution times are used, while for large homogeneous ones the best algorithm is clearly noticed. Several other conclusions regarding the efficiency of powerful instance types as compared to weaker ones, and of dynamic methods against static ones are also made.

Schlouder: A broker for IaaS clouds

In the field of cloud computing, Infrastructure as a Service (IaaS) provides virtualized on-demand computing resources on a pay-per-use model. IaaS Cloud differ from traditional mutualized infrastructures in that the resources can be dynamically claimed and released, and the real hardware infrastructure is unknown to its users. These properties drastically changes the way resource pro-visioning and job scheduling can be addressed by the user because i) the large number of jobs and resources to handle becomes rapidly overwhelming for human operators, and ii) the real performances of the platform should be inferred from observations to make robust scheduling decisions. In order to optimize the resources usage by the client, we advocate the need for brokers on the client-side. This article presents our work based on Schlouder, a broker of IaaS cloud resources able to provision and schedule independent jobs or static work-flows according to strategies chosen by the client. Further, we advocate that simulation can be a precious auxiliary to help the user to choose between provi-sioning strategies. Schlouder brings a unique feature which is to predict through simulation the makespan and cost of executions under various strategies. The contribution of this work is twofold. First, it presents the broker, available as an open source project, in which new provisioning strategies can be plugged in by third parties. The effectiveness of the tool is demonstrated through experiments involving actual applications and platforms. Second, we show that simulation produces accurate predictions making this feature a helpful means for the user to choose the appropriate strategy.

Refinement of data parallel programs in PEI

Parallel programs mainly differ from sequential ones in that they include geometrical aspects involved by the hardware architecture. We present in this paper the PEI formalism, which enables to take into account both the geometrical and functional aspects of programs. It provides a refinement calculus mainly used to transform the geometrical characteristics of parallel programs, and we show how it may apply on data parallel programs, in particular for data alignments.

Parallelization of an Advection-Diffusion Problem Arising in Edge Plasma Physics Using Hybrid MPI/OpenMP Programming

This work presents a hybrid MPI/OpenMP parallelization strategy for an advection-diffusion problem, arising in a scientific application simulating tokamak’s edge plasma physics. This problem is the hotspot of the system of equations numerically solved by the application. As this part of the code is memory-bandwidth limited, we show the benefit of a parallel approach that increases the aggregated memory bandwidth in using multiple computing nodes. In addition, we designed some algorithms to limit the additional cost, induced by the needed extra inter nodal communications. The proposed solution allows to achieve good scalings on several nodes and to observe 70 % of relative efficiency on 512 cores. Also, the hybrid parallelization allows to consider larger domain sizes, unreachable on a single computing node.

Optimization and parallelization of Emedge3D on shared memory architecture

This report presents a study of techniques used to speedup a scientific simulation code. The techniques include sequential optimizations as well as the parallelization with OpenMP. This work is carried out on two different multicore shared memory architectures, namely a cutting edge 8x8 core CPU and a more common 2x6 core board. Our target application is representative of many memory bound codes, and the techniques we present show how to overcome the burden of the memory bandwidth limit, which is quickly reached on multi-core or many-core with shared memory architectures. To achieve efficient speedups, strategies are applied to lower the computation costs, and to maximize the use of processors caches. Optimizations are: minimizing memory accesses, simplifying and reordering computations, and tiling loops. On 12 cores processor Intel X5675, aggregation of these optimizations results in an execution time 21.6 faster, compared to the original version on one core.

Improving Cloud Simulation Using the Monte-Carlo Method.

In the cloud computing model, cloud providers invoice clients for resource consumption. Hence, tools helping the client to budget the cost of running his application are of pre-eminent importance. However, the opaque and multi-tenant nature of clouds make task runtimes variable and hard to predict, and hamper the creation of reliable simulation tools. In this paper, we propose an improved simulation framework that takes into account this variability using the Monte-Carlo method.

Transformation Techniques in PEI

This article presents a few examples of program transformation strategies in the language Pei [Vio94]. Three strategies are developed: a simplification of the communications, the introduction of broadcasts by removing recursion from data field definitions, and the introduction of a reduction operator. These transformations emphasize the relationships between several programs solving a given problem, especially in the data parallelism area.