Jonathan Passerat-Palmbach

Distribution of Random Streams for Simulation Practitioners

There is an increasing interest in the distribution of parallel random number streams in the high‐performance computing community particularly, with the manycore shift. Even if we have at our disposal statistically sound random number generators according to the latest and thorough testing libraries, their parallelization can still be a delicate problem. Indeed, a set of recent publications shows it still has to be mastered by the scientific community. With the arrival of multi‐core and manycore processor architectures on the scientist desktop, modelers who are non‐specialists in parallelizing stochastic simulations need help and advice in distributing rigorously their experimental plans and replications according to the state of the art in pseudo‐random numbers parallelization techniques. In this paper, we discuss the different partitioning techniques currently in use to provide independent streams with their corresponding software. In addition to the classical approaches in use to parallelize stochastic simulations on regular processors, this paper also presents recent advances in pseudo‐random number generation for general‐purpose graphical processing units. The state of the art given in this paper is written for simulation practitioners. Copyright

Pseudo-Random Number Generation on GP-GPU

Random number generation is a key element of stochastic simulations. It has been widely studied for sequential applications purposes, enabling us to reliably use pseudo-random numbers in this case. Unfortunately, we cannot be so enthusiastic when dealing with parallel stochastic simulations. Many applications still neglect random stream parallelization, leading to potentially biased results. Particular parallel execution platforms, such as Graphics Processing Units (GPUs), add their constraints to those of Pseudo-Random Number Generators (PRNGs) used in parallel. It results in a situation where potential biases can be combined to performance drops when parallelization of random streams has not been carried out rigorously. Here, we propose criteria guiding the design of good GPU-enabled PRNGs. We enhance our comments with a study of the techniques aiming to correctly parallelize random streams, in the context of GPU-enabled stochastic simulations.

Parallel stochastic simulations with rigorous distribution of pseudo-random numbers with DistMe: Application to life science simulations

This paper presents an open source toolkit allowing a rigorous distribution of stochastic simulations. It is designed according to the state of the art in pseudo‐random numbers partitioning techniques. Based on a generic XML format for saving pseudo‐random number generator states, each state contains adapted metadata. This toolkit named DistMe is usable by modelers who are non‐specialists in parallelizing stochastic simulations, it helps in distributing the replications and in the generation of experimental plans. It automatically writes ready for runtime scripts for various parallel platforms, encapsulating the burden linked to the management of status files for different pseudo‐random generators. The automation of this task avoids many human mistakes. The toolkit has been designed based on a model driven engineering approach: the user builds a model of its simulation and the toolkit helps in distributing independent stochastic experiments. In this paper, the toolkit architecture is exposed, and two examples in life science research domains are detailed. The preliminary design of the DistMe toolkit was achieved when dealing with the distribution of a nuclear medicine application using the largest European computing grid: European Grid Initiative (EGI). Thanks to our alpha version of the software toolbox, the equivalent of 3u2009years of computing was achieved in a few days. Next, we present the second application in another domain to show the potential and genericity of the DistMe toolkit. A small experimental plan with 1024 distributed stochastic experiments was run on a local computing cluster to explore scenarios of an environmental application. For both applications, the proposed toolkit was able to automatically generate distribution scripts with independent pseudo‐random number streams, and it also automatically parameterized the simulation input files to follow an experimental design. The automatic generation of scripts and input files is achieved, thanks to model transformations using a model driven approach. Copyright

Pseudo-random streams for distributed and parallel stochastic simulations on GP-GPU

Random number generation is a key element of stochastic simulations. It has been widely studied for sequential applications purposes, enabling us to reliably use pseudo-random numbers in this case. Unfortunately, we cannot be so enthusiastic when dealing with parallel stochastic simulations. Many applications still neglect random stream parallelization, leading to potentially biased results. In particular parallel execution platforms, such as Graphics Processing Units (GPUs), add their constraints to those of Pseudo-Random Number Generators (PRNGs) used in parallel. This results in a situation where potential biases can be combined with performance drops when parallelization of random streams has not been carried out rigorously. Here, we propose criteria guiding the design of good GPU-enabled PRNGs. We enhance our comments with a study of the techniques aiming to parallelize random streams correctly, in the context of GPU-enabled stochastic simulations.

ShoveRand: A model-driven framework to easily generate random numbers on GP-GPU

Stochastic simulations are often sensitive to the randomness source that characterizes the statistical quality of their results. Consequently, we need highly reliable Random Number Generators (RNGs) to feed such applications. Recent developments try to shrink the computation time by using more and more General Purpose Graphics Processing Units (GP-GPUs) to speed-up stochastic simulations. Such devices bring new paral-lelization possibilities, but they also introduce new programming difficulties. Since RNGs are at the base of any stochastic simulation, they also need to be ported to GP-GPU. There is still a lack of well-designed implementations of quality-proven RNGs on GP-GPU platforms. In this paper, we introduce ShoveRand, a framework defining common rules to generate random numbers uniformly on GP-GPU. Our framework is designed to cope with any GPU-enabled development platform and to expose a straightforward interface to users. We also provide an existing RNG implementation with this framework to demonstrate its efficiency in both development and ease of use.

TaskLocalRandom: a statistically sound substitute to pseudorandom number generation in parallel java tasks frameworks

Several software efforts have been produced over the past few years in various programming languages to help developers handle pseudorandom streams partitioning. Parallel and distributed stochastic simulations can obviously benefit from this kind of high‒level tools. The latest release of the Java Development Kit (JDK 7) tries to tackle this problem by providing facilities to partition a pseudorandom stream across various threads thanks to the new class ThreadLocalRandom. Meanwhile, Java 7 offers a framework to split a problem in a divide and conquer way through the new class called ForkJoinPool. As any other Java thread pool, ForkJoin exploits threads as workers and manipulates the tasks that will be run on the workers. In ThreadLocalRandom, pseudorandom number generation is handled at a thread level. As a consequence, a scientific application taking advantage of a Java thread pool to parallelize its computation may suffer from a bad pseudorandom stream partitioning due to the behavior of ThreadLocalRandom. The present work introduces TaskLocalRandom, a task‒level alternative to ThreadLocalRandom that solves this partitioning problem and assigns an independent pseudorandom stream to each task run in the thread pool. TaskLocalRandom is compatible with existing Java thread pools such as Executors or ForkJoin. Copyright

ThreadLocalMRG32k3a: A statistically sound substitute to pseudorandom number generation in parallel Java applications

Parallel And Distributed Simulations (PADS) become more and more spread since scientists always want more accurate results in the shortest time. PADS are often sensitive to several parameters, and when they own a stochastic component, one has to ensure he knows how to correctly deal with randomness in a parallel application. This point can appear to be very tricky for non-experts, who might be tempted to move this part of the simulation aside. Several software efforts have been produced in various programming languages to help developers handle pseudorandom streams partitioning in PADS. However, these tools remain third-party libraries that need to be integrated in already existing applications, and that might be hardly swappable. The latest release of the Java Development Kit (JDK 7) tries to tackle this problem with a new class called ThreadLocalRandom. The latter is in charge of safe pseudorandom number generation across Java threads. The present work studies the pros and cons of this approach, and introduces ThreadLocalMRG32k3a, an alternative to ThreadLocalRandom that shows better results in terms of generation speed and statistical quality. ThreadLocalMRG32k3a respects the same Application Programming Interface (API) as ThreadLocalRandom, thus enabling clients to use it in place of its JDK counterpart at no cost.

Parallel stepwise stochastic simulation: harnessing GPUs to explore possible futures states of a chromosome folding model thanks to the possible futures algorithm (PFA)

For the sake of software compatibility, simulations are often parallelized without much code rewriting. Performances can be further improved by optimizing codes so that to use the maximum power offered by parallel architectures. While this approach can provide some speed-up, performance of parallelized codes can be strongly limited a priori because traditional algorithms have been designed for sequential technologies. Thus, additional increase of performance should ultimately rely on some redesign of algorithms.n Here, we redesign an algorithm that has traditionally been used to simulate the folding properties of polymers. We address the issue of performance in the context of biological applications, more particularly in the active field of chromosome modelling. Due to the strong confinement of chromosomes in the cells, simulation of their motion is slowed down by the laborious search for the next valid states to progress. Our redesign, that we call the Possible Futures Algorithm (PFA), relies on the parallel computation of possible evolutions of the same state, which effectively increases the probability to obtain a valid state at each step. We apply PFA on a GPU-based architecture, allowing us to optimally reduce the latency induced by the computation overhead of possible futures. We show that compared to the initial sequential model the acceptance rate of new states significantly increases without impacting the execution time. In particular, the stronger the confinement of the chromosome, the more efficient PFA becomes, making our approach appealing for biological applications.n While most of our results were obtained using Fermi architecture GPUs from NVIDIA, we highlight improved performance on the cutting-edge Kepler architecture K20 GPUs.

Prototyping parallel simulations on manycore architectures using Scala: A case study

At the manycore era, every simulation practitioner can take advantage of the computing horsepower delivered by the available high performance computing devices. From multicore CPUs (Central Processing Unit) to thousand-thread GPUs (Graphics Processing Unit), several architectures are now able to offer great speed-ups to simulations. However, it is often tricky to harness them properly, and even more complicated to implement a few declinations of the same model to compare the parallelizations. Thus, simulation practitioners would mostly benefit of a simple way to evaluate the potential benefits of choosing one platform or another to parallelize their simulations. In this work, we study the ability of the Scala programming language to fulfill this need. We compare the features of two frameworks in this study: Scala Parallel Collections and ScalaCL. Both of them provide facilities to set up a data-parallelism approach on Scala collections. The capabilities of the two frameworks are benchmarked with three simulation models as well as a large set of parallel architectures. According to our results, these two Scala frameworks should be considered by the simulation community to quickly prototype parallel simulations, and choose the target platform on which investing in an optimized development will be rewarding.

HPCS 2012 tutorials: Tutorial I: High performance computing in biomedical informatics

The last decade has witnessed significant developments in various aspects of Biomedical Informatics, including Bioinformatics, Medical Informatics, Public Health Informatics, and Biomedical Imaging. The explosion of medical and biological data requires an associated increase in the scale and sophistication of the automated systems and intelligent tools to enable the researchers to take full advantage of the available databases. The availability of vast amount of biological data continues to represent unlimited opportunities as well as great challenges in biomedical research. Developing innovative data mining techniques and clever parallel computational methods to implement them will surely play an important role in efficiently extracting useful knowledge from the raw data currently available. The proper integration of carefully selected/developed algorithms along with efficient utilization of High Performance Computing (HPC) systems form the key ingredients in the process of reaching new discoveries from biological data. This tutorial focuses on addressing several key issues related to the effective utilization of HPC in biomedical informatics research, in particular, how to efficiently utilize high performance systems in the analysis of massive biological data. A major issue is that regard is how to design energy-aware models for executing computationally-intensive biomedical applications on HPC systems. Another key issue is how to develop innovative network filters for massive biological networks that would utilize parallel algorithms to construct networks samplers that would preserve original network structures while uncovering new ones.