Vicente Blanco

Dynamic Load Balancing on Dedicated Heterogeneous Systems

Parallel computing in heterogeneous environments is drawing considerable attention due to the growing number of these kind of systems. Adapting existing code and libraries to such systems is a fundamental problem. The performance of this code is affected by the large interdependence between the code and these parallel architectures. We have developed a dynamic load balancing library that allows parallel code to be adapted to heterogeneous systems for a wide variety of problems. The overhead introduced by our system is minimal and the cost to the programmer negligible. The strategy was validated on several problems to confirm the soundness of our proposal.

Towards the Dynamic Load Balancing on Heterogeneous Multi-GPU Systems.

The advent of multicore systems, joined to the potential acceleration of the graphics processing units, alleviates some well known important architectural problems at the expense of a considerable increment of the programmability wall. The heterogeneity, both at architectural and programming level at the same time, raises the programming difficulties. Adapting existing code and libraries to such systems is a fundamental problem. The performance of this code is affected by the large interdependence between the code and the parallel architecture. We have developed a dynamic load balancing library that allows parallel code to be adapted to a wide variety of heterogeneous systems. The overhead introduced by our system is minimal and the cost to the programmer negligible. This system has been applied to solve load imbalance problems appearing in homogeneous and heterogeneous multi-GPU platforms. As case studies, we consider matrix multiply and resource allocation problems, in different heterogeneous scenarios in multi-GPU systems. The unbalanced nature of these algorithms and heterogeneous systems allowed us to test the success of our load balancing approach.

Predicting the performance of parallel programs

This work presents a new approach to the relation between theoretical complexity models and performance analysis and tuning. The analysis of an algorithm produces a complexity function that gives an approach to the asymptotic number of operations performed by the algorithm. The time spent on these operations depends on the software-hardware platform being used. Usually such platforms are described, from the performance point of view, through a number of parameters. Those parameters are evaluated by a benchmarking program. Though for a given available platform, the algorithmic constants associated with the complexity formula can be computed using multidimensional linear regression, there is still the problem of predicting the performance when the platform is not available. We introduce the concept of Universal Instruction Class and derive from it a set of equations relating the values of the algorithmic constants with the platform parameters. Due to the hierarchical design of current memory systems, the performance behavior of most algorithms varies in a small number of large regions corresponding to small size, medium size and large size inputs. The constants involved in the complexity formula usually have different values for these regions. Assuming we have a complexity formula for the memory resources, it is possible to find a partition of the input size space and the different values of the algorithmic constants. This way, though the complexity formula is the same, the family of constants provides the adaptability of the formula to the different stationary uses of the memory.

Adaptive load balancing of iterative computation on heterogeneous nondedicated systems

Dynamic load balancing in heterogeneous systems is a fundamental research topic in parallel computing due to the high availability of such systems. The efficient utilization of the heterogeneous resources can significantly enhance the performance of the parallel system. At the same time, adapting parallel codes to state-of-the-art parallel computers composed of heterogeneous multinode–multicore processors becomes a very hard task because parallel codes are highly dependent on the parallel architectures. That means that applications must be tailored requiring a great deal of programming effort. We have developed the ALBIC (Adaptive Load Balancing of Iterative Computation) system that allows for the dynamic load balancing of iterative codes in heterogeneous dedicated and nondedicated Linux based systems. In order to validate the system several parallel codes have been analyzed in different scenarios. The results show that the ALBIC approach achieves better performance than the other proposal. This lightweighted library eases porting homogeneous parallel codes to heterogeneous platforms, since the code intrusion is low and the programming effort is quite reduced.

Modeling and improving locality for the sparse-matrix-vector product on cache memories

Abstract A model for representing and improving the locality exhibited by the execution of sparse irregular problems is developed in this work. We focus on the product of a sparse matrix by a dense vector (SpM×V). We consider the cache memory as a representative level of the memory hierarchy. Locality is evaluated through four functions based on two parameters called entry matches and line matches. In order to increase the locality, two algorithms are applied: one based on the construction of minimum spanning trees and the other on the nearest-neighbor heuristic. These techniques were tested and compared with some standard ordering algorithms.

Measuring energy consumption using EML (energy measurement library)

Energy consumption and efficiency is a main issue in high performance computing systems in order to reach exascale computing. Researchers in the field are focusing their effort in reducing the first and increasing the latter while there is no current standard for energy measurement. Current energy measurement tools are specific and architectural dependent and this has to be addressed. By creating a standard tool, it is possible to generate independence between the experiments and the hardware, and thus, researchers effort can be focused in energy, by maximizing the portability of the code used for experimentation with the multiple architectures we have access nowadays. We present the energy measurement library (EML) library, a software library that eases the access to the energy measurement tools and can be easily extended to add new measurement systems. Using EML, it is viable to obtain architectural and algorithmic parameters that affect energy consumption and efficiency. The use of this library is tested in the field of the analytic modeling of the energy consumed by parallel programs.

Dynamic load balancing on heterogeneous multicore/multiGPU systems

Parallel computing in heterogeneous environments is drawing considerable attention due to the growing number of these kind of systems. Adapting existing code and libraries to such systems is a fundamental problem. The performance of this code is affected by the large interdependence between the code and these parallel architectures. We have developed a dynamic load balancing library that allows parallel code to be adapted to heterogeneous systems for a wide variety of problems. The overhead introduced by our system is minimal and the cost to the programmer negligible. The strategy was applied to a Dynamic Programming Algorithm, the Resource Allocation Problem. This code has been implemented on different heterogeneous architectures, including an heterogeneous cluster, a multicore system, a single GPU, and a multi-GPU system. The unbalance nature of the RAP algorithm shows the success of our load balancing library on such architectures.

Dynamic load balancing on heterogeneous multi-GPU systems

Actual HPC systems are composed by multicore processors and powerful graphics processing units. Adapting existing code and libraries to these new systems is a fundamental problem due to the important increment on programming difficulties. The heterogeneity, both at architectural and programming levels at the same time, raises the programmability wall. The performance of the code is affected by the large interdependence between the code and the parallel architecture. We have developed a dynamic load balancing library that allows parallel code to be adapted to a wide variety of heterogeneous systems. The overhead introduced by our system is minimal and the cost to the programmer negligible. This system has been successfully applied to solve load imbalance problems appearing in homogeneous and heterogeneous multiGPU platforms. We consider the Dynamic Programming technique as case of study to validate our proposals using different heterogeneous scenarios in multiGPU systems.

Accurate analytical performance model of communications in MPI applications

This paper presents a new LogP-based model, called LoOgGP, which allows an accurate characterization of MPI applications based on microbenchmark measurements. This new model is an extension of LogP for long messages in which both overhead and gap parameters perform a linear dependency with message size. The LoOgGP model has been fully integrated into a modelling framework to obtain statistical models of parallel applications, providing the analyst with an easy and automatic tool for LoOgGP parameter set assessment to characterize communications. The use of LoOgGP model to obtain a statistical performance model of an image deconvolution application is illustrated as a case of study.

Lightweight web services for high performace computing

Web Services-based technologies have emerged as a technological alternative for computational web portals. Facilitating access to distributed resources through web interfaces while simultaneously ensuring security is one of the main goals in most of the currently existing manifold tools and frameworks. OpenCF, the Open Source Computational Framework that we have developed, shares these objectives and adds others, like enforced portability, genericity, modularity and compatibility with a wide range of High Performance Computing Systems. OpenCF has been implemented using lightweight technologies (Apache + PHP), resulting in a robust framework ready to run out of the box that is compatible with standard security requirements.