José Luis Bosque

Study of neural net training methods in parallel and distributed architectures

Artificial Neural Nets are among the most commonly used methods in high-energy applications for data pre-processing. The training phase of the ANN is critical in obtaining a net that can generalize the available data for use in new situations. However, from the computational viewpoint this phase is very costly and resource intensive. Therefore, the aim of this work is to parallelize and evaluate the performance and scalability of the kernel of a training algorithm of a multilayer perceptron artificial neural net used for analyzing data from the Large Electron Positron Collider at CERN. The training methods selected were linear-BFGS and hybrid linear-BFGS. Different approaches for the parallel implementation will be presented and evaluated in this paper. In order to perform a complete performance and scalability evaluation of the proposed approach, three different parallel architectures will be used: A shared memory multiprocessor, a cluster and a grid environment.

An agents-based cooperative awareness model to cover load balancing delivery in grid environments

This paper presents an extension of the AMBLE model, an awareness model which manage load balancing by means of a multi-agent based architecture, with the aim to establish a cooperative load balancing model for collaborative grid environments. This model, named C-AMBLE (Cooperative Awareness Model for Balancing the Load in grid Environments) applies some theoretical principles of multi-agents systems, awareness models, and third party models, to promote an efficient autonomous cooperative task delivery in grid environments. This cooperative task management, implemented using web services, has been tested in a real and heterogeneous grid infrastructure with very successful results. This paper presents some of these outcomes while emphasizing on the performance speedup of the system using this model.

HLogGP: a new parallel computational model for heterogeneous clusters

Heterogeneous clusters claim for new models and algorithms. In this paper a new parallel computational model is presented. The model, based on the LogGP model, has been extended to be able to deal with heterogeneous parallel systems. For that purpose, the LogGPs scalar parameters have been replaced by vector and matrix parameters to take into account the different nodes features. The work presented here includes the parameterization of a real cluster which illustrates the impact of node heterogeneity over the models parameters. Finally, the paper presents some experiments performed in a real heterogeneous cluster that can be used for assessing the methods validity, together with the main conclusions and future work.

Static Multi-device Load Balancing for OpenCL

This paper presents the Load Balancing for OpenCL (lbcl) library, devoted to automatically solve load balancing issues on both multi-platform and heterogeneous environments. Using this library, a single kernel can be executed on a set of heterogeneous devices, giving each device an amount of work proportional to its computing power. A wrapper has been developed so the library can balance the workload of an existing application not only without introducing any changes into its source code, but without any recompilation stage. Also a general OpenCL profiler has been developed to easily do a detailed profiling of the obtained results.

Managing dynamic virtual organizations to get effective cooperation in collaborative grid environments

This paper presents how to manage Virtual Organizations to enable efficient collaboration and/or cooperation as a result of a flexible and parametrical model. The CAM (Collaborative/Cooperative Awareness Management) model promotes collaboration around resources-sharing infrastructures, endorsing interaction by means of a set of rules. This model focuses on responding to specific demanding circumstances at a given moment, while optimizes resources communication and behavioural agility to get a common goal: the establishment of collaborative dynamic virtual organizations. This paper also describes how CAM works in some specific examples and scenarios, and how the CAM Rules-Based Management Application (based on Web Services and named WS-CAM) has been designed and validated to encourage resources to be involved in collaborative performances, tackling efficiently demanding situations without hindering the own purposes of each of these resources.

Dealing with Heterogeneity in Load Balancing Algorithms

Cluster heterogeneity increases the difficulty of balancing the load across the system nodes. Although the relationship between heterogeneity and load balancing is difficult to describe analytically, in this paper an exhaustive analysis of the effects of this system feature on load balancing algorithms performance is presented. Considering the performed analysis, there are two main challenges that need to be faced when dealing with cluster heterogeneity in load balancing algorithms: one related to the state measurement stage and another to the initiation rule. In this paper techniques to deal with heterogeneity in these two algorithm stages are proposed. Furthermore, suggestions to improve the performance of the rest of algorithm stages in heterogeneous environments are made too

Parallel CBIR implementations with load balancing algorithms

The purpose of content-based information retrieval (CBIR) systems is to retrieve, from real data stored in a database, information that is relevant to a query. When large volumes of data are considered, as it is very often the case with databases dealing with multimedia data, it may become necessary to look for parallel solutions in order to store and gain access to the available items in an efficient way.Among the range of parallel options available nowadays, clusters stand out as flexible and cost effective solutions, although the fact that they are composed of a number of independent machines makes it easy for them to become heterogeneous. This paper describes a heterogeneous cluster-oriented CBIR implementation. First, the cluster solution is analyzed without load balancing, and then, a new load balancing algorithm for this version of the CBIR system is presented.The load balancing algorithm described here is dynamic, distributed, global and highly scalable. Nodes are monitored through a load index which allows the estimation of their total amount of workload, as well as the global system state. Load balancing operations between pairs of nodes take place whenever a node finishes its job, resulting in a receptor-triggered scheme which minimizes the systems communication overhead. Globally, the CBIR cluster implementation together with the load balancing algorithm can cope effectively with varying degrees of heterogeneity within the cluster; the experiments presented within the paper show the validity of the overall strategy.Together, the CBIR implementation and the load balancing algorithm described in this paper span a new path for performant, cost effective CBIR systems which has not been explored before in the technical literature.

A rule based resources management for collaborative grid environments

Something that is still missing, but strongly needed, in collaborative grid environments is a stable, flexible and dynamic resource management. This management should optimise collaboration and cooperation among several resources keeping resources constraints, preconditions and rules. This paper presents how to achieve these objectives, by means of a Collaborative Awareness Management (CAM) model. CAM optimises resources collaboration, promotes resources cooperation and responds to the specific demanded circumstances. This paper also describes how this model works in some specific examples and scenarios, emphasising on how the WS-CAM Rules-Based Management Application has been designed, implemented, and validated to accomplish these purposes.

Shot boundary detection using Zernike moments in multi-GPU multi-CPU architectures

This paper presents an analysis of a Multi-GPU Multi-CPU environment, along with the different possible hybrid combinations. The analysis has been performed for a shot boundary detection application, based on Zernike moments, although it is general enough to be applied to many different application areas. A deep study of the performance, bottlenecks and design challenges is carried out showing the validity of this approach and achieving very high frame per second rates. In this paper, Zernike calculations are carried out on GPUs, taking advantage of a packing strategy proposed to minimize host-device communication time.

Leveraging OmpSs to Exploit Hardware Accelerators

CUDA and OpenCL are the most widely used programming models to exploit hardware accelerators. Both programming models provide a C-based programming language to write accelerator kernels and a host API used to glue the host and kernel parts. Although this model is a clear improvement over a low-level and ad-hoc programming model for each hardware accelerator, it is still too complex and cumbersome for general adoption. For large and complex applications using several accelerators, the main problem becomes the explicit coordination and management of resources required between the host and the hardware accelerators that introduce a new family of issues (scheduling, data transfers, synchronization, ) that the programmer must take into account. In this paper, we propose a simple extension to OmpSs -- a data-flow programming model -- that dramatically simplifies the integration of accelerated code, in the form of CUDA or OpenCL kernels, into any C, C++ or Fortran application. Our proposal fully replaces the CUDA and OpenCL host APIs with a few pragmas, so we can leverage any kernel written in CUDA C or OpenCL C without any performance impact. Our compiler generates all the boilerplat code while our runtime system takes care of kernels scheduling, data transfers between host and accelerators and synchronizations between host and kernels parts. To evaluate our approach, we have ported several native CUDA and OpenCL applications to OmpSs by replacing all the CUDA or OpenCL API calls by a few number of pragmas. The OmpSs versions of these applications have competitive performance and scalability but with a significantly lower complexity than the original ones.