José L. Roda

Integral knapsack problems: parallel algorithms and their implementations on distributed systems

The parallelization of the dynamic programming algorithm for the integral knapsack problem is approached from several perspectives. Two of them proceed by dividing the set of objects, while a third one proceeds by partitioning the set of capacities. Furthermore, we propose a new sequential algorithm and its parallelization by reducing the integral knapsack problem to a maximum path problem. The theoretical complexity analysis of the algorithms proves that for all the algorithms the product of the number of processors by the parallel time equals the corresponding sequential time. Computational results are presented both for transputer networks using occam and LAN using PVM. Although for many cases the best running times are obtained for the LAN, the speedup and the scalability are better for the transputer network.

Design of parallel algorithms for the single resource allocation problem

Abstract Three new optimal parallel algorithms are presented for the single resource allocation problem. They run in the simplest networks: pipelines and rings. All of them have been implemented using PVM and MPI. Four representative platforms of the current state of parallel computing hardware were used for the experiences presented in this paper: the IBM SP2, the Cray T3E, the Silicon Origin 2000 and the Digital Alpha Server 8400. Computational results prove the good scalability of these three algorithms for practical cases.

A Skeleton for Parallel Dynamic Programming

The development of skeleton tools constitutes an alternative to cover the gap between current parallel architectures and sequential programmers. Its contruction involves formal models, paradigms and methologies. Based in the automata theory we have developed a formal model for Parallel Dynamic Programming over pipeline networks. This model makes up a paradigm which is the core of skeleton tools oriented to the Dynamic Programming Technique. Following the methodology coerced by the model, we present a tool that provides the user with the ability to obtain parallel programs adapted to the parallel architecture. The efficiency is contrasted on three current parallel platforms: Cray T3E, IBM SP2 and SG Origin 2000.

Parallel dynamic programming and automata theory

Abstract Following Karps discrete Dynamic Programming (DP) approach, this work extends the sequential model for monadic DP to the parallel case. We propose general parallel DP algorithms for pipeline and ring networks. The study of the optimality of these algorithms leads us to the introduction of new classes of multistage automata. However, the important class of Polyadic Dynamic Problems is out of the scope of this theory. Based on the concept of tree automaton, a new theory covering both Sequential and Parallel Dynamic Programming Polyadic Programs is presented. Monadic Programs appear as a particular case of this new theory. A large number of experiments have proved that the schemes proposed in this work lead to efficient implementations.

A new parallel model for the analysis of asynchronous algorithms

The BSP model barrier synchronization imposes some limits both in the range of available algorithms and also in their performance. Although BSP programs can be translated to MPI/PVM programs, the counterpart is not true. The asynchronous nature of some MPI/PVM programs does not easily fit inside the BSP model. Through the suppression of barriers and the generalization of the concept of superstep we propose two new models, the BSP-like and the BSP without barriers (BSPWB) models. While the BSP-like extends the BSP* model to programs written using collective operations, the more general BSPWB model admits the MPI/PVM parallel asynchronous programming style. The parameters of the models and their quality are evaluated on four standard parallel platforms: the CRAY T3E, the IBM SP2, the Origin 2000 and the Digital Alpha Server 8400. The study shows that the time spent in an h-relation is more independent on the number of processors than on the communication pattern. We illustrate the use of these BSP extensions through two problem-solving paradigms: the Nested Parallel Recursive Divide and Conquer Paradigm and the Virtual Pipeline Dynamic Programming Paradigm. The proposed paradigms explain how nested parallelism and processor virtualization can be introduced in MPI and PVM without having any negative impact in the performance and model accuracy. The prediction of the communication times is robust even for problems, where communication is dominated by small messages.

h-Relation Models for Current Standard Parallel Platforms

This paper studies the validity of the BSP h-relation hypothesis on four current standard parallel platforms. The error introduced by the influence of the number of processors is measured on five communication patterns. We also measure the influence of the communication patterns on the time invested in an h-relation. The asynchronous nature of many current standard message passing programs do not easily fits inside the BSP model. Often this has been criticized as the most serious drawback of BSP. Based in the h-relation hypothesis we propose an extension to BSP model valid for standard message passing parallel programs. The use and accuracy of h-relation models on standard message passing programs are illustrated using a parallel algorithm to compute the Discrete Fast Fourier Transform.

Paradigms for parallel dynamic programming

We extend the sequential model for dynamic programming to a parallel model. We propose three general parallel dynamic programming algorithms for pipeline and ring networks for multistage automatons. The study of the optimality lead us to the introduction of two new classes of multistage automatons: nondecreasing automatons and strongly increasing automatons. As an example, this parallel dynamic programming approach is applied to the single resource allocation problem. Results both for transputer networks and for local area networks using PVM are reported. The experience proves that the proposed algorithms can be easily and efficiently implemented. Furthermore, these procedures constitute a suitable kernel to build general parallel tools for dynamic programming.

Groups in bulk synchronous parallel computing

An extension to the Bulk Synchronous Parallel Model (BSP) to allow the use of asynchronous BSP groups of processors is presented. In this model, called Nested BSP, processor groups can be divided and processors in a group synchronize through group dependent collective operations generalizing the concept of barrier synchronization. A classification of problems and algorithms attending to their parallel input-output distribution is provided. For one of these problem classes, the called common-common class, we present a general strategy to derive efficient parallel algorithms. Algorithms belonging to this class allow the arbitrary division of the processor subsets, easing the opportunities of the underlying BSP software to divide the network in independent sub networks, minimizing the impact of the traffic in the rest of the network in the predicted cost. The expressiveness of the model is exemplified through three divide and conquer programs. The computational results for these programs in six high performance supercomputers show both the accuracy of the model and the optimality of the speedups for the class of problems considered.

Predicting the Performance of Injection Communication Patterns on PVM

The range of communication performance on current parallel computers goes from slow LANs to high speed switches. To find a general, efficient and easy to use complexity model seems to be a big challenge. This difficulty can be relieved if the parallel algorithm is designed using a small class of basic communication patterns as building blocks. Our approach is to obtain different formulas to predict the performance of each of these building blocks. One of those communications schemes is the general class of injection patterns. Although the value obtained for the bandwidth using the classical ping-pong algorithm is suitable to estimate the communication time of injection patterns for high speed networks, our experiments show its inaccuracy for Ethernet networks. In this paper, we introduce a computational complexity method to predict the performance of injection patterns that reduces the communication architecture factor to a few parameters. The computational results prove the correctness of the predictions for both parallel multicomputers and networks of workstations. These results show also how it can be used to determine the appropriate number of processors to obtain maximum efficiency.

Practical Experiments to Improve PVM Algorithms

This paper describes several experiments to measure different network parameters under PVM. Mapping of tasks over different branches of the network and different broadcasting strategies based on pvm_mcast and pvm_send are compared. Intensive communication experiments have been also measured. As an example of how to apply this information, we broach the implementation of two parallel algorithms using PVM to perform the communications on a LAN.