Porfidio Hernández

CISNE: a new integral approach for scheduling parallel applications on non-dedicated clusters

Our main interest is oriented towards keeping both local and parallel jobs together in a non-dedicated cluster. In order to obtain some profits from the parallel applications, it is important to consider time and space sharing as a mean to enhance the scheduling decisions. In this work, we introduce an integral scheduling system for non-dedicated clusters, termed CISNE. It includes both a previously developed dynamic coscheduling system and a space-sharing job scheduler to make better scheduling decisions than can be made separately. CISNE allows multiple parallel applications to be executed concurrently in a non dedicated Linux cluster with a good performance, as much from the point of view of the local user as that of the parallel application user. This is possible without disturbing the local user and obtaining profits for the parallel user. The good performance of CISNE has been evaluated in a Linux cluster.

n-step FM-Index for Faster Pattern Matching

Fast pattern matching is a requirement for many problems, specially for bioinformatics sequence analysis like short read mapping applications. This work presents a variation of the FM-index method, denoted n-step FM-index, that is applied in exact match genome search. We propose an alternative two-dimensional FM-index structure that allows backward-search navigation giving steps of n symbols at a time. The main advantages of this arrangement are the reduction of the computational work, but most importantly, the reduction by n of the chain of dependent data accesses, and the increase in the temporal locality of the data access pattern. This benefit comes at the expense of increasing the total amount of data required for the index. We present an in-depth performance analysis of a multi-core implementation of the algorithm using large references (up to 1.5G). We identify memory latency as the major performance limiter for single-thread execution and memory bandwidth for multi-thread execution. Our proposal provides speedups ranging from 1.4× to 2.4×, when there is no limitation on DRAM capacity. We also analyse the trade-off of compacting the proposed data structure in order to reduce memory capacity requirements, now at the expense of increasing execution time. An extra 33% of DRAM space allows our proposal to improve performance by 1.2×, while doubling DRAM size enables an additional 1.5×. Our proposal of n-step algorithm provides an alternative for pseudo-random memory access algorithms to be redesigned to scale in current and future computer systems.

Coscheduling and Multiprogramming Level in a Non-dedicated Cluster

Our interest is oriented towards keeping both local and parallel jobs together in a time-sharing non-dedicated cluster. In such systems, dynamic coscheduling techniques, without memory restriction, that consider the MultiProgramming Level for parallel applications (MPL), is a main goal in current cluster research. In this paper, a new technique called Cooperating Coscheduling (CCS), that combines a dynamic coscheduling system and a resource balancing schema, is applied.

Designing a Video-on-Demand System for a Brazilian High Speed Network

Despite the availability of video-on-demand (VoD) services in a number of cities around the world, the large-scale deployment of VoD services has not been widely explored due to economic constraints. The high complexity founded in these systems makes the design process very difficult. This study presents a VoD system design for a highspeed metropolitan network, based on ATM technology, located in Brazil. To accomplish this goal, we have utilized our own simulation-based tool for VoD design called VODSim. This tool allows VoD research on any VoD platform, simulating the complete flow path followed by requests from clients to servers passing through the network infrastructure, allowing the configuration and tuning of multiple system parameters. Using VODSim, we analyse possible VoD architectures for the proposal environment (i.e. Centralized-server, Independent-servers, One-level-proxies and Distributedservers), studying key-parameters that have influence in system design. Experimental results show that Distributed-server architecture is the one that adjusts better to the analyzed infrastructure, improving streaming capacity by 16%, 25% and 28% more than One-level-proxies, Centralized and Independent-servers respectively. Taking in account future growth in VoD system, the Distributed-server even provides an unlimited scalability through new local network incorporation and using P2P techniques.

Distributed P2P merging policy to decentralize the multicasting delivery

Advances in network technology make multicast one of the most feasible video streaming delivery techniques for the near future. However, the scalability of a multicast VoD system is limited by the server bandwidth. In this paper, we propose a new multicast delivery scheme that allows every active client to collaborate with the server in order to scale the VoD system performance beyond the servers physical limitations. The solution combined the multicast delivery scheme and peer-2-peer paradigm in order to decentralize the delivery process. The new video delivery scheme is able to merge two or more multicast channels using distributed collaborations between a group of clients. We compared the new policy with chaining and patching schemes and the experimental results showed that our policy is better than previous schemes in terms of reduction of resource requirements and local network load. Compared with multicast patching policy, the new scheme reduced the resource requirement up to 77.5% while the local network load was 66.9% lower than a peer-2-peer chaining policy.

Implementing Explicit and Implicit Coscheduling in a PVM Environment

Our efforts are directed towards the understanding of the coscheduling mechanism in a NOW system when a parallel job is executed with local workloads, balancing parallel efficiency against the local interactive response. Explicit and implicit coscheduling techniques in a PVM-Linux NOW (or cluster) has been implemented. Their performance and overheads executing local tasks and representative distributed benchmarks have been analyzed and compared.

Job scheduling for optimizing data locality in Hadoop clusters

We describe the use of non-dedicated clusters by a known group of local applications sharing the computational resources with additional bioinformatics MapReduce applications. We have studied how to effectively use the resources shared by both application types during their execution. In order to keep local application execution times unaffected we consider the configuration of a group of parameters of the Hadoop platform. One of the most relevant aspects to consider is the job scheduling policy. Our aim is to allow that tasks from different jobs that handle the same data blocks are grouped to be run on the same node where the blocks are allocated. Experimental results show that our approach outperforms traditional policies.

Predictive Coscheduling Implementation in a Non-dedicated Linux Cluster

Our research is focussed on keeping both local and parallel jobs together in a non-dedicated cluster or NOW (Network Of Workstations) and efficiently scheduling them by means of coscheduling mechanisms. A real implementation of a predictive coscheduling technique in a Linux cluster is presented in this article and its performance analyzed and compared with other coscheduling algorithms in the literature.

Impact of task duplication on static-scheduling performance in multiprocessor systems with variable execution-time tasks

The problem of scheduling a set of partially ordered tasks on a nonpreemptive multiprocessor system of identical processors assuming that the execution time of some tasks can vary within a set of known values is studied in an effort to construct a more realistic static schedule. A new heuristic algorithm (CP/MISF/TD) based on task duplication is proposed. The effectiveness of the algorithm is proved by comparing the results obtained for a wide variety of task graphs with the ones obtained by applying the classical list scheduling algorithms which consider fixed time tasks. Keywords: Static scheduling, List scheduling, Heuristic algorithms, Task duplication, Multiprocessor systems.

Dynamic distributed collaborative merging policy to optimize the multicasting delivery scheme

The advance of Internet 2 and the proliferation of switches and routers with level three functionalities made the multicast one of the most feasible video streaming delivering techniques for the near future. Assuming this to be true, this study addressed the over-load situation that a streaming server could suffer due to client requests. As a solution, we proposed new multicast delivery scheme that allows every active client to collaborate with the server regardless of the video that they are watching, alleviating server loads, and therefore server resource requirements. The solution combined the multicast delivery scheme and client-side buffer collaboration in order to decentralize the delivery process. The new video delivering scheme was designed as two separate policies: the first policy used client collaboration to deliver first part of videos and the second policy could merge two or more multicast channels using distributed collaboration between a group of clients. Experimental results show that this scheme is better than previous schemes in terms of resource requirements and scalability.