Edward de Oliveira Ribeiro

Towards a Hybrid Federated Cloud Platform to Efficiently Execute Bioinformatics Workflows

Current generation of high-throughput DNA sequencing machines [1, 35, 66] can generate large amounts of DNA sequence data. For example, the machine HiSeq 2000 from the company Illumina, a current workhorse of genome centers, is capable of generating 600 Giga base-pairs of sequence in one single run [35]. The Human Microbiome project (https://commonfund.nih.gov/hmp) and the 1000 Genomes project (http://www.1000genomes.org) are two examples of projects that are generating terabyte-scale amounts of DNA sequence.

ACOsched: A scheduling algorithm in a federated cloud infrastructure for bioinformatics applications

Task scheduling in a federated cloud environment is a complex problem since there are several cloud providers presenting distinct memory and storage capacities that should be addressed. This article focus on the task scheduling problem in BioNimbuZ, a federated cloud infrastructure for executing bioinformatics applications, which was previously proposed by our group. We present a scheduling algorithm based on Load Balancing Ant Colony (LBACO), called ACOsched, to perform efficient distribution of tasks by finding the best cloud in the federation to execute these tasks. We developed experiments using real biological data, executing the Bowtie mapping tool on one instance of BioNimbuZ, composed by two cloud providers, Amazon EC2 and a bioinformatics laboratory at the University of Brasilia/Brazil. The obtained results show that ACOsched led to a significant improvement in the makespan time of Bowtie executing in BioNimbuZ, when compared to the simple round robin algorithm called DynamicAHP, previously developed in this federated cloud infrastrucutre.

A Storage Policy for a Hybrid Federated Cloud platform: A Case Study for Bioinformatics

Bioinformatics tools require large-scale processing mainly due to very large databases achieving gigabytes of size. In federated cloud environments, although services and resources may be shared, storage is particularly difficult, due to distinct computational capabilities and data management policies of several separated clouds. In this work, we propose a storage policy for BioNimbuZ, a hybrid federated cloud platform designed to execute bioinformatics applications. Our storage policy, BioClouZ, aims to perform efficient choices to distribute and replicate files to the best available cloud resources in the federation in order to reduce computational time. BioClouZ uses four parameters - latency, uptime, free size and cost, weighted (according to ad hoc tests) to model their influences to data storage and recovery. Experiments were performed with real biological data executing a commonly used tool to map short reads in a reference genome in BioNimbuZ, composed of clouds executing in Amazon EC2, Azure and University of Brasilia. The results showed that, when compared to the greedy algorithm first used in BioNimbuZ, the BioClouZ policy significantly improved the total execution time due to more efficient choices of the clouds to store the files. Other bioinformatics applications can be used with BioClouZ in BioNimbuZ as well, since the platform was designed independently from particular tools and databases.

BioNimbuZ: A federated cloud platform for bioinformatics applications

Challenges in bioinformatics include tools to treat large-scale processing, mainly due to the large volumes of data generated by high-throughput sequencing machines. Besides, many of these tools are not user friendly, and do not distribute their workloads properly. In federated cloud environments, even though services and resources are shared and available online, the processes of a workflow execution are almost entirely not automated, and the majority of these processes do not efficiently balance their workloads. This paper presents the federated cloud platform, called BioNimbuZ, a hybrid platform designed to execute bioinformatics applications easily and efficiently, with good workload balance. Our tests were performed using a real bioinformatics workflow, with fragments generated by the Illumina sequencer, having achieved good performance in practice.

p2pBIOFOCO: Proposing a Peer-to-Peer System for Distributed BLAST Execution

The peer-to-peer (P2P) model is currently being used for applications that demand great amount of time and space resources, taking advantage of idle and geographically distributed machines. Bioinformatics is a field that requires large amount of computational resources. In this context, we propose a P2P model that executes a particular bioinformatics application, named BLAST. This is a method for comparing two biological sequences at a time. But it requires a great amount of computation, since it compares a big file containing thousands of sequences that are being investigated and a very large database containing millions of biological sequences. We describe in detail the architecture of our framework, and present experiments using real biological data, that were executed on machines available in three institutions of Brazil - University of Brasilia, Catholic University of Brasilia and Embrapa/Genetic Resources and Biotechnology. The obtained results, when comparing the execution of BLAST in this P2P framework and its execution in the best machine of the three institutions, show that our P2P model can efficiently run other bioinformatics applications.

An Evaluation of Data Model for NoSQL Document-Based Databases

NoSQL databases offer flexibility in the data model. The document-based databases may have some data models built with embedded documents, and others made with referenced documents. The challenge lies in choosing the structure of the data. This paper proposes a study to analyze if different data models can have an impact on the performance of database queries. To this end, we created three data models: embedded, referenced, and hybrid. We ran experiments on each data model in a MongoDB cluster, comparing the response time of 3 different queries in each model. Results showed a disparity in performance between the data models. We also evaluated the use of indexes in each data model. Results showed that, depending on the type of query and field searched some types of indexes presented higher performance compared to others. Additionally, we carried out an analysis of the space occupied on the storage disk. This analysis shows that the choice of model also affects disk space for storing data and indexes.