Carlos R. Senna

Scheduling service workflows for cost optimization in hybrid clouds

Cloud computing has recently emerged as a convergence of concepts such as cluster computing, grid computing, utility computing, and virtualization. In hybrid clouds, the user has its private cloud available for use, but she can also request new resources to public clouds in a pay-per-use basis when there is an increase in demand. In this scenario it is important to decide when and how to request these new resources to satisfy deadlines and/or to get a reasonable execution time, while minimizing the monetary costs involved. In this paper we propose a strategy to schedule service workflows in a hybrid cloud. The strategy aims at determining which services should use paid resources and what kind of resource should be requested to the cloud in order to minimize costs and meet deadlines. Experiments suggest that the strategy can decrease the execution costs while maintaining reasonable execution times.

Enabling execution of service workflows in grid/cloud hybrid systems

Cloud computing systems provide on demand access to computational resources for dedicated use. Grid computing allows users to share heterogeneous resources from multiple administrative domains applied to common tasks. In this paper we discuss the characteristics and requirements of a hybrid system composed of both grid and cloud technologies. We propose an infrastructure which is able to manage the execution of service workflows in such system. We show how the infrastructure can be expanded by acquiring computational resources on demand from the cloud during the workflow execution, and how it manages these resources and the workflow execution without user interference.

Execution of service workflows in grid environments

Grid computing has emerged as a powerful environment for parallel processing. Nowadays, many organizations participate in one or more grid computing infrastructures, sharing computational resources to achieve high computational power and/or very large storage capacities. Mostly, these grids provide application interfaces for the user to submit jobs or workflows. These interfaces receive the submissions and distribute them among the grid resources. In this paper we evaluate a service-oriented grid testbed. Jobs are submitted through a workflow manager using a workflow composition language which allows services to be invoked sequentially or in parallel. Experimental results show that the overhead when using a service composition scheme does not prevent the grid from giving fast workflow execution. In our testbed, for a median filter application, we were able to get executions in the order of five times faster using the grid when compared to the local execution.

Bicriteria Service Scheduling with Dynamic Instantiation for Workflow Execution on Grids

Nowadays the grid is turning into a service-oriented environment. In this context, there exist solutions to the execution of workflows and most of them are web-service based. Additionally, services are considered to exist on a fixed host, limiting the resource alternatives when scheduling the workflow tasks. In this paper we address the problem of dynamic instantiation of grid services to schedule workflow applications. We propose an algorithm to select the best resources available to execute each task of the workflow on the already instantiated services or on services dynamically instantiated when necessary. The algorithm relies on the existence of a grid infrastructure which could provide dynamic service instantiation. Simulation results show that the scheduling algorithm associated with the dynamic service instantiation can bring more efficient workflow execution on the grid.

An environment for evaluation and testing of service workflow schedulers in clouds

Workflows built through service composition bring new challenges, making the scheduling task even more complex. Besides that, scheduling researchers need a wide range of workflows and their respective services to validate new achievements. In this paper we present a service oriented testbed where experiments can be conducted to develop and evaluate algorithms, heuristics, and scheduling policies for workflows. Our testbed offers an emulator service which allows the workflow characterization through the description of its services. With this, researchers can build workflows which have similar behavior to the real workflow applications, emulating them without the need of implementing all applications and services involved in a real application execution. To demonstrate the utilization of both the testbed and emulator, we conducted workflow executions emulating service workflow applications, such as Montage and LIGO.

Service workflow monitoring in private clouds: The user point of view

Cloud computing environments offer, in general, tools and mechanisms for the user to monitor computational, storage, and networking resources. However, only a few middlewares for cloud computing provide tools that help the user in the application monitoring. When we consider workflows and their dependencies, monitoring from the user point of view is not found in the literature of cloud middlewares. In this paper, we present an infrastructure for execution of service workflows in private clouds with tools that allow the user to keep track of his/her workflows, measuring each activity performed in the cloud resources. Through this monitoring, our workflow manager is able to react during the workflow execution, requesting new or better resources in order to the execution to comply with the application quality of service requirements. Besides that, the metrics that our monitoring system offers allow users to keep a detailed track of the organization internal costs involved in the processing, and then use this information to evaluate other alternatives, such as leasing resources from public cloud providers.

An Architecture for Orchestrating Hadoop Applications in Hybrid Cloud

MapReduce is a programming model for processing and generating large data sets, and Hadoop, a MapReduce implementation, is a good tool to handle Big Data. Cloud computing with its ubiquitous characteristic, on demand and dynamic resource provisioning at low cost has potential to be the environment to treat big data. However, using Hadoop on the cloud spends time and requires technical knowledge from users. The hybrid cloud leverages these requirements, because its necessary to evaluate the resources in private cloud and, if necessary, obtain and prepare on-demand resources in the public cloud. Moreover, the simultaneous management of private and public domains requires an appropriate model that combines performance with minimal cost. In this paper we propose an architecture to make the orchestration of Hadoop applications in hybrid clouds. The core of the model consists of a web portal for submissions, an orchestration engine and an execution services factory. Through these three components its possible to automate the preparation of a cross-domain cluster, performing the provisioning of files involved, managing the execution of the application, and making the results available to the user.

Fog orchestration for the Internet of Everything: state-of-the-art and research challenges

Recent developments in telecommunications have allowed drawing new paradigms, including the Internet of Everything, to provide services by the interconnection of different physical devices enabling the exchange of data to enrich and automate people’s daily activities; and Fog computing, which is an extension of the well-known Cloud computing, bringing tasks to the edge of the network exploiting characteristics such as lower latency, mobility support, and location awareness. Combining these paradigms opens a new set of possibilities for innovative services and applications; however, it also brings a new complex scenario that must be efficiently managed to properly fulfill the needs of the users. In this scenario, the Fog Orchestrator component is the key to coordinate the services in the middle of Cloud computing and Internet of Everything. In this paper, key challenges in the development of the Fog Orchestrator to support the Internet of Everything are identified, including how they affect the tasks that a Fog service Orchestrator should perform. Furthermore, different service Orchestrator architectures for the Fog are explored and analyzed in order to identify how the previously listed challenges are being tackled. Finally, a discussion about the open challenges, technological directions, and future of the research on this subject is presented.

An Emulator for Evaluating Resource Allocation and Performance in Clouds

Efficient utilization of physical hosts in a private cloud depends on the configuration of the virtual machines (VMs), which must precisely comply with application requirements to avoid resource wastage. To achieve this, it is necessary to build knowledge about the performance of each physical host according to the number and type of VMs to be hosted to fulfil application demands. However, such a mapping would need the deployment and testing of each application that could be hosted in the cloud, which can be impracticable. To help this mapping of resources versus applications, we present the Cloud Workflow Emulator (CWE), a tool that allows to emulate the behavior of workflows and applications in virtualized environments. Through CWE, it is possible to evaluate machines performance when hosting VMs, enabling their classification according to the desired performance metrics in order to ensure the machine will comply with the necessary application requirements. Moreover, the evaluation using the emulator can help in the development of more efficient VM allocation strategies and algorithms. To demonstrate the CWE usefulness, we show an evaluation and classification of our test bed machines through the emulation of an image processing application and discuss how this classification improves the scheduling in the test beds hosts.

Performance evaluation of virtual machines in a service-oriented Grid testbed

In this paper we analyze the performance of execution of service workflows in virtual machines (VMs) used as resources in our service-oriented grid testbed (SGT).We chosen an application widely known by computer network researchers and developers to evaluate the testbed: the simulation of networks using the Network Simulator. We constructed workflows which execute several WiMAX network scenarios in parallel, and we executed them to evaluate the performance of virtual and physical machines in the grid. The experiments show that the adequate support to the parallelism can minimize the consequences of the overhead introduced by the VMs, allowing the simulation of several scenarios with execution time as low as one simulation alone would take. We conclude that it is important to have a VM-aware resource manager to completely explore the hardware and its VMs.