Sucha Smanchat

Taxonomies of workflow scheduling problem and techniques in the cloud

Scientific workflows, like other applications, benefit from the cloud computing, which offers access to virtually unlimited resources provisioned elastically on demand. In order to efficiently execute a workflow in the cloud, scheduling is required to address many new aspects introduced by cloud resource provisioning. In the last few years, many techniques have been proposed to tackle different cloud environments enabled by the flexible nature of the cloud, leading to the techniques of different designs. In this paper, taxonomies of cloud workflow scheduling problem and techniques are proposed based on analytical review. We identify and explain the aspects and classifications unique to workflow scheduling in the cloud environment in three categories, namely, scheduling process, task and resource. Lastly, review of several scheduling techniques are included and classified onto the proposed taxonomies. We hope that our taxonomies serve as a stepping stone for those entering this research area and for further development of scheduling technique. We propose taxonomies of cloud workflow scheduling problem and techniques.Aspects and classifications unique to cloud workflow scheduling are identified.Several techniques are reviewed and classified based on the proposed taxonomies.Some issues of future concern in cloud workflow scheduling are discussed.

A survey on context-aware workflow adaptations

Recently, workflow has been employed in pervasive computing systems to orchestrate and provide services to the users. Since pervasive computing systems emphasise context-awareness and adaptability, these two features must be included in the workflow mechanism in order to utilise workflow in pervasive environment. In this paper we present various existing approaches to adaptation in context-aware workflow. These approaches are compared based on the common adaptation characteristics identified from our study. Lastly, we discuss and point out the future challenges of predictive adaptation for context-aware workflows.

Scheduling Multiple Parameter Sweep Workflow Instances on the Grid

Due to its ability to provide high-performance computing environment, the grid has become an important infrastructure to support eScience. To utilise the grid for parameter sweep experiments, workflow technology combined with tools such as Nimrod/K are used to orchestrate and automate scientific services provided on the grid. As parameter sweeping over a workflow needs to be executed numerous times, it is more efficient to execute multiple instances of the workflow in parallel. However, this parallel execution can be delayed as every workflow instance requires the same set of resources leading to resource competition problem. Although many algorithms exist for scheduling grid workflows, there is little effort in considering multiple workflow instances and resource competition in the scheduling process. In this paper, we proposed a scheduling algorithm for parameter sweep workflow based on resource competition. The proposed algorithm aims to support multiple workflow instances and avoid allocating resources with high resource competition to minimise delay due to the blocking of tasks. The result is evaluated using simulation to compare with an existing scheduling algorithm.

Scheduling parameter sweep workflow in the Grid based on resource competition

Workflow technology has been adopted in scientific domains to orchestrate and automate scientific processes in order to facilitate experimentation. Such scientific workflows often involve large data sets and intensive computation that necessitate the use of the Grid. To execute a scientific workflow in the Grid, tasks within the workflow are assigned to Grid resources. Thus, to ensure efficient execution of the workflow, Grid workflow scheduling is required to manage the allocation of Grid resources. Although many Grid workflow scheduling techniques exist, they are mainly designed for the execution of a single workflow. This is not the case with parameter sweep workflows, which are used for parametric study and optimisation. A parameter sweep workflow is executed numerous times with different input parameters in order to determine the effect of each parameter combination on the experiment. While executing multiple instances of a parameter sweep workflow in parallel can reduce the time required for the overall execution, this parallel execution introduces new challenges to Grid workflow scheduling. Not only is a scheduling algorithm that is able to manage multiple workflow instances required, but this algorithm also needs the ability to schedule tasks across multiple workflow instances judiciously, as tasks may require the same set of Grid resources. Without appropriate resource allocation, resource competition problem could arise. We propose a new Grid workflow scheduling technique for parameter sweep workflow called the Besom scheduling algorithm. The scheduling decision of our algorithm is based on the resource dependencies of tasks in the workflow, as well as conventional Grid resource-performance metrics. In addition, the proposed technique is extended to handle loop structures in scientific workflows without using existing loop-unrolling techniques. The Besom algorithm is evaluated using simulations with a variety of scenarios. A comparison between the simulation results of the Besom algorithm and of the three existing Grid workflow scheduling algorithms shows that the Besom algorithm is able to perform better than the existing algorithms for workflows that have complex structures and that involve overlapping resource dependencies of tasks.

A Scheduler based on Resource Competition for Parameter Sweep Workflow

Grid workflow scheduling has been a prevalent field of research in order to allocate scientific workflow tasks to grid resources. To actuate these grid workflow scheduling algorithms, schedulers need to be developed for grid workflow management systems. A scheduler is a component that gathers information, such as estimated execution times and lists of available grid resources, as inputs for scheduling algorithms. Once a grid schedule is generated, the scheduler uses it to allocate grid resources to the tasks in the workflow. This is even more complicated for parameter sweep workflow scheduling. As parameter sweep workflows are repeatedly executed a number of times with different inputs, to schedule them in parallel, the scheduler must be able to handle multiple workflow instances and multiple scheduling iterations. In this paper, we present a scheduling algorithm for parameter sweep workflows and suggest an implementation of a scheduler for parameter sweep workflows based on the algorithms. We highlight the implementation issues encountered in our experience of scheduler development.

MapReduce join strategies for key-value storage

This paper analyses MapReduce join strategies used for big data analysis and mining known as map-side and reduce-side joins. The most used joins will be analysed in this paper, which are theta-join algorithms including all pair partition join, repartition join, broadcasting join, semi join, per-split semi join. This paper can be considered as a guideline for MapReduce application developers for the selection of join strategies. The analysis of several join strategies for big data analysis and mining is accompanied by comprehensive examples.

Enabling Parallel Streaming of Multiple Video Sections by Segment Scheduling

A type of application that occupies most of the Internet is online multimedia, which is usually referred to online video or video streaming with providers such as Youtube, Netflix and Hulu. With the advancement in the Internet technology, people can watch an entire 2-hours long movie. However, people sometimes want to skip to different sections of a video to see whether the whole video is of their interest. Doing so usually requires the video to restart buffering and causes delay. To alleviate this issue, this research proposes a parallel streaming method based on the modification of the Dynamic Adaptive Streaming over HTTP (DASH) technique. By splitting video into smaller segments and modifying the segment scheduling, we can stream a video at many different positions in parallel so that users can jump to the different sections with minimal delay. The performance evaluation of the proposed technique shows that such streaming is possible through video segment scheduling with higher throughput in exchange for higher bandwidth requirement and utilization.

Toward grid workflow scheduling based on resource competition

Grid has become an infrastructure to support scientific research due to its ability to provide high-performance computing environment. In order to automate scientific process, workflow has been used to orchestrate tasks to be executed in grid environment. Hence, the management of the workflow execution, especially the scheduling of workflow tasks in grid under high resource competition situation, becomes an important issue to improve the quality of service. In this paper, we propose a workflow scheduling algorithm based on resource competition among the tasks in the workflow. Our preliminary result is compared with the existing Min-Min algorithm to demonstrate the advantage of our algorithm in situations where there is a high degree of resource competition in the grid workflow.

A Technique for Streaming Multiple Video Parts in Parallel Based on Dash.js

Watching video on the Internet has become an alternative to offline video playback on video player software. Videos on the Internet are streamed to video player so that they can be played without having to wait for the whole files to be downloaded. However, when playing a video, some users may skip ahead to different positions or scenes in the video to see whether the video is really of their interest. Most video players only stream videos from the beginning, thus jumping to a part of a video that has not been streamed causes a delay while the video starts to buffer. In this paper, we propose a technique to stream multiple parts of a video simultaneously by modifying the dash.js framework so that users can navigate to different points in a video with minimal delay. The prototype that realizes our concept proves the feasibility of our approach. With further improvement, our technique may contribute to the scene selection function for video streaming, similar to that of video player software.

Identifying Information Requirement for Scheduling Kepler Workflow in the Cloud

Abstract Kepler scientific workflow system has been used to support scientists to automatically perform experiments of various domains in distributed computing systems. An execution of a workflow in Kepler is controlled by a director assigned in the workflow. However, users still need to specify compute resources on which the tasks in the workflow are executed. To further ease the technical effort required by scientists, a workflow scheduler that is able to assign workflow tasks to resources for execution is necessary. To this end, we identify from a review of several cloud workflow scheduling techniques the information that should be made available in order for a scheduler to schedule Kepler workflow in the cloud computing context. To justify the usefulness, we discuss each type of information regarding workflow tasks, cloud resources, and cloud providers based on their benefit on workflow scheduling.