Dahai Cao

The Design of Cloud Workflow Systems

Cloud computing is the latest market-oriented computing paradigm which brings software design and development into a new era characterized by XaaS, i.e. everything as a service. Cloud workflows, as typical software applications in the cloud, are composed of a set of partially ordered cloud software services to achieve specific goals. However, due to the low QoS (quality of service) nature of the cloud environment, the design of workflow systems in the cloud becomes a challenging issue for the delivery of high quality cloud workflow applications. To address such an issue, this book presents a systematic investigation to the three critical aspects for the design of a cloud workflow system, viz. system architecture, system functionality and quality of service. Specifically, the system architecture for a cloud workflow system is designed based on the general four-layer cloud architecture, viz. application layer, platform layer, unified resources layer and fabric layer. The system functionality for a cloud workflow system is designed based on the general workflow reference model but with significant extensions to accommodate software services in the cloud. The support of QoS is critical for the quality of cloud workflow applications. This book presents a generic framework to facilitate a unified design and development process for software components that deliver lifecycle support for different QoS requirements. While the general QoS requirements for cloud workflow applications can have many dimensions, this book mainly focuses on three of the most important ones, viz. performance, reliability and security. In this book, the architecture, functionality and QoS management of our SwinDeW-C prototype cloud workflow system are demonstrated in detail as a case study to evaluate our generic design for cloud workflow systems. To conclude, this book offers a general overview of cloud workflow systems and provides comprehensive introductions to the design of the system architecture, system functionality and QoS management.

A Generic QoS Framework for Cloud Workflow Systems

Due to the dynamic nature of cloud computing, how to achieve satisfactory QoS (Quality of Service) in cloud workflow systems becomes a challenge. Meanwhile, since QoS requirements have many dimensions, a unified system design for different QoS management components is required to reduce the system complexity and software development cost. Therefore, this paper proposes a generic QoS framework for cloud workflow systems. Covering the major stages of a workflow lifecycle, the framework consists of four components, viz. QoS requirement specification, QoS-aware service selection, QoS consistency monitoring and QoS violation handling. While there are many QoS dimensions, this paper illustrates a concrete performance framework as a case study and briefly touches others. We also demonstrate the system implementation and evaluate the effectiveness of the performance framework in our cloud workflow system.

Selecting checkpoints along the time line: a novel temporal checkpoint selection strategy for monitoring a batch of parallel business processes

Nowadays, most business processes are running in a parallel, distributed and time-constrained manner. How to guarantee their on-time completion is a challenging issue. In the past few years, temporal checkpoint selection which selects a subset of workflow activities for verification of temporal consistency has been proved to be very successful in monitoring single, complex and large size scientific workflows. An intuitive approach is to apply those strategies to individual business processes. However, in such a case, the total number of checkpoints will be enormous, namely the cost for system monitoring and exception handling could be excessive. To address such an issue, we propose a brand new idea which selects time points along the workflow execution time line as checkpoints to monitor a batch of parallel business processes simultaneously instead of individually. Based on such an idea, a set of new definitions as well as a time-point based checkpoint selection strategy are presented in this paper. Our preliminary results demonstrate that it can achieve an order of magnitude reduction in the number of checkpoints while maintaining satisfactory on-time completion rates compared with the state-of-the-art activity-point based checkpoint selection strategy.

Novel Client-Cloud Architecture for Scalable Instance-Intensive Workflow Systems

Though workflow technology is relatively mature and has been one of the most popular components of process aware systems over the last two decades, few workflow architectures can efficiently support a large number of concurrent workflow instances, i.e. instance-intensive workflows. The basic requirements include high throughput, elastic scalability, and cost-effectiveness. This paper proposes a novel client-cloud architecture which takes advantages of cloud computing to support instance-intensive workflows, presents an application level real-time resource utilization estimation model, and identifies two primary principles to ensure the sustainable scalability, namely: (1) the time for a load balancer checking must be less than the decaying time of a server instance when it is overloaded, (2) the sampling time for an alarming service plus the launching time of new server instance must be less than the decaying time of a server instance when it is overloaded. Based on the above, we design and implement the SwinFlow-Cloud prototype. Finally, we deploy and evaluate the prototype on Amazon Web Services cloud. The results show that the prototype is able to satisfy all the basic requirements for instance-intensive workflows.

Workflow Systems in the Cloud

In this chapter, we will present an overview about the background of cloud computing and workflow systems. Specifically, Sect. 1.1 introduces the novel cloud computing paradigm. Section 1.2 reviews the workflow systems, especially in the distributed computing environments. Section 1.3 introduces the cloud workflow systems. In Sect. 1.4, we demonstrate two motivating examples, one for large-scale data and computation intensive e-science application in Astrophysics and one for instance intensive e-business application in the stock market. Finally, Sect. 1.5 points out the key issue in the design of cloud workflow systems.

Cloud Workflow System Quality of Service

Along with system functionality, the management of quality of service (QoS) in cloud workflow system is attracting increasing and even more efforts [3, 31, 45, 47, 54, 73]. This is mainly because of the following two reasons. First, the cloud computing environment is very dynamic and uncertain. Therefore, it is difficult to achieve targeted service quality if without effective QoS management strategies; Second, cloud computing adopts the market-oriented model and strict service contracts. Therefore, high service quality is necessary for improving customer satisfaction and avoiding penalty for the breach of service contracts. Therefore, QoS management plays a significant role in cloud workflow systems, and hence included as significant part of this book. In Sect. 4.1, we will first present an overview about the QoS of Web and cloud services. In Sect. 4.2, we introduce the QoS of cloud workflows. In Sect. 4.3, a generic QoS framework is presented as a high level guideline for the design of software components to deliver lifecycle QoS support in cloud workflow systems. Afterwards, as concrete examples, specific strategies for performance management (on workflow response time), cost management (on data storage), reliability management (on data replication), and security management, will be discussed and demonstrated.

Case Study: SwinDeW-C Cloud Workflow System

The previous chapters have given a general overview of cloud workflow system architecture, functionality and quality of service. In this chapter, we will demonstrate our SwinDeW-C cloud workflow system as a concrete case study to illustrate the design and development of a cloud workflow system for running large scale workflow applications.

Cloud Workflow System Functionality

In this chapter, we will present the cloud workflow system functionality. In Sect. 3.1, we will first introduce the classical workflow reference model which defines the basic functionalities for a workflow system. In Sect. 3.2, we will then illustrate those system functionalities which are typical for the running of workflows in the cloud computing environment.