Ji Wang

Real-Time Tasks Oriented Energy-Aware Scheduling in Virtualized Clouds

Energy conservation is a major concern in cloud computing systems because it can bring several important benefits such as reducing operating costs, increasing system reliability, and prompting environmental protection. Meanwhile, power-aware scheduling approach is a promising way to achieve that goal. At the same time, many real-time applications, e.g., signal processing, scientific computing have been deployed in clouds. Unfortunately, existing energy-aware scheduling algorithms developed for clouds are not real-time task oriented, thus lacking the ability of guaranteeing system schedulability. To address this issue, we first propose in this paper a novel rolling-horizon scheduling architecture for real-time task scheduling in virtualized clouds. Then a task-oriented energy consumption model is given and analyzed. Based on our scheduling architecture, we develop a novel energy-aware scheduling algorithm named EARH for real-time, aperiodic, independent tasks. The EARH employs a rolling-horizon optimization policy and can also be extended to integrate other energy-aware scheduling algorithms. Furthermore, we propose two strategies in terms of resource scaling up and scaling down to make a good trade-off between tasks schedulability and energy conservation. Extensive simulation experiments injecting random synthetic tasks as well as tasks following the last version of the Google cloud tracelogs are conducted to validate the superiority of our EARH by comparing it with some baselines. The experimental results show that EARH significantly improves the scheduling quality of others and it is suitable for real-time task scheduling in virtualized clouds.

Internet-based Virtual Computing Environment

The two dominating characteristics of new and emerging Internet applications are ultra-large scales and utility. Centralized data centers alone are often inadequate for running such applications. In this paper we introduce the concept of an Internet-based Virtual Computing Environment (iVCE), which aims to provide Cloud services by a dynamic combination of data centers and other multi-scale computing resources on the Internet. We present a model that addresses two critical challenges in iVCE: multi-scale resource aggregation and elastic binding. We then describe the design and implementation of our iVCE software platform that embodies the model. Comprehensive experiments show that iVCE provides a novel, promising way to deal with scalability and utility, thereby enabling economical and elastic Cloud Computing. Highlights? Centralized data centers alone are often inadequate for new emerging Internet applications. ? Dynamic combination of data centers and other multi-scale resources on the Internet. ? Model to address two challenges: multi-scale resource aggregation and elastic binding. ? iVCE software platform has been designed and implemented to embody the model. ? iVCE is a novel promising way for economical and elastic Cloud Computing.

Fault-Tolerant Scheduling for Real-Time Scientific Workflows with Elastic Resource Provisioning in Virtualized Clouds

Clouds are becoming an important platform for scientific workflow applications. However, with many nodes being deployed in clouds, managing reliability of resources becomes a critical issue, especially for the real-time scientific workflow execution where deadlines should be satisfied. Therefore, fault tolerance in clouds is extremely essential. The PB (primary backup) based scheduling is a popular technique for fault tolerance and has effectively been used in the cluster and grid computing. However, applying this technique for real-time workflows in a virtualized cloud is much more complicated and has rarely been studied. In this paper, we address this problem. We first establish a real-time workflow fault-tolerant model that extends the traditional PB model by incorporating the cloud characteristics. Based on this model, we develop approaches for task allocation and message transmission to ensure faults can be tolerated during the workflow execution. Finally, we propose a dynamic fault-tolerant scheduling algorithm, FASTER, for realtime workflows in the virtualized cloud. FASTER has three key features: 1) it employs a backward shifting method to make full use of the idle resources and incorporates task overlapping and VM migration for high resource utilization, 2) it applies the vertical/horizontal scaling-up technique to quickly provision resources for a burst of workflows, and 3) it uses the vertical scaling-down scheme to avoid unnecessary and ineffective resource changes due to fluctuated workflow requests. We evaluate our FASTER algorithm with synthetic workflows and workflows collected from the real scientific and business applications and compare it with six baseline algorithms. The experimental results demonstrate that FASTER can effectively improve the resource utilization and schedulability even in the presence of node failures in virtualized clouds.

SVM-Based tumor classification with gene expression data

Gene expression data that are gathered from tissue samples are expected to significantly help the development of efficient tumor diagnosis and classification platforms. Since DNA microarray experiments provide us with huge amount of gene expression data and only a few of genes are related to tumor, gene selection algorithms should be emphatically explored to extract those informative genes related tumor from gene expression data. So we propose a novel feature selection approach to further improve the SVM-based classification performance of gene expression data, which projects high dimensional data onto lower dimensional feature space. We examine a set of gene expression data that include sets of tumor and normal clinical samples by means of SVMs classifier. Experiments show that SVM has a superior performance in classification of gene expression data as long as the selected features can represent the principal components of all gene expression samples.

Fault-Tolerant Scheduling for Real-Time Tasks on Multiple Earth-Observation Satellites

Fault-tolerance plays an important role in improving the reliability of multiple earth-observing satellites, especially in emergent scenarios such as obtaining photographs on battlefields or earthquake areas. Fault tolerance can be implemented through scheduling approaches. Unfortunately, little attention has been paid to fault-tolerant scheduling on satellites. To address this issue, we propose a novel dynamic fault-tolerant scheduling model for real-time tasks running on multiple observation satellites. In this model, the primary-backup policy is employed to tolerate one satellites permanent failure at one time instant. In the light of the fault-tolerant model, we develop a novel fault-tolerant satellite scheduling algorithm named FTSS. To improve the resource utilization, we apply the overlapping technology that includes primary-backup copy overlapping (i.e., PB overlapping) and backup-backup copy overlapping (i.e., BB overlapping). According to the satellites characterized with time windows for observations, we extensively analyze the overlapping mechanism on satellites. We integrate the overlapping mechanism with FTSS, which employs the task merging strategies including primary-backup copy merging (i.e., PB merging), backup-backup copy merging (i.e., BB merging) and primary-primary copy merging (i.e., PP merging). These merging strategies are used to decrease the number of tasks required to be executed, thereby enhancing system schedulability. To demonstrate the superiority of our FTSS, we conduct extensive experiments using the real-world satellite parameters supplied from the satellite tool kit or STK; we compare FTSS with the three baseline algorithms, namely, NMFTSS, NOFTSS, and NMNOFTSS. The experimental results indicate that FTSS efficiently improves the scheduling quality of others and is suitable for fault-tolerant satellite scheduling.

The Classification of Tumor Using Gene Expression Profile Based on Support Vector Machines and Factor Analysis

Gene expression data that is being used to gather information from tissue samples is expected to significantly improve the development of efficient tumor diagnosis and to provide understanding and insight into tumor related cellular processes. In this paper, we propose a novel feature selection approach which integrates the feature score criterion with factor analysis to further improve the SVM-based classification performance of gene expression data. We examine two sets of published gene expression data to validate the novel feature selection method by means of SVM classifier with different parameters. Experiments show that the proposed hybrid method can select a small quantity of principal factors to represent a large number of genes and SVM has a superior classification performance with the common factors which are extracted from gene expression data. Moreover, experiment results demonstrate successful cross-validation accuracy of 92% for the colon dataset and 100% for the leukemia dataset

SP-Partitioner: A novel partition method to handle intermediate data skew in spark streaming

Abstract Spark Streaming, a popular tool for processing live data streams, offers a good divide-and-conquer solution, where data stream is split into batches that are then processed in parallel by mappers, and the intermediate data from the mappers are finally reduced by reducers. However, one of the key issues with such an approach for live data processing is partitioning skew in which data distributed over the processing units are not balanced due to uncertainty of the coming data streams. This imbalance is rippled through the mappers and become prominent to the reducers, making reduce a performance bottleneck to the overall system. To address this issue, we present a Partitioner, SP-Partitioner, that sits between the map and reduce stages to re-balance the workload of the reducers. With our design, we treat the arrived batches of data as candidate samples and choose samples based on systematic sampling to predict the characteristics of intermediate data. According to the prediction, our method generates a reference table to guide the allocation of next batches of data evenly. We implement SP-Partitioner in Spark 1.6.1 and evaluate its performance with widely used applications. Experimental results conducted on a real VMs cluster show that our algorithms can not only achieve higher balancing performance on data with varying degree of data skew, but also decrease the average processing time of one batch of these data.

Analysis and Design of Fault-Tolerant Scheduling for Real-Time Tasks on Earth-Observation Satellites

Fault-tolerant scheduling is an efficient approach to improving the reliability of multiple earth-observing satellites especially in some emergent scenarios such as obtaining photographs on battlefields or earthquake areas. Unfortunately, little work has been done to deal with the fault-tolerant scheduling on satellites. To address this issue, this paper presents a novel dynamic fault-tolerant scheduling model using primary-backup policy to tolerate one satellites permanent failure at one time instant. On this basis, we propose a novel fault-tolerant satellite scheduling algorithm named FTSS, in which an overlapping technology is adopted to improve the resource utilization. Besides, the FTSS employs the task merging strategies to further enhance the schedulability. To demonstrate the superiority of our FTSS, we conduct extensive experiments by simulations using real-world satellite parameters from STK to compare FTSS with other baseline algorithms. The experimental results indicate that FTSS efficiently improves the scheduling quality of others and is suitable for fault-tolerant satellite scheduling.

Towards collaborative storage scheduling using alternating direction method of multipliers for mobile edge cloud

We propose a collaborative storage architecture of mobile edge cloud.We propose a collaborative storage scheduling algorithm named ACMES.ACMES minimizes power usage and withdrawal risk with assured reliability.ACMES works in a distributed and parallel way.We conduct extensive experiments to validate the superiority of ACMES. Performance of cloud computing would be much improved by extending storage capabilities to devices at the edge of network. Unfortunately, the commonly employed algorithms fail to be adaptive to the new storage pattern on mobile edge cloud. To address this issue, we propose a collaborative storage architecture model and an alternating-direction-method-of-multipliers-based collaborative storage scheduling algorithm called ACMES (Algorithm of Collaborative Mobile Edge Storage), in which heterogeneous information of nodes in mobile edge cloud is considered and integrated to make decisions. Besides, feasible solutions for storage will be acquired after iterations of computing. By formulating the collaborative storage scheduling problem in the mobile edge cloud and designing the collaborative decision-making process with the theory of Alternating Direction Method of Multipliers (ADMM), the proposed ACMES is able to minimize power usage and the risk of node withdrawal without reducing the reliability of node storage, and meanwhile make storage scheduling decisions at the edge environment directly and work in a distributed and parallel way. The convergence analysis shows that ACMES has the ability to solve complicated mobile edge cloud storage problems in reality. Extensive experiments validate its effectiveness as well as its superiority to three existing strategies (ADM, RDM and ERASURE) in total cost, reliability, power usage and withdrawal risks.

MECCAS: Collaborative Storage Algorithm Based on Alternating Direction Method of Multipliers on Mobile Edge Cloud

The commonly existing employed centralized algorithms fail to be adaptive to the new storage pattern on mobile edge cloud. To address this issue, we propose an alternating-direction-method-of-multipliers-based collaborative storage algorithm called MECCAS (Mobile Edge Cloud Collaborative Storage). The proposed MECCAS is able to minimize the delay of task execution and total costs for the overall operation, and meanwhile maximize the utilization of local information of nodes and system reliability. Nodes on mobile edge cloud storage are capable of adaptively allocating resources for storage to increase power usage effectiveness and reduce the risk of nodes withdrawal. Extensive experiments demonstrate the superiority of our MECCAS algorithm compared with other three baselines, i.e., ADM, RDM and ERASURE. The optimization utility of our algorithm is higher than other three algorithms by 41.72%, 44.52% and 22.94% on average, respectively