Zhuo Tang

An Energy-Efficient Task Scheduling Algorithm in DVFS-enabled Cloud Environment

The growth of energy consumption has been explosive in current data centers, super computers, and public cloud systems. This explosion has led to greater advocacy of green computing, and many efforts and works focus on the task scheduling in order to reduce energy dissipation. In order to obtain more energy reduction as well as maintain the quality of service by meeting the deadlines, this paper proposes a DVFS-enabled Energy-efficient Workflow Task Scheduling algorithm: DEWTS. Through merging the relatively inefficient processors by reclaiming the slack time, DEWTS can leverage the useful slack time recurrently after severs are merged. DEWTS firstly calculates the initial scheduling order of all tasks, and obtains the whole makespan and deadline based on Heterogeneous-Earliest-Finish-Time (HEFT) algorithm. Through resorting the processors with their running task number and energy utilization, the underutilized processors can be merged by closing the last node and redistributing the assigned tasks on it. Finally, in the task slacking phase, the tasks can be distributed in the idle slots under a lower voltage and frequency using DVFS technique, without violating the dependency constraints and increasing the slacked makespan. Based on the amount of randomly generated DAGs workflows, the experimental results show that DEWTS can reduce the total power consumption by up to 46.5 % for various parallel applications as well as balance the scheduling performance.

A MapReduce task scheduling algorithm for deadline constraints

The current works about MapReduce task scheduling with deadline constraints neither take the differences of Map and Reduce task, nor the cluster’s heterogeneity into account. This paper proposes an extensional MapReduce Task Scheduling algorithm for Deadline constraints in Hadoop platform: MTSD. It allows user specify a job’s deadline and tries to make the job be finished before the deadline. Through measuring the node’s computing capacity, a node classification algorithm is proposed in MTSD. This algorithm classifies the nodes into several levels in heterogeneous clusters. Under this algorithm, we firstly illuminate a novel data distribution model which distributes data according to the node’s capacity level respectively. The experiments show that the node classification algorithm can improved data locality observably to compare with default scheduler and it also can improve other scheduler’s locality. Secondly, we calculate the task’s average completion time which is based on the node level. It improves the precision of task’s remaining time evaluation. Finally, MTSD provides a mechanism to decide which job’s task should be scheduled by calculating the Map and Reduce task slot requirements.

A self-adaptive scheduling algorithm for reduce start time

MapReduce is by far one of the most successful realizations of large-scale data-intensive cloud computing platforms. When to start the reduce tasks is one of the key problems to advance the MapReduce performance. The existing implementations may result in a block of reduce tasks. When the output of map tasks become large, the performance of a MapReduce scheduling algorithm will be influenced seriously. Through analysis for the current MapReduce scheduling mechanism, this paper illustrates the reasons of system slot resources waste, which results in the reduce tasks waiting around, and proposes an optimal reduce scheduling policy called SARS (Self Adaptive Reduce Scheduling) for reduce tasks start times in the Hadoop platform. It can decide the start time point of each reduce task dynamically according to each job context, including the task completion time and the size of map output. Through estimating job completion time, reduce completion time, and system average response time, the experimental results illustrate that, when comparing with other algorithms, the reduce completion time is decreased sharply. It is also proved that the average response time is decreased by 11% to 29%, when the SARS algorithm is applied to the traditional job scheduling algorithms FIFO, FairScheduler, and CapacityScheduler. This paper illustrates the reasons of the system slots waster for reduces tasks waiting around.The model can determine the start time of reduce tasks, dynamically according to job context.As an optimal scheduling algorithm, SARS can decrease the reduce completion time for jobs.

ESTIMATING PARAMETERS OF MUSKINGUM MODEL USING AN ADAPTIVE HYBRID PSO ALGORITHM

In order to accelerate the convergence and improve the calculation accuracy for parameter optimization of the Muskingum model, we propose a novel, adaptive hybrid particle swarm optimization (AHPSO) algorithm. With the decreasing of inertial weight factor proposed, this method can gradually converge to a global optimal with elite individuals obtained by hybrid PSO. In the paper, we analyzed the feasibility and the advantages of the AHPSO algorithm. Then, we verified its efficiency and superiority by application of the Muskingum model. We intensively evaluated the error fitting degree based on the comparison with four known formulas: the test method (TM), the least residual square method (LRSM), the nonlinear programming method (NPM), and the Broyden–Fletcher–Goldfarb–Shanno (BFGS) method. The results show that the AHPSO has a higher precision. In addition, we compared the AHPSO algorithm with the binary-encoded genetic algorithm (BGA), the Gray genetic algorithm (GGA), the Gray-encoded accelerating genetic algorithm (GAGA) and the particle swarm optimization (PSO), and results show that AHPSO has faster convergent speed. Moreover, AHPSO has a competitive advantage compared with the above eight methods in terms of robustness. With the efficiency of this approach it can be extended to estimate parameters of other dynamic models.

CRFs based parallel biomedical named entity recognition algorithm employing MapReduce framework

As the rapid growth of the biomedical literature, the model training time in biomedical named entity recognition increases sharply when dealing with large-scale training samples. How to increase the efficiency of named entity recognition in biomedical big data becomes one of the key problems in biomedical text mining. For the purposes of improving the recognition performance and reducing the training time, this paper proposes an optimization method for two-phase recognition using conditional random fields. In the first stage, each named entity boundary is detected to distinguish all real entities. In the second stage, we label the semantic class of the entity detected. To expedite the training speed, in these two phases, we implement the model training process on a parallel optimization program framework based on MapReduce. Through dividing the training set into several parts, the iterations in the training algorithm are designed as map tasks which can be executed simultaneously in a cluster, where each map function is designed to complete the calculation of a gradient vector component for each part in the training set. Our experiments show that the proposed method in this paper can achieve high performance with short training time, which has important implications for the current biological big data processing.

MTSD: A Task Scheduling Algorithm for MapReduce Base on Deadline Constraints

The previous works about MapReduce task scheduling with deadline constraints neither take the diffenences of Map and Reduce task, nor the clusters heterogeneity into account. This paper proposes an extensional MapReduce Task Scheduling algorithm for Deadline constraints in Hadoop platform: MTSD. It allows user specify a jobs deadline and tries to make the job be finished before the deadline. Through measuring the nodes computing capacity, a node classification algorithm is proposed in MTSD. This algorithm classifies the nodes into several levels in heterogeneous clusters. Under this algorithm, we firstly illuminate a novel data distribution model which distributes data according to the nodes capacity level respectively. The experiments show that the data locality is improved about 57%. Secondly, we calculate the tasks average completion time which is based on the node level. It improves the precision of tasks remaining time evaluation. Finally, MTSD provides a mechanism to decide which jobs task should be scheduled by calculating the Map and Reduce task slot requirements.

A new RBAC based access control model for cloud computing

Access Control is an important component of Cloud Computing; specially, User access control management; however, Access Control in Cloud environment is different from traditional access environment and using general access control model cant cover all entities within Cloud Computing, noting that Cloud environment includes different entities such as data owner, end user, and service provider. In this paper, we propose a new access control based on Role-based access control (RBAC) model. This model includes two kind of roles, user role (UR) and owner role (OR); such that, Users get credential from owners to communicate with service provider and to get access permissions of resources. We also discuss the aspects of user access control management, such as authentication, privilege management, and deprovisioning. Moreover, we use administrative scope to update hierarchy when there is a role added or revoked to simplify the user access control management. By applying the model in Cloud environment the results shows that it can reduce the security problems to two classes in the RT [←,∩] role-based trust-management language with a test-paper system.

An optimized MapReduce workflow scheduling algorithm for heterogeneous computing

The MapReduce framework is considered to be an effective resolution for huge and parallel data processing. This paper treats a massive data processing workflow as a DAG graph consisting of MapReduce jobs. In a heterogeneous computing environment, the computation speed can be different even on the same slot depending on various jobs. For this problem, this paper proposes an optimized MapReduce workflow scheduling algorithm. This algorithm comprises a job prioritizing phase and a task assignment phase. First, the jobs can be classified as I/O-intensive and computing-intensive, and the priorities of all jobs are computed according to their corresponding types. Then, the suitable slots are allocated for each block, and the MapReduce tasks in the workflow are scheduled with respect to data locality. The experimental results show that the optimized MapReduce workflow scheduling algorithm can improve the performance of task scheduling and the rationality of resources allocation in heterogeneous computing.

Selection and replacement algorithms for memory performance improvement in Spark

As a parallel computation framework, Spark can cache repeatedly resilient distribution datasets (RDDs) partitions in different nodes to speed up the process of computation. However, Spark does not have a good mechanism to select reasonable RDDs to cache their partitions in limited memory. In this paper, we propose a novel selection algorithm, by which Spark can automatically select the RDDs to cache their partitions in memory according to the number of use for RDDs. Our selection algorithm speeds up iterative computations. Nevertheless, when many new RDDs are chosen to cache their partitions in memory while limited memory has been full of them, the system will adopt the least recently used (LRU) replacement algorithm. However, the LRU algorithm only considers whether the RDDs partitions are recently used while ignoring other factors such as the computation cost and so on. We also put forward a novel replacement algorithm called weight replacement (WR) algorithm, which takes comprehensive consideration of the partitions computation cost, the number of use for partitions, and the sizes of the partitions. Experiment results show that with our selection algorithm, Spark calculates faster than without the algorithm, and we find that Spark with WR algorithm shows better performance. Copyright

Research on trust-based access control model in cloud computing

In this paper we propose a trust-based dynamic access control model for cloud computing environment inspired by the GTRBAC model, where the users can validate their legal identities and acquire their access control privileges for the resources according to the role information and the trust-degree in the lightweight certificates. The trust-degree in the certificate can be calculated by the direct trust-degree (DT) and recommendation trust-degree(RT), while the access permission for the resources can be decided by comparing the trust-degree with trust-degree threshold, in order to achieve effective control for cloud computing resource. Our theoretical analysis results show that this method can effectively provide dynamic and secure access control.