Jiadong Yang

A heuristic-based hybrid genetic-variable neighborhood search algorithm for task scheduling in heterogeneous multiprocessor system

Effective task scheduling, which is essential for achieving high performance in a heterogeneous multiprocessor system, remains a challenging problem despite extensive studies. In this article, a heuristic-based hybrid genetic-variable neighborhood search algorithm is proposed for the minimization of makespan in the heterogeneous multiprocessor scheduling problem. The proposed algorithm distinguishes itself from many existing genetic algorithm (GA) approaches in three aspects. First, it incorporates GA with the variable neighborhood search (VNS) algorithm, a local search metaheuristic, to exploit the intrinsic structure of the solutions for guiding the exploration process of GA. Second, two novel neighborhood structures are proposed, in which problem-specific knowledge concerned with load balancing and communication reduction is utilized respectively, to improve both the search quality and efficiency of VNS. Third, the proposed algorithm restricts the use of GA to evolve the task-processor mapping solutions, while taking advantage of an upward-ranking heuristic mostly used by traditional list scheduling approaches to determine the task sequence assignment in each processor. Empirical results on benchmark task graphs of several well-known parallel applications, which have been validated by the use of non-parametric statistical tests, show that the proposed algorithm significantly outperforms several related algorithms in terms of the schedule quality. Further experiments are carried out to reveal that the proposed algorithm is able to maintain high performance within a wide range of parameter settings.

A hybrid harmony search algorithm for the flexible job shop scheduling problem

In this paper, a novel hybrid harmony search (HHS) algorithm based on the integrated approach, is proposed for solving the flexible job shop scheduling problem (FJSP) with the criterion to minimize makespan. First of all, to make the harmony search (HS) algorithm adaptive to the FJSP, the converting techniques are developed to convert the continuous harmony vector to a kind of discrete two-vector code for the FJSP. Secondly, the harmony vector is mapped into a feasible active schedule through effectively decoding the transformed two-vector code, which could largely reduce the search space. Thirdly, a resultful initialization scheme combining heuristic and random strategies is introduced to make the initial harmony memory (HM) occur with certain quality and diversity. Furthermore, a local search procedure is embedded in the HS algorithm to enhance the local exploitation ability, whereas HS is employed to perform exploration by evolving harmony vectors in the HM. To speed up the local search process, the improved neighborhood structure based on common critical operations is presented in detail. Empirical results on various benchmark instances validate the effectiveness and efficiency of our proposed algorithm. Our work also indicates that a well designed HS-based method is a competitive alternative for addressing the FJSP.

Task scheduling using Bayesian optimization algorithm for heterogeneous computing environments

Abstract: Efficient task scheduling, as a crucial step to achieve high performance for multiprocessor platforms, remains one of the challenge problems despite of numerous studies. This paper presents a novel scheduling algorithm based on the Bayesian optimization algorithm (BOA) for heterogeneous computing environments. In the proposed algorithm, scheduling is divided into two phases. First, according to the task graph of multiprocessor scheduling problems, Bayesian networks are initialized and learned to capture the dependencies between different tasks. And the promising solutions assigning tasks to different processors are generated by sampling the Bayesian network. Second, the execution sequence of tasks on the same processor is set by the heuristic-based priority used in the list scheduling approach. The proposed algorithm is evaluated and compared with the related approaches by means of the empirical studies on random task graphs and benchmark applications. The experimental results show that the proposed algorithm is able to deliver more efficient schedules. Further experiments indicate that the proposed algorithm maintains almost the same performance with different parameter settings.

Effective search for Pittsburgh learning classifier systems via estimation of distribution algorithms

Pittsburgh-style learning classifier systems (LCSs), in which an entire candidate solution is represented as a set of variable number of rules, combine supervised learning with genetic algorithms (GAs) to evolve rule-based classification models. It has been shown that standard crossover operators in GAs do not guarantee an effective evolutionary search in many sophisticated problems that contain strong interactions between features. In this paper, we propose a Pittsburgh-style learning classifier system based on the Bayesian optimization algorithm with the aim of improving the effectiveness and efficiency of the rule structure exploration. In the proposed method, classifiers are generated and recombined at two levels. At the lower level, single rules contained in classifiers are produced by sampling Bayesian networks which characterize the global statistical information extracted from the current promising rules in the search space. At the higher level, classifiers are recombined by rule-wise uniform crossover operators to keep the semantics of rules in each classifier. Experimental studies on both artificial and real world binary classification problems show that the proposed method converges faster while achieving solutions with the same or even higher accuracy compared with the original Pittsburgh-style LCSs.

Effective search for genetic-based machine learning systems via estimation of distribution algorithms and embedded feature reduction techniques

Genetic-based machine learning (GBML) systems, which employ evolutionary algorithms (EAs) as search mechanisms, evolve rule-based classification models to represent target concepts. Compared to Michigan-style GBML, Pittsburgh-style GBML is expected to achieve more compact solutions. It has been shown that standard recombination operators in EAs do not assure an effective evolutionary search to solve sophisticated problems that contain strong interactions between features. On the other hand, when dealing with real-world classification tasks, irrelevant features not only complicate the problem but also incur unnecessary matchings in GBML systems, which increase the computational cost a lot. To handle the two problems mentioned above in an integrated manner, a new Pittsburgh-style GBML system is proposed. In the proposed method, classifiers are generated and recombined at two levels. At the high level, classifiers are recombined by rule-wise uniform crossover operators since each classifier consists of a variable-size rule set. At the low level, single rules contained in classifiers are reproduced via sampling Bayesian networks that characterize the global statistical information extracted from promising rules found so far. Furthermore, according to the statistical information in the rule population, an embedded approach is presented to detect and remove redundant features incrementally following the evolution of rule population. Results of empirical evaluation show that the proposed method outperforms the original Pittsburgh-style GBML system in terms of classification accuracy while reducing the computational cost. Furthermore, the proposed method is also competitive to other non-evolutionary, highly used machine learning methods. With respect to the performance of feature reduction, the proposed embedded approach is able to deliver solutions with higher classification accuracy when removing the same number of features as other feature reduction techniques do.

Task Scheduling for Heterogeneous Computing Based on Bayesian Optimization Algorithm

Efficient task scheduling, as a crucial step to achieve high performance for multiprocessor platform, remains one of the challenge problems despite of numrous studies. This paper presents a novel scheduling algorithm based on Bayesian optimization algorithm (BOA) for heterogeneous computing environment. In the proposed algorithm, BOA constructs and updates Bayesian network according to the task graph of scheduling problems to find the optimal solution assigning tasks to different processors, and the execution sequence of tasks on the same processor is set by the heuristic used in the list scheduling approach. The proposed algorithm is sufficiently evaluated and compared with the related approaches by means of the empirical studies on benchmark applications. The experimental results confirm that the proposed algorithm is able to deliver more efficient schedules. Further experiments also indicate that the proposed algorithm maintains almost the same performance with different parameter settings.

Effective structure learning for EDA via L1-regularizedbayesian networks

The Bayesian optimization algorithm (BOA) uses Bayesian networks to explore the dependencies between decision variables of an optimization problem in pursuit of both faster speed of convergence and better solution quality. In this paper, a novel method that learns the structure of Bayesian networks for BOA is proposed. The proposed method, called L1BOA, uses L1-regularized regression to find the candidate parents of each variable, which leads to a sparse but nearly optimized network structure. The proposed method improves the efficiency of the structure learning in BOA due to the reduction and automated control of network complexity introduced with L1-regularized learning. Experimental studies on different types of benchmark problems are carried out, which show that L1BOA outperforms the standard BOA when no a-priori knowledge about the problem structure is available, and nearly achieves the best performance of BOA that applies explicit complexity controls.

A heuristic-based hybrid genetic algorithm for heterogeneous multiprocessor scheduling

Effective task scheduling, which is essential for achieving high performance of parallel processing, remains challenging despite of extensive studies. In this paper, a heuristic-based hybrid Genetic Algorithm (GA) is proposed for solving the heterogeneous multiprocessor scheduling problem. The proposed algorithm extends traditional GA-based approaches in three aspects. First, it incorporates GA with Variable Neighborhood Search (VNS), a local search metaheuristic, to enhance the balance between global exploration and local exploitation of search space. Second, two novel neighborhood structures, in which problem-specific knowledge concerned with load balancing and communication reduction is utilized, are proposed to improve both the search quality and efficiency of VNS. Third, the use of GA is restricted to map tasks to processors while an upward-ranking heuristic is introduced to determine the task sequence assignment in each processor. Simulation results indicate that our proposed algorithm consistently outperforms several state-of-art scheduling algorithms in terms of the schedule quality while maintaining high performance within a wide range of parameter settings. Further experiments are carried out to validate the effectiveness of the hybridized VNS.

Effective Structure Learning for Estimation of Distribution Algorithms via L1-Regularized Bayesian Networks

Estimation of distribution algorithms (EDAs), as an extension of genetic algorithms, samples new solutions from the probabilistic model, which characterizes the distribution of promising solutions in the search space at each generation. This paper introduces and evaluates a novel estimation of a distribution algorithm, called L1-regularized Bayesian optimization algorithm, L1BOA. In L1BOA, Bayesian networks as probabilistic models are learned in two steps. First, candidate parents of each variable in Bayesian networks are detected by means of L1-regularized logistic regression, with the aim of leading a sparse but nearly optimized network structure. Second, the greedy search, which is restricted to the candidate parent-child pairs, is deployed to identify the final structure. Compared with the Bayesian optimization algorithm (BOA), L1BOA improves the efficiency of structure learning due to the reduction and automated control of network complexity introduced with L1-regularized learning. Experimental studies on different types of benchmark problems show that L1BOA not only outperforms BOA when no prior knowledge about problem structure is available, but also achieves and even exceeds the best performance of BOA that applies explicit controls on network complexity. Furthermore, Bayesian networks built by L1BOA and BOA during evolution are analysed and compared, which demonstrates that L1BOA is able to build simpler, yet more accurate probabilistic models.

Comparison of Parallelization Strategies for Min-Sum Decoding of Irregular LDPC Codes

Low-Density Parity-Check (LDPC) codes are powerful error correcting codes. LDPC decoders have been implemented as efficient error correction codes on dedicated VLSI hardware architectures in recent years. This paper describes two strategies to parallelize min-sum decoding of irregular LDPC codes. The first implements min-sum LDPC decoders on multicore platforms using OpenMP, while the other uses the Compute Unified Device Architecture (CUDA) to parallelize LDPC decoding on Graphics Processing Units (GPUs). Empirical studies on data with various scales show that the performance of these decoding processes is improved by these parallel strategies and the GPUs provide more efficient, fast implementation decoder.